I captured the reverse angle of this view, from the steps immediately behind Covell Cross, last year.

Lancaster's oldest town house was built in ~1625 on the site of a mediaeval hall, though the current Grade I Listed appearence dates from reconstruction immediately after the death of its most notable occupant, Thomas Covell (1569-1639). The witchfinder ('Witch Baiter') and Keeper of the Castle (i.e. chief gaoler) for 48 years, including for the Pendle Witches' trial of 1612, Covell was also Lancaster's mayor six times and coroner for 46 years.
As the name suggests, the house was later used by visiting Assize Court judges presiding over trials at the castle between 1776 (or 1826, according to another source) and 1975.
The lens has compressed the apparent distance between the house and the castle behind it, but not much; Covell and the judges wouldn't have had far to stroll to work.

Since 1978, the building has been a museum specialising in childhood (including an Edwardian schoolroom) and a remarkable collection of Gillow furniture, some of which was made only a couple of dozen metres away.