The Tholsel was once a gate tower in the town walls, still widely seen around Carlingford. This building was later used as the town jail.