Photo Charles Taylor

A fortified manor house surrounded by a stone curtain wall with a gatehouse was erected on the site in the second half of the 12th century. The present three storey pele tower with a vaulted undercroft was built about the year 1410 and was used as a residence until 1680. In 1775 much of the castle was demolished to provide building stone, including the bridge over the River Derwent. The tower has been in ruins since then but now a local Friends of Ayton Castle group are working with Scarborough Borough Council who own the site to repair and secure the future of the pele tower.