A square moat encloses a small island with a low motte at the centre, Haresfield Mount, close to the village church. It is believed to be the site of the manor house of Haresfield which was still standing in 1680.