The castle was erected in 1086 on rock one hundred foot above the river Teme. Stone walls were erected with four towers protected on two sides by the cliff and on the other two sides by a moat cut into the rock. The gatehouse was later enlarged into the tall tower that can be seen now. The castle was continuously extended until the end of the 16th Century but then fell into decay during the
1700's. There is an unusual early circular Norman church inside dating from around 1120.
Large sections of the 13th town wall remain as well as the Broad Gate, one of the original seven entrances to the town, although much of the wall is hidden by later building.