Bronllys castle is a motte and bailey fortress standing south of the village, towards Talgarth. The castle was founded in or soon after 1144 when the district was granted to Walter de Clifford by Roger Fitzmiles, 2nd Earl of Hereford. Walter seems to have been responsible for building the round tower on the motte for in 1165 it caught fire and a stone tumbling from the battlements killed Earl Roger's last surviving brother Mahel de Hereford. In September 1233 Walter III de Clifford had a force of over 200 men here defending the castle against his father-in-law Llywelyn ab Iorwerth. The castle passed from the Cliffords to the Giffards and eventually the de Bohun Earls of Hereford.