Pencelli Castle was initially built around 1080AD by Bernard de Newnarch as a motte and bailey structure to defend that part of the Usk valley. Over the following centuries it changed hands several times through conflict. Not being built on top of a hill it was hard to defend, especially after gunpowder became widely used, and it fell into disuse in the 14th century. The canal follows part of the original moat. The existing Elizabeth manor house is thought to be part of the chapel of St Leonard which stood within the castle walls.