They then reinforced the fortifications and succeeded in defending the fort until the Dutch, commanded by Francois Caron, captured it in January 1644. The original bastions were demolished during the fort's siege by the Dutch cannons. However, in 1672, the Dutch later rebuilt it not on the usual square pattern, but on a pentagonal one, although it had only four bulwarks, the fifth was never built. The fort was situated between a lagoon and a sea inlet on a small strip of land. It was encircled by a dry moat, and through a drawbridge the door was accessed. It was occupied without resistance by the British in February 1796. The British authorities decided in the early 1800s to demolish the fort and construct in its location a prison built from the fort's rocks.

All that remains today is a part of the eastern rampart with mounds on its northern and southern ends, where there were earlier bastions, and a recessed arched gateway. There is a granite slab above the gateway, inscribed on the date 1678, surmounted by a highly decorative gable. The Prison Department still uses the site as a prison.