I felt disjointed, detached from time. My eyes were fixed on the ornate, Scottish castle in front of me. Its sandstone red towers were half-standing, victims of a huge gun powder blast. I swore I could smell smoke in the heavy, humid air.

But then I saw the flash of an iPhone, and my mind reluctantly snapped back to reality. I was not in 16th Century Scotland after a great battle, but on a small island in the Hudson River. It is home to a replica Scottish castle built by Frank Bannerman. Like me, he wanted to escape modern life in New York City – and to house his expanding stockpile of arms.

I fought to ignore the cues of the modern world around me and let my imagination slip back into the fairy tale Bannerman created one hundred years ago. I took photo after photo, trying to capture the melancholy I felt standing in front of something that was once so imposing, but was now crippled and propped up by steel crutches. 

The derelict castle was always just a passing curiosity from the Metro North train. But this time, I got off at the Beacon station, took a deep breath to fill my lungs with the clean air, and relaxed on a 20-minute ride aboard the island’s only ferry, The Estuary Steward. The Hudson’s tides make it tricky to navigate, and standing on the bow, I found myself searching for the goblins that the early Dutch sailors warned enchanted the waters.

By the time I arrived on the Bannerman Castle Island dock, the stress of modern life had retreated from my shoulders. My tour of the island was included as part of the Bannerman Castle Island Farm Fresh Dinner, a September fundraiser to preserve the castle complex, which has been beaten down by explosions, fires, and unforgiving New York winters. My $135 ticket didn’t just buy me a farm-to-table meal and the good feeling from having donated. I enjoy connecting my love of food with my love of history, so the dinner was the perfect way for me to relish the medieval world Bannerman created. 

I felt minuscule compared to the huge turret-topped castle. I willed my eyes to recreate the missing towers and walls so I could understand the awe and respect it inspired for its creator, Frank Bannerman, who immigrated to Brooklyn from Scotland as a toddler in 1851. He amassed a fortune buying surplus military equipment and reselling it to civilians, governments, and militias. When he hit the jackpot at the end of the Spanish-American war, Bannerman outgrew his Manhattan warehouse at 501 Broadway, now a Capital One bank in trendy SoHo. He bought the island and copied the fortresses from his homeland, including towers, a drawbridge, and a moat.

I was frustrated that I couldn’t get closer than a few hundred yards and peak into the rooms now open to the sky. It seemed odd that a huge stone structure barely a century old could be in ruins, but there’s an inherent weakness in all military storehouses: the contents. The Arsenal was devastated by an explosion of its own gun powder in 1920. It was injured again in 1969 when peacenik hippies accidentally started a fire, and then in 2010 when a brutal Catskill winter collapsed two walls. 

As I continued along the mossy path, despite my best intention, I grew contemplative, something I rarely have the time or calm mind to do in the city. The island reminded me about the fleeting nature of power and beauty. Behind me was a collapsing arms warehouse and in front of me were shadows of what were once the elaborate flower gardens that Mrs. Bannerman cultivated to bring feminine warmth and colorful vibrancy to her husband’s creations.

My eyebrows raised when I reached the Bannerman Residence – I was shocked by how close the family lived to a warehouse full of gunpowder and ammunition. The home was smaller than I expected, but when I turned around, the panoramic view of the Hudson demanded to be photographed. I felt commanding, just as Bannerman intended. 

The gray stones and ornate red turrets of the Residence were also in a state of disrepair. Although the 1920 Arsenal explosion damaged the Residence a little, it also fell victim to hubris: Bannerman was a shrewd businessman, but not an astute architect. He designed the buildings himself, perhaps not realizing that being Scottish didn’t make him an architect.

The annual fundraiser is usually held outside, but the impending rain made me one of the first people to have dinner inside since the 1940s. As I stepped across the worn threshold, I could immediately see why it has taken that long: Inside were missing walls and reconstructed plywood floors and ceilings. Yet I still felt like I should have taken off my shoes and not touched anything. The grandeur remained, including faint lines of religious quotations adorning the walls, perhaps Bannerman’s atonement for how he amassed his fortune. 

Our farm-to-table dinner was courtesy of the Chefs’Consortium of the Greater Hudson Valley. Despite being cooked under tiny tents,the multiple courses were vibrantly fresh with complex flavors. Long communal tables were decorated with orange and yellow mums, and simple white tablecloths disguised the plastic lawn furniture beneath. The flickering candles and sparkling white string lights cast a glow like the fire that once burned in the now empty hearth. The local acoustic guitarist melodically plucked his strings like a medieval mistrial, and I closed my eyes and pretended I was at a feudal banquet. 

If you want to go: The island is open to visitors from May through October. The Bannerman Castle Trust’s website has various ways to experience this beautiful and quirky bit of New York history. Although you could just take a ferry ride and self-guided tour, I highly recommend visitors spend more time on the island to really unwind and enjoy this little escape from frantic modern life. If food doesn’t excite you, then pick one of the many other special events that fit with your interests, such as a kayak excursion or one of the various concerts and performances held by local artists. The Metro North train arrives next to the Beacon ferry dock and is about one-and-half hours from Grand Central.