I've been meaning to do this trip for 4 years. An abandoned castle in the middle of the Hudson?! Whaaa! Unfortunately, I hadn't realized that everyone else felt the same way. And as the managers of the estate, NYS Parks had tour guides on site doing hourly tours to hordes of tourists they brought over on a big boat. Sigh. The castle was cordoned off because it is falling apart and a brick might fall on your head if you get too close. The tour guides were reading off a script and it wasn't even a good script. And the tourists created a tourist trap feel. Since we paddled, we started our tour late and were promptly told to get off the island by "senior staff" after 40 minutes. Ugh. Whatever. Basically, save your money and skip this tour.
But getting there by kayak, that's a whole other thing. You're paddling upriver, against wind and currents. The Hudson flows both ways, but you gotta time it and set your course. We missed our timing and course, so it was an almost 3 hour struggle to paddle to the island. And it was cold. Natalie started shivering. But, if you got what it takes, just seeing the island and ruins from the water, and the views of the fall colors from the water vs. on a hike!!! This trip is epic if you're only going for the paddle. Consider the "tour" break time. Bring your favorite brew and just enjoy the waves on the opposite side of the island where tourist disembark. Private beach, friends, and then a real easy float back to Cold Spring. Word.