Ray Sharpe was one of them.
"I was trying to determine if I was going to go Virginia to stay with a friend," he explained. "Fortunately, when it was downgraded to a Category 1 or 2, we decided to stay unlike a number of my neighbors around here and it was a wise move, otherwise I wouldn't be able to get back and check on the damage on the house."
With only a downed tree, a broken window, and a jet ski washed up in his backyard, Sharpe said he was pleasantly surprised.
"I thought I was the luckiest guy in the world. This house stood up very well."
And that was the case around much of the island.
There was very little major structural damage left behind despite the 90 mph wind gusts and torrential downpours.
"All in all, compared to what's going on around the state, we're real lucky," said Mayor Bill Blair. "Since we took the eye you would never know that we were the entry point of the storm."
Drone11 captured some of the damage on the island; several docks will need to be repaired.
But overall Sharpe said he's now excited to get back home.
"Very glad to come home and have power and have air conditioning because we've been without power and air conditioning," he said. "Somehow even though the eye crossed here, we got very very lucky."
It's still unclear when the general public will be allowed back on the island.