"He hit the inmate lottery as far as being designated to the best place that he could possibly hope for," federal sentencing attorney Alan Ellis said.
Ellis said he has clients in Butner and called the jail the "crown jewel of the federal prisons."
It does include a medical facility, with interns from nearby Duke and UNC.
Madoff is rumored to have cancer and at the age of 71, was likely assigned to Butner for potential medical problems.
Raleigh attorney Robert Nunley who also has clients at Butner said while it's not a penitentiary, "it's definitely no cake walk."
"When it comes to medium security, make no mistake about it, it's still a prison," Nunley said. "He still sleeps in a cell and will have people he shares that cell with more than likely."
Attorneys said Madoff would want to stick close to guards, considering some of the clients he swindled have connections and may have a hit on him.
Officials said eventually he'll be allowed to take classes and would be assigned a job either as a cook, a dishwasher or a landscaper.
Madoff will also be allowed to have visitors, but no conjugal visits are allowed.
ABC News has learned Madoff's wife Ruth may move to North Carolina so she'll be able to see him.