The 71-year-old former financier was transfered Tuesday from a corrections facility in Atlanta to Butner - about 30 miles north of Raleigh.
He is expected to undergo a physical and psychological evaluation at Butner before he learns where he'll serve his sentence. It's not clear if Butner is where Madoff will stay.
According to ABC News, concerns were reportedly raised about the "stress" Madoff may have suffered before and after he recieved his sentence.
Madoff was sentenced in June to 150 years in prison for a multibillion-dollar fraud that burned thousands of investors. He decided not to appeal the sentence.
Madoff reportedly confided to people close to him that he viewed going to prison as a new adventure akin to the time he served as a second lieutenant in the army reserves at Fort Bragg, North Carolina in the 1960s.
Prosecutors said the massive Ponzi scheme run by Madoff since at least the early 1990s demolished the life savings of thousands of people, wrecked charities and shook confidence in the U.S. financial system.
The actual loss so far has been put at $13.2 billion, but the judge who sentenced Madoff said that was a conservative estimate and noted that even Madoff told his sons in December it was a $50 billion fraud.
Before Madoff became a symbol of Wall Street greed, he earned a reputation as a trusted money manager with a Midas touch. Even as the market fluctuated, clients of his secretive investment advisory business — from Florida retirees to celebrities such as Steven Spielberg, actor Kevin Bacon and Hall of Fame pitcher Sandy Koufax — for decades enjoyed steady double-digit returns.
But late last year, Madoff made a dramatic confession: Authorities say he pulled his sons aside and told them it was "all just one big lie."
Madoff pleaded guilty in March to securities fraud and other charges, saying he was "deeply sorry and ashamed." He insisted that he acted alone, describing a separate wholesale stock-trading firm run by his sons and brother as honest and legitimate.
Aside from an accountant accused of cooking Madoff's books, no one else has been criminally charged. But the family, including his wife, Ruth Madoff, and brokerage firms who recruited investors have come under intense scrutiny by the FBI, regulators and a court-appointed trustee overseeing the liquidation of Madoff's assets.
It's not clear if his wife will visit him in North Carolina and spend some of the $2.5 million cash prosecutors let her keep. Just last week, she was evicted from the couple's luxury penthouse apartment in New York. The $7 million pad will be sold as part of a restitution plan for victims.
The trustee and prosecutors have sought to go after assets to compensate thousands of victims who have filed claims against Madoff. How much is available to pay them remains unknown, though it's expected to be only a fraction of the astronomical losses associated with the fraud.
Some of his victims live in the Heart of Carolina.
When Eyewitness News asked what they thought of Madoff starting his sentence in North Carolina, one man said, "I wonder if he'll let me visit."
There are other well-known prisoners serving time in Butner like white-collar criminals from Adelphia Communications, a Colombo crime family boss and the man known as the Blind Sheik behind terrorist attacks in New York.
Other notables evaluated there were John Hinckley Jr. who shot President Ronald Reagan and fallen evangelist Jim Baker.
