Bulldozers began pulling down the home at 610 North Buchanan Street in the Trinity Park neighborhood Monday morning.
As recently as May, Duke said it planned to sell the property - just as it has done with several other homes in the area.
The university has successfully renovated several homes known as "party houses" and then sold them to owner occupants instead of landlords, who typically rent properties near Duke's East Campus.
Duke would not say Monday exactly what it plans to do with the lot where the home sat - other than to say something would be constructed. When asked when that would happen, officials would only say "soon."
It has been four years since the house was occupied. In 2006, stripper Crystal Mangum accused members of the lacrosse team of rape during a party that was held at the property.
Three players were indicted, but the allegations were later found to be false and all charges were dropped against the players. There are still pending civil suits stemming from the case.
Duke university had to get permission from every attorney linked to the Duke lacrosse case, specifically those handling the civil trial, in order to tear down the house.
Ironically, the house was bulldozed the same day Mangum was in court.
She returned to court Monday, fighting for custody of her children after she was charged with attempted murder, arson, and child abuse after a domestic dispute with her boyfriend.
According to an arrest warrant, the 33-year-old scratched, punched and threw objects at Milton Walker before setting his clothes on fire in a bathtub. Arresting officers said Mangum threatened to stab Walker while they were at her home. They went there after one of her children called 911.
Mangum spent three months behind bars before Hammond and Hammond bail bond service posted her $100,000 bail in May free of charge. Mangum's bail originally was set at $1 million before being lowered.
Mangum told reporters at a news conference June 30 she didn't attack Walker and didn't burn his clothes. She said she believes her involvement in the Duke lacrosse case influenced what happened to her in February.
Mangum is no longer on house arrest and now allowed to live with her sick mother through electronic monitoring. She's also allowed supervised visitation with her three children.
Her goal, her attorney says, is to find a job, take care of her family and prepare for her upcoming trial. But she'll have to do it with a new attorney, because Mangum's attorney asked to be removed from her case.
A judge agreed and so did Magnum.