The fire ignited shortly before 10 p.m. at 314 W. Jones St., where the Metropolitan apartment complex was still being built.
Stunned Raleigh residents and visitors poured out onto sidewalks and streets as the massive blaze filled the area with heavy smoke. The smoky haze is expected to linger for days.
"It was massive. It looked like the entire block was one fire," a downtown business owner and resident named Patrick told ABC11's Angelica Alvarez Thursday night.
Stay on top of breaking news with the ABC11 News App
The blaze forced the evacuation of nearby businesses and residential buildings, as well as road closures.
Eyewitnesses said it appeared the fire ignited on the second floor, though police and fire officials have not yet made such a determination.
LISTEN TO THE 911 CALLS
No one was living in the building, but there are numerous buildings nearby, including other residential apartment units.
A crane used at the construction site collapsed minutes after firefighters arrived.
By 1 a.m., the fire was under control with crews still spraying the burnt structure with water to extinguish any hot spots.
The fire was so large that the smoke was visible on First Alert Doppler XP.
The wood construction of the Metropolitan apartment complex made fighting the blaze challenging.
"They immediately tried to put water on it but as they approached they noticed it was a lot of heavy fire," Raleigh Fire Department Assistant Chief Brad Harvey said. "Especially under construction. The wood is not protected and the fire moves very rapidly. We're very fortunate tonight."
Jason, who lives directly across the street from the construction site, said he got a little concerned as he saw what began as a small fire quickly escalate.
He told ABC11's Joel Brown that he was at a downtown Raleigh establishment and tried to return to his apartment because he was worried about his dog.
"I tried to make my way back into the building and I couldn't get back in, the police and the fire department were evacuating people and they wouldn't let anyone else back inside," Jason said.
Former North Carolina Gov. Pat McCrory tweeted late Thursday night: "Sec. Kluttz and husband got out of building near #RaleighFire. God bless the firefighters and first responders. Praying for their safety!"
Clay and Hugo, a couple of Peace University students said they had to park off campus when they were returning from the gym because the streets were blocked off.
The two students said lots of friends and co-workers were calling and texting them to make sure they were OK, a scene repeated countless times in downtown Raleigh on Thursday evening.
Some people who lived south of the building were bringing in items from their balconies and watching out for hot embers blowing toward their homes.
Eyewitnesses told ABC11 it took only a few minutes for the fire to develop into a massive blaze that engulfed the building.
"I heard a bunch of sirens," one woman who lives nearby told ABC11's Joel Brown. "I looked around me and everything was orange."
Conner, another downtown Raleigh resident, told ABC11's Joel Brown the "flames were pretty intense and pretty hot, and the whole building went up in about 10 minutes."
A church on the corner of Salisbury and Hillsborough streets offered to shelter anyone displaced by the fire.
The cause of the fire is not yet known.