Deputies responded around 12:55 p.m. along the Neuse River Greenway Trail and Loch Raven Parkway.
The body was found by a Raleigh resident who told ABC11 he was in the area to clean up trash when he mistook the body for a rock until he got closer.
"I was shaking at first. I never seen it before," said Kevin Ortega. "I come to this river a lot, and I hear the stories of drowning. But seeing it right in front of you, it's crazy."
Ortega said he called 911 and alerted authorities of the discovery. He said dispatchers asked if the body had any signs of life.
ALSO SEE | 14-month-old left unattended in bathtub drowns, stepfather charged with murder
"No, I think the body's fully done. I don't think they could come back," he told dispatchers on the phone.
The sheriff's office said the man was pronounced dead at the scene. The Office of the Chief Medical Examiner will determine the cause of death.
The alleged drowning comes less than a month after a 12-year-old girl died after a reported drowning in the river and two days after a man was pulled from a pond in Raleigh off Millbrook Road after going under and did not resurface.
A man also recently died at Lake Royale in Franklin County.
There are multiple warning signs posted at the entrance of Raleigh Beach alerting swimmers of the potential dangers of the river and its currents.
"If you guys wanna continue swimming here, swim at your own risk," Ortega said.
The case remains under active investigation.
Download the ABC11 News app