They say that the Internal Affairs Unit met with officers over the weekend at the Southeast District Substation on Cross Link Road, emptied their lockers, and took their guns and badges.
Mayor Charles Meeker wouldn't comment specifically on the allegations Tuesday, but said he's shocked by the investigation.
"It certainly is the worst thing I've heard about in the police force," Meeker said.
The mayor said he first learned about the investigation over the weekend in a message from the city manager.
"I was shocked that anything like this could occur," Meeker said. "Obviously, it's a very serious matter and the police force will take care of what needs to be done."
Police Chief Harry Dolan told ABC11 Tuesday that the investigation was launched in late 2009, after an officer in the department filed a complaint with Internal Affairs.
In a statement Chief Dolan said, "that complaint concerned the possibility of improper conduct on the part of certain RPD police officers."
The statement went on to say that a comprehensive investigation found, "that administrative violations may have occurred and that they stemmed from voluntary interactions between a small number of officers" and individuals outside of the department.
Sources tell ABC11 the department used GPS units as well as cameras during the investigation and determined that the officers were having sex with a known prostitute.
Sources also say one of those officers is Sgt. Rick Armstrong, who represents the Raleigh Police Protective Association.
RPD says it is near the end of its investigation and Mayor Meeker says he believes this was an isolated incident.
"We have good officers," Meeker said. "Every now and then something like this happens, unfortunately, and an action has to be taken about it."
The Raleigh Police Department has not identified the officers involved or specifically said what will happen to them.
The Wake County District Attorney says criminal charges are not likely to be filed.