It turns out the gun was not a real firearm but a BB gun.
But it looks and feels so much like the real thing the department decided to warn parents to make sure their children aren't out in public with the guns.
"We want parents just to be very mindful of what they're giving their kids to play with," said Capt. Ann Stephens with the Apex Police Department.
She said that Wednesday, police got a call about two boys with handguns sitting on the hill next to the Walmart store.
Officers arrived on the scene and ordered the boys to put their hands up.
When they did, the gun tucked in one of the boy's pants was exposed and it looked real.
It was only on close inspection that officers discovered it was a BB gun.
ABC11 bought the same gun and showed it to parents.
Sabrina Wakefield said, "It feels real."
When asked whether it looked real, she replied, "It does."
She said she could see why parents are being warned, saying, "It's a safety hazard for kids to be carrying them around even just to play with them. You know, they're not toys."
Police fear the replica guns are a tragedy waiting to happen.
"We don't want anybody to get hurt because they've got a BB gun on them. But nowadays they're so lifelike, they're so realistic-looking, it's very difficult to tell in that split instant that an officer had to make a decision whether it's real or not," Stephens told ABC 11.
Parental discretion may not have mattered in the case of the 13-year old boy who had the gun.
Police said it was stolen from the Walmart store.