Police responded to a home in the 1600 block of Stanford Court just before 10:45 a.m.
Investigators said Killian Perez got the loaded handgun from on top of a refrigerator and shot himself.
Police said Killian's father, 32-year-old Jeffery David Perez, was home alone watching the 4-year-old and two twin babies while the mom was in Durham visiting a family member at the hospital.
When officers arrived, police said Perez was frantically trying to save his son's life.
"He was understandably very distraught and when we arrived on scene he had attempted to administer aid to the child. He was in shock, he had all the symptoms of being in shock and understandably upset," said Fayetteville Police Sgt. Steven Bates.
Emergency crews tried to save the child, but he was pronounced dead at the home.
Detectives arrested Perez and charged him with involuntary manslaughter and failure to secure a firearm. He was taken to the Cumberland County Detention Center but has been released on his own recognizance.
Perez is a soldier in the United States Army, stationed at Fort Bragg.
Some of his fellow soldiers came to comfort and support Killian's grieving mom, who learned of her son's death once coming home from her trip.
A neighbor who would see the 4-year-old around is heartbroken.
"It hurts, it hurts, it makes me feel bad. I feel sorry for them, because they lost someone that they really love," said Everett Lyles.
This is the second fatal accidental shooting among children in Fayetteville since October. Last month, a 2-year-old girl found a gun under a couch in her family's living room. Her father was arrested and charged with involuntary manslaughter. He could face up to 10 years in prison if convicted.
Police say parents should have and use safes in their homes so kids do not confuse a gun for a toy.
"Our children sometimes just don't know the difference and that's kind of what happened in this case," said Sgt. Bates.