Police said Officer Z.B. Starritt recognized the man in the video and took out a felony larceny warrant on Christopher Columbus Hargrove, 48. Hargrove was arrested on Cleveland Street after a brief foot chase.
Officers were able to recover the stolen pump.
Keep up to date with what's going on in your area with the ABC11 News App.
5-year-old Greyson Corwin has Type 1 diabetes and relies on his insulin pump to get through the day.
After learning of the arrest, Josh Corwin, Greyson's dad, said he is glad the police took the time to make an arrest.
"I just feel like there's a lot of negativity with the (Durham) police department," he told ABC11 via telephone Sunday evening. "I'm a new resident of Durham and I want to give a shout out to the police department. They worked really hard and I'm appreciative."
Corwin says the police department plans to return the pump Sunday evening.
A security camera at the house caught the suspect in the act Wednesday evening. The family said they bought the security system after someone trespassed on their property in August.
Read more about the incident here.
Watch DeJuan Hoggard's original report in the video player below.