The Lee County Sheriff's Office said 46-year-old Antwan Lopez Clemons showed up at a woman's home in the Lemon Springs area Monday morning and took her hostage.
The woman managed to get word to a family member, who called 911.
Deputies set up a perimeter and began negotiating with Clemons, who eventually released the woman and surrendered without incident.
That's when deputies noticed that Clemons was wearing a prison uniform.
He was supposed to be serving an active sentence at Butner Federal Prison but had walked away from the facility.
Download the ABC11 News app
Clemons was charged with first-degree kidnapping, breaking and/or entering to terrorize or injure, two counts of assault by pointing a gun, misdemeanor crime of domestic violence, communicating threats, and possession of a firearm by a convicted felon.
He is being held in the Lee County Jail without bond.