The decision came down to a 6-to-2 vote.
The council issued a statement about the decision, saying in part that, "Just as Raleigh has grown and changed, so have the skills needed to manage and grow the City."
"We are excited and look forward to the new possibilities and insight that a new city manager will bring us, and we are grateful for all the work Russell has done as well," the statement went on to read. "Russell has agreed to stay through July 1 and we appreciate his willingness to work with us on an orderly transition."
Allen also released a statement following the announcement.
"I have always strived to make this a better city and have loved doing so," Allen said. "Raleigh has a very engaged citizenry and I hope they feel that I have been respectful, accessible and responsive."
"Raleigh is one of the most successful cities in the country and is poised for even greater achievements," Allen went on to say. "As much as I will miss this job, I am thankful for the experiences and confident in the City's future."
Allen has held the position as Raleigh city manager for 12 years.
In 2009, he made headlines when the Raleigh City Council granted him a controversial $10,000 pay raise.
The 4-and-a-half percent raise garnered criticism in part because of the the recession.
Allen was already making $200,000 a year at the time of the increase. He is currently earning over $230,000. When Allen was originally hired in 2001, he received a salary of $140,000.