Three years ago, he threatened to actually close some schools like Durham's Hillside High if test scores did not improve.
"You're not safe, unless those scores go up," he said.
That year, North Carolina's Department of Public Instruction started a program known as High School Turnaround to raise test scores at struggling schools.
Three years later, test scores have gone up in 59 of the 66 schools the state targeted.
"High School Turnaround has been very successful," offered Dr. Patricia Ashley with the North Carolina's Department of Public Instruction - who's the state's lead administrator for the program.
But there's another side of the story. While test score are up, graduation rates in many of the schools have gone on the other direction.
"This idea of High School Turnaround is not working," said Terry Stoops with the conservative think tank the John Locke Foundation.
Of the 66 low-performing high schools in the Turnaround Program, 32 of them - almost half - have actually seen graduation rates go down. How could this be? It turns out the program has not focused on graduating students. The focus has been on test scores.
Some schools have seen success in both areas. E.E Smith in Cumberland County has seen an increase in both test scores and graduation rates.
But others have seen a big drop.
Three years ago, Manning said he was encouraged that new principals were coming to Durham's Southern and Hillside highs.
"As long as they let the principals run the schools, without interference, I think they've got a shot," he said.
Hillside's test scores have gone up marginally. But, graduation rates after three years of High School Turnaround have fallen from 68 percent to 52 percent.
The drop of almost 16 percent was the steepest among the 66 high schools in the state program.
The numbers for High School Turnaround were presented to state lawmakers earlier this year, and some had concerns.
"If they don't come away with a diploma, what do they really gain from it?" asked Rep. Bryan Holloway, (R) Rockingham Co.
Dr. Ashley says graduation rates are very important.
"Our ultimate goal is to see every student successfully graduate," she said.
But, she says graduation rates are not a reflection of the program's effectiveness.
"Changing the way high schools have operated for years and years and years is not something that happens overnight for every school," she explained.
But after three years of High School Turnaround, others say the state is not moving fast enough to improve student achievement.
"I think we're right back where we started. I don't think there's been any progress made in curbing our dropout problem or increasing our graduation rates," said Stoops.
If parents log onto Hillside High School's website, they may read that under the heading of "The Real Story." It says Hillside last year met expected growth and was no longer in danger of being low-performing.
There's no mention of falling graduation rates.
"When you are a school, you want your website to reflect you positively," said Stoops. "Administrators at the school level and at the district level want to make it appear that their schools are making progress in order to keep their jobs."
The state does show most Turnaround high schools improving graduation rates when they allow students to graduate in five years.
"We need to look at both. We need to see how many graduated in four years, and how many graduated in five years," said Ashley.
What does Judge Manning think of students taking five years to graduate high school instead of the traditional four? The judge would not comment for this story.