Margiotta tells ABC11 Eyewitness News that the only group that considers the remarks racist is the NAACP.
"I said the comments in defense of a black speaker who was being jeered by a nearly all white crowd," he said. "I've tried to put the statement behind me. It was clearly not the best statement to make. I have no plans to resign."
The NAACP president filed a complaint with the accreditation organization that works with school systems saying the comment has racial implications.
"The racial insensitivity exhibited by Mr. Margiotta underscores the lack of consideration for the interests, needs, and concerns of Blacks and other racial and ethnic minorities in North Carolina," said Benjamin Todd Jealous, President and CEO of the NAACP. "We support the North Carolina NAACP in their call for justice and sensitivity in Wake County, and believe the resignation of Mr. Margiotta as Chairman is a necessary step in that direction."
Margiotta's comments come in the midst of a battle over proposed changes to Wake County school district busing policy.
"I object to the comments certainly," member of board minority Anne McLaurin said. "I think that the comments were offensive. I think an apology is appropriate."
Chris Malone, a member of board majority, says he does believe people on both sides have the right to express themselves, but he says he hopes everyone can keep their emotions in check during future debates.
"We've all in our lives have said things," he said. "I really don't have anything more to say than that."
Director of the North Carolina Office of the Council on Accreditation and School Improvement Donna James received the letter from the NAACP Tuesday afternoon and says she will be reviewing it with several of her colleagues to determine what response is needed.
She says they have no authority over the selection or removal of board members or district leadership.
"Until such time that we are able to review the letter from the NAACP, it is premature to speculate on the next steps," James said.