A local resident told CNN he used to worship with a couple of the suspects in Raleigh and was "not surprised they were arrested."
The man claimed his concerns about the extremists at the Raleigh mosque were ignored.
But the Islamic Center of Raleigh said they take security very seriously.
"The IAR does have an open relationship with the FBI," said Imran Aukhil with Islamic Center. "We were notified of a person's behavior, violent threatening action that we considered to be dangerous and we did report them to the FBI," Aukhil said.
The men they were concerned about were two of seven charged with trying to carry out worldwide terror.
Daniel Boyd of Willow Springs is the accused ringleader.
Aukhil said they don't tolerate certain types of behavior, but take things on a case by case basis and work with local law enforcement openly.
"We make sure that we are in constant communication with them," Aukhil said. "We answer any questions that they have as necessary. We provide them with any information they request."
Some of the men accused once worshipped at the Islamic Center.
Aukhil emphasizes the men are innocent until proven guilty, but said the arrests have called negative attention to their faith.
"It definitely makes things difficult, but we look at the bright side," Aukhil said. "It gives us an opportunity to speak up more with the media and bring more attention to Muslims and their struggles as a minority."
The suspects will appear in federal court on Tuesday, where a judge will decide whether they will be given bonds.
