It happened on 26 Yonkers Avenue around 4 p.m.; police evacuated the people inside the building.
Police said they found a sophisticated marijuana growing operation inside.
An initial call came in about a gas leak, but the fire department and utility company quickly discovered there was not a gas leak.
That's when drug enforcement officers and K-9 teams responded to the scene.
The growing operation was located inside two of the building's apartments. One of the apartments was housing 80 marijuana plants.
The suspects had allegedly tapped into the power grid illegally and brought in other chemicals that help in the development and the growing of the marijuana plants.
"We then as a precaution evacuated the building, and as we were evacuating the building we then found a second grow house on the fourth floor," said Chief Charles Gardner, Yonkers Police Department. "So, we have two apartments in this building that are being used as grow houses and we are currently in the process of removing the evidence at this time."
Officials propped open the windows in the building to air it out to dissipate the fumes.
Similar circumstances last year in the Bronx resulted in an explosion that killed New York City Fire Battalion Chief Michael Fahy.
Police are working to determine who was renting the apartments. So far, there are no arrests.