In his tweet, the president said that CIA Director Mike Pompeo will take Tillerson's place.
Mike Pompeo, Director of the CIA, will become our new Secretary of State. He will do a fantastic job! Thank you to Rex Tillerson for his service! Gina Haspel will become the new Director of the CIA, and the first woman so chosen. Congratulations to all!— Donald J. Trump (@realDonaldTrump) March 13, 2018
"The President wanted to make sure to have his new team in place in advance of the upcoming talks with North Korea and various ongoing trade negotiations," a senior White House official told ABC News.
Tillerson had just returned from a shortened trip to Africa hours before Trump's announcement.
The Associated Press says an aide to Tillerson told them that the president never explained to Tillerson the reason why he was fired, and Tillerson had wanted to stay in the job.
Under Secretary of State for Public Diplomacy and Public Affairs Steve Goldstein released the following statement to ABC News:
"The Secretary had every intention of staying because of the critical progress made in national security. He will miss his colleagues at the the Department of State and the foreign ministers he has worked with throughout the world.
"The Secretary did not speak to the President and is unaware of the reason, but he is grateful for the opportunity to serve, and still believes strongly that public service is a noble calling.
"We wish Secretary Designate Pompeo well."
Two officials familiar with the situation said that Tillerson had been fired by Trump on Friday, while in Africa. There were no obvious indications as Tillerson flew home early Tuesday from Nigeria that his departure was imminent, nor that it was his last trip abroad as top diplomat.
But Tillerson had cut short his trip by one night, telling reporters he had been sick in Africa and had barely slept two nights in a row because of urgent, middle-of-the-night matters, including the North Korea announcement.
"I felt like, look, I just need to get back," Tillerson said.
Dismissing Tillerson had been discussed at multiple levels for a long time, said a senior White House official, adding that the North Korea overture and invitation brought more urgency to the decision. Two officials said Trump wanted to have a new team in place ahead of an upcoming meeting with North Korean leader Kim Jong Un as well as trade talks.
The officials weren't authorized to speak publicly and demanded anonymity.
Tillerson also was known for his Russia connections and would be Trump's most concrete outreach yet to Russian President Vladimir Putin.
Trump's announcement came the day after the Republican-controlled House Intelligence committee announced it found no collusion between Trump's presidential campaign and Russia.
Tillerson was slated to retire from Exxon in March 2017 at age 65 under the company's mandatory retirement policy. Paid $27.3 million last year, Tillerson has accumulated roughly $160 million in Exxon stock along with $149 million of unvested stock options, according a proxy statement the company filed with the Securities and Exchange Commission
Pompeo is to be replaced at CIA by Gina Haspel, Pompeo's deputy at CIA. She would be the first woman in that role.
This is a breaking news story and will be updated.
ABC News and the Associated Press contributed to this report