The source said Olsen is optimistic he can wait until after the season for any surgery.
Olsen refractured his right foot in the first quarter of Sunday's 16-8 season-opening win at home over the Dallas Cowboys.
The team said Tuesday there are no plans for surgery at this time, and Olsen will be evaluated on a monthly basis.
Olsen first broke his foot nearly a year ago, in Week 2 against the Buffalo Bills. He had surgery for that fracture and missed nine games.
On Sunday, Olsen returned to the sideline in the second half on crutches and in a walking boot.
Olsen tweeted a statement Tuesday afternoon, saying in part, "To say I am disappointed in this development would be an understatement. I believe my greatest career achievement has been my ability to play every game for over a decade. Unfortunately, the last two seasons have not continued that trend.
"With that being said, we are optimistic this is not how my 2018 season will end."
Also Tuesday, the Panthers announced that offensive tackle Daryl Williamshad injured his right knee during Sunday's contest when a Cowboys player fell on him. Williams left the field in a golf cart with less than 10 minutes remaining in the game. Early in training camp, Williams suffered a torn MCL and dislocated his right patella.
Surgery is recommended for Williams, and the Panthers are expected to place him on injured reserve, a league source told ESPN. Amini Silatolu, who replaced Williams on Sunday, is expected to get the first shot at the job. Recently acquired Corey Robinson also could figure into the equation.
Olsen, 33, in April signed a two-year extension through the 2020 season. He was a late addition to the injury report on Saturday with a stiff back, but he was able to start on Sunday.
Olsen, the first tight end in NFL history to have three consecutive 1,000-yard receiving seasons (2014-16), had two catches for 33 yards before leaving the game.
Rookie tight end Ian Thomas will be first up to replace Olsen as the receiving tight end. Chris Manhertz is the only other tight end on the roster, so look for Carolina to add another player at that position.
In an interview with ESPN during training camp, Olsen said he couldn't imagine there being a more complete tight end to come out of the 2018 draft class than Thomas.
"Ian has all the traits to be a true NFL tight end," Olsen said. "He's strong enough. He can engage at the line of scrimmage. He's smooth. He's faster than you think he is. He catches the ball well. So I think Ian has a chance to have all the traits to be a complete guy.
"There's a handful of them throughout the league that are asked to play every down. Ian is a guy that has those traits early."
Panthers coach Ron Rivera said Monday that Thomas was "pretty well prepared" to step into Olsen's role.
"There are some things, obviously, that he still has to learn, and he has a long way to go in terms of having the natural feel that Greg has and having that rapport Greg has," Rivera said. "He's been good with his blocking. He's been good with his route running. He's been good with his receiving. But he's got to get better as a route runner, a blocker and a receiver. He's a young guy. There's a lot to his game he's got to learn."
ESPN's David Newton contributed to this report.
Could injury speed up Olson's retirement plans?
Mike Golic Jr. and Trey Wingo examine Panthers TE Greg Olsen's foot injury and what it means for his future in the NFL.