So are members of the Charlotte City Council.
Richardson is being investigated by the NFL for workplace misconduct that, according to a Sports Illustrated article published on Sunday, included settlement with at least four people in the organization for his inappropriate behavior.
He announced on Sunday he will put the team up for sale after this season.
Richardson, 81, made a deal in 2013, when the city agreed to pay $75 million in stadium upgrades, to keep the team in Charlotte through the 2018 season.
So the new ownership would have the option in 2019 to move the team that has played all but its first year -- the inaugural 1995 season was played in Clemson, South Carolina, while Bank of America Stadium was being completed -- in Charlotte to another city.
Lyles and members of the City Council are committed to make sure that doesn't happen, the mayor said.
"The City of Charlotte values its long-running relationship with the Panthers after more than 22 seasons of NFL football,'' Lyles, a season-ticket holder, said in a statement. "The Panthers are part of Charlotte's fabric. We've celebrated victories and anguished over defeats. We understand transitions are inevitable, and we look forward to working with current and future ownership.''
Councilman James Mitchell, the chairman of the new economic development committee charged with working with the Panthers, was on the council in 2002 when the NBA's Charlotte Hornets moved to New Orleans. He remembers the grief it brought the city and said losing the Panthers would be just as devastating, if not more so.
"The question will become, 'What's wrong with Charlotte?''' Mitchell said. "I'm committed to being the City Council that kept the Panthers in Charlotte.
"So the role I'd like our committee to play is to make sure we work with the new owner and make sure he or she knows how valuable the Panthers are to the city of Charlotte and then say, 'What we else can we do? What is your checklist to make sure you're committed to the Charlotte community?'''
Councilman Ed Diggs said it would be a "great loss'' for Charlotte if the Panthers were to go elsewhere.
"The Carolinas, generally, have been real enthusiastic about our team,'' he said. "We've had a couple of Super Bowl appearances. It helps define us as a city.''
Richardson was taken out of the day-to-day running of the organization on Monday when the Panthers named Tina Becker as the new chief operating officer.
But Richardson still controls selling the team that he purchased in 1993 for $206 million. Forbes valued the team in September at $2.3 billion.
Among those who have expressed interest in the Panthers are Sean "Diddy'' Combs, whom Forbes estimates to be worth $820 million. Charlotte native Stephen Curry, the two-time NBA MVP with the Golden State Warriors, and former San Francisco quarterback Colin Kaepernick also weighed in on the team when Combs tweeted his interest.
Hall of Famer Eddie Debartolo Jr., the former owner of the 49ers, also has expressed interest.
Mitchell said somebody sent him a text that said Oprah Winfrey should buy the team but added that he thinks the serious suitors haven't come out publicly yet.
Richardson in his announcement to sell said he won't begin looking at potential new owners until after the Panthers finish the regular season and postseason. They are 10-4 and can lock up a playoff spot with a win at home on Sunday against Tampa Bay.
"We are at a standstill until after the season is over,'' Mitchell said. "We really don't know until the end of the season, after we win the Super Bowl for him and send him out in style, what's going to happen.''
Mitchell declined to speculate on whether the allegations of sexual harassment and the use of a racial slur to a former team scout are accurate. He said Richardson did not exhibit that kind of behavior since they began meeting quarterly in 2009.
Diggs said it would "make me sad'' if the allegations are proved true.
"The guy has been such a pillar in Charlotte,'' he said. "You think of all the good things people have said he's done for them, and the sort of caring nature he has, the paternalistic approach he's had to running his organization, it would be too bad to see all of that erased by any impropriety that may have occurred.''
Rivera: 'We are still here to play football'
Panthers head coach Ron Rivera addresses the allegations made against owner Jerry Richardson and reveals when he found out the team was going to be sold.