SAN DIEGO -- San Diegans could soon be paying for a football stadium without a team if the Chargers decide to relocate.

Qualcomm Stadium has been a part of San Diego for more than 40 years, and San Diegans are still paying it off. Now, even if the Chargers leave town, 10News learned taxpayers would still be stuck with a massive bill.

"It's just another nail in the coffin of San Diego's finances," said Richard Rider, chairman of the San Diego Tax Fighters.

Rider said many San Diegans don’t realize the city still owes more than $54 million for Qualcomm Stadium and it is a debt that needs to be paid regardless if the team leaves.

"Our finances for the city are dropping. Our obligations are growing. Our services are being cut left and right, and here comes another problem," said Rider.

The problem could get even more expensive if the Chargers are able to get a proposed downtown stadium location next to Petco Park. Rider said that move would mean San Diegans would be paying off Qualcomm Stadium and paying for much of the $800 million it would cost for the new stadium.

"Only a fool would do that … or the City Council," said Rider.

10News contacted city officials and they said a downtown stadium would make Qualcomm Stadium obsolete. They said that site "would likely be redeveloped in some fashion to generate revenue for the city …"

Rider, however, fears it could be too little too late.

"This is just one more straw on the camel's back, and this one might be enough to break it," said Rider.

If the Chargers break their lease at Qualcomm Stadium they would be stuck with a fee which would go toward city debt, but that fee decreases each year, officials said.