(CNN) -- Famed cyclist Lance Armstrong is facing the loss of seven Tour de France titles and his fabled championship legacy after ending his fight against charges of illegal doping.
Armstrong's move prompted the U.S. Anti-Doping Agency early Friday to slap a lifetime ban on Armstrong and strip him of his wins since 1998. He announced late Thursday that he had decided not to contest what he called unfair charges of illegal doping, allegations he has consistently denied.
There's a question of whether the agency has authority in the case and whether international agencies might have to weigh in before Armstrong would face the prospect of losing his titles.
The International Cycling Union has opposed the anti-doping actions by claiming it has jurisdiction. That position has been recently backed by USA Cycling, the official cycling organization recognized by the U.S. Olympic Committee. The cycling union said Friday it won't comment until the anti-doping group issues "a reasoned decision" explaining its stance. Armstrong has said the cycling union and not the agency should conduct the arbitration in his case.
"If I thought for one moment that by participating in USADA's process I could confront these allegations in a fair setting and -- once and for all -- put these charges to rest, I would jump at the chance," Armstrong said in a statement released Thursday. "I refuse to participate in a process that is so one-sided and unfair."
The World Anti-Doping Agency's president, John Fahey, said on Australian radio he is disappointed that doping allegations made against Armstrong won't be tested because of his decision not to challenge them. He asserted that the allegations have credibility.
Asked whether the U.S. agency has the authority to strip Armstrong of his Tour de France titles, Fahey said, "Olympic medals and titles are for other agencies to decide, not WADA."
But the U.S. agency said the anti-doping rule violations Armstrong is being sanctioned for are defined as violations by the cycling union and other agencies, all of which have adopted the World Anti-Doping Code and the World Anti-Doping Agency's prohibited list.
The U.S. agency has accused Armstrong of using, possessing, trafficking and giving to others performance-enhancing drugs, as well as covering up doping violations. It alleges he took steroids throughout his career, saying it has testimony from former teammates to support the charges. It has refused to reveal who provided the evidence.
"Nobody wins when an athlete decides to cheat with dangerous performance-enhancing drugs, but clean athletes at every level expect those of us here on their behalf to pursue the truth to ensure the win-at-all-cost culture does not permanently overtake fair, honest competition" said U.S. Anti-Doping Agency CEO Travis Tygart.
"Any time we have overwhelming proof of doping, our mandate is to initiate the case through the process and see it to conclusion as was done in this case."
Armstrong has been an icon for his feats and celebrity, bringing more status to a sport wildly popular in some nations but lacking big-name recognition, big money and mass appeal in the United States.
He fought back from testicular cancer to win the Tour from 1999 to 2005. He raised millions via his Lance Armstrong Foundation to help cancer victims and survivors, an effort illustrated by trendy yellow "LiveSTRONG" wristbands that helped bring in the money.
The cyclist's one-time high-profile relationship with singer Sheryl Crow also kept him in the public eye.
But Armstrong has been dogged by doping allegations in recent years, with compatriot Floyd Landis -- who was stripped of his 2006 Tour de France title after failing a drug test -- making a series of claims last year.
Armstrong sued the U.S. Anti-Doping Agency to stop its investigation of him, arguing it did not have the right to prosecute him.
But a federal judge on Monday dismissed Armstrong's lawsuit after ruling the court did not have jurisdiction.
After that ruling, Armstrong said "enough is enough" in the face of his battle with the USADA and Tygart, whose investigation he describes as a farce. The U.S. agency is the official anti-doping agency for Olympic, Pan American and Paralympic sports in the United States.
"I have been dealing with claims that I cheated and had an unfair advantage in winning my seven Tours since 1999," Armstrong said. Over the past three years, I have been subjected to a two-year federal criminal investigation followed by Travis Tygart's unconstitutional witch hunt. The toll this has taken on my family, and my work for our foundation and on me leads me to where I am today -- finished with this nonsense."
Armstrong said he plans to help people "affected by cancer" and is "looking forward to an end to this pointless distraction."
"Today I turn the page. I will no longer address this issue, regardless of the circumstances," he said "I have a responsibility to all those who have stepped forward to devote their time and energy to the cancer cause. I will not stop fighting for that mission. Going forward, I am going to devote myself to raising my five beautiful (and energetic) kids, fighting cancer, and attempting to be the fittest 40-year old on the planet."
Armstrong came out fighting in May 2011, in the face of fresh allegations made on CBS News' "60 Minutes" show by another American, Tyler Hamilton.
In the CBS interview, Hamilton -- who retired in 2009 after twice testing positive himself and who, earlier this month, was stripped of his 2004 gold medal by the International Olympic Committee because of doping -- said he first saw Armstrong use EPO in 1999.
EPO, or erythropoietin, boosts the number of red blood cells, which carry oxygen to the muscles.
Armstrong has said in court documents that he has never had "a single positive test" in the 500 to 600 drug tests he's taken in his more than two decades in cycling.
"From the beginning, however, this investigation has not been about learning the truth or cleaning up cycling, but about punishing me at all costs. I am a retired cyclist, yet USADA has lodged charges over 17 years old despite its own 8-year limitation," Armstrong said in Thursday's statement.
"The only physical evidence here is the hundreds of controls I have passed with flying colors. I made myself available around the clock and around the world. In competition. Out of competition. Blood. Urine. Whatever they asked for I provided. What is the point of all this testing if, in the end, USADA will not stand by it?"
In February, Justice Department prosecutors said they closed a criminal investigation after reviewing allegations against Armstrong. They had called witnesses to a federal grand jury in Los Angeles, but apparently determined they lacked evidence to bring a charge that Armstrong used performance-enhancing drugs.
In a June letter to Armstrong, a copy of which was obtained by CNN, the USADA said it collected blood samples from Armstrong in 2009 and 2010 that were "fully consistent with blood manipulation including EPO use and/or blood transfusions."
After Armstrong filed his suit to stop the agency's investigation, the USADA said all of the evidence "would have been presented in an open legal proceeding for him to challenge. He chose not to do this knowing these sanctions would immediately be put into place."
U.S. District Judge Sam Sparks, in ruling Monday that the court did not have jurisdiction, acknowledged "the appearance of a conflict on the part of both organizations creates doubt the charges against Armstrong would receive fair consideration in either forum." But that doesn't mean federal courts should intervene, the judge said, adding that "these matters should be resolved internally, by the parties most affected."
"If these bodies wish to damage the image of their sport through bitter infighting, they will have to do so without the involvement of the United States courts," Sparks said.
Armstrong said he had hoped that a federal court would stop what he calls the agency's "charade."
"Although the court was sympathetic to my concerns and recognized the many improprieties and deficiencies in USADA's motives, its conduct, and its process, the court ultimately decided that it could not intervene."
Armstrong has support. Johan Bruyneel, his former team manager, said he's "disappointed" that the case reached a point where Armstrong is "no longer willing to participate in USADA's campaign against him"
"Lance has never withdrawn from a fair fight in his life, so his decision today underlines what an unjust process this has been," he said.
John Eustice, a two-time U.S. professional cycling champion, said the group is wrong to assert that it would strip Armstrong of honors such as the Tour de France titles. The tour owner would decide that, he said.
"It's a private enterprise and a private race," he said.
He said people who go into arbitration "have no chance to win."
Armstrong's move isn't an admission of guilt. It's a reaction to an agency's "unmitigated power with complete overreach" that is cowing other sports agencies.
"They feel that they have this world authority to do whatever they want," he said. "He's Moby Dick. They finally got Moby Dick."