The controversial star was a judge when the music show first started in 2004, and returned to the panel last year after a six year break.
However, despite mentoring last year's winning contestant Sam Bailey, Sharon has now said she doesn't want to be a judge on the show when it returns to screens later this year.
She told The Sun: "I cannot go there... I can’t", despite sources this week claiming she would be happy to judge the show again, and Louis Walsh backing her return.
A show insider also said: "Last year was seen as Sharon’s last hurrah. She isn’t in consideration for this year’s show.”
It's believed that Sharon's commitments in America - including appearing on the panel of 'The Talk' - means she is too busy to return to the UK for 'X Factor'.
The question is: with Sharon now out of the picture who will be joining Simon and Cheryl on the panel?