Simon Cowell: 'No 'X Factor' Judge Is Safe For Next Series'

After ratings fell on the last series of The X Factor, we knew that Simon Cowell was bound to be far from happy.

And it appears he is laying the blame firmly at the door of last year's judging panel, revealing they are all facing the chop - even pal Louis Walsh.

The music mogul has criticised Gary Barlow, Kelly Rowland, Tulisa Contostavlos and Louis for not "doing enough different things" with their contestants and rated their performance as a "could do better" eight out of ten.

Speaking to the Daily Mirror, Simon said: "We have got to look at everyone, the whole show. I don't know if I will keep the panel the same. I haven't decided yet.

"Judging can be tough, but if we can't find a star out of 120,000 applicants then someone isn't doing their job properly."

But it looks like some decisions could be made for him, as Simon thinks Gary Barlow is likely to vacate his seat of his own accord because of his commitments with Take That.

"I always thought that with Gary he would have the Take That commitment in 2012," he told The Sun.

"But there has to be a certain amount of unpredictability with this show, it is what keeps people interested."

It has also been rumoured Kelly Rowland has already quit, blaming the "circus" that surrounds the show and spats with her fellow judges as the reason for her departure.

A source revealed last week: "She enjoyed her time on the programme but was taken aback by the circus surrounding it."

So could all this mean the return of Mr Nasty to the X Factor panel?

Well, for now Simon is ruling nothing out, even though he is contracted to judge a second series of the US version of the show, telling the tabloid he would consider jetting back and forth to do both.

Would you like to see him return to X Factor now he's back on Britain's Got Talent?

X FACTOR: THE BIGGEST LOSERS