As part of the changes, Nicole Scherzinger is strongly rumoured not to have had her contract renewed, with speculation suggesting Louis and Sharon’s three-year deals would be be terminated early.
However, he has told The Sun he can only be sacked if he’s given a huge payout.
“My contract is watertight. I’ve checked it out with someone and have told Simon that,” he said. “He has asked me to come and see him in LA.
“I have spoken to Sharon. She’s said they can only sack us if they pay us.”
Louis continued: “I don’t think they’ve finalised what they’re going to do with the panel. We have three-year contracts. We both do.”
He also slammed the decision to axe the “brilliant” Nicole from the show, claiming she was the hardest-working and “always brought something new”.
Louis has been sacked from ‘The X Factor’ twice before - once in 2007, when he was briefly replaced by Brian Friedman, only to be brought back a few days into filming, and then again in 2015, when Nick Grimshaw took up his seat on the panel for a full series.
However, he returned the following year when Simon admitted he had made a mistake letting the long-standing judge go.
Ratings for the show have plummeted in recent years, with last year’s final becoming the least-watched since it began in 2004.
Auditions for the 15th series of ‘The X Factor’ are due to kick off in the summer, with the show expected to return to ITV in September.