The music mogul was due to make an appearance on the UK show, summoning the judges - Gary Barlow, Nicole Scherzinger, Sharon Osbourne and Louis Walsh - aboard his yacht in the South of France to reveal which categories they will be mentoring in the new series.
Open Image Modal
Simon Cowell aboard his yacht earlier this year
However, following the claims he had got his friend's wife Lauren Silverman pregnant, he has cancelled the plans at the last minute, according to The Mirror.
The paper reports that the judges were due to fly out on Thursday, but will now instead be told their categories by Simon over the phone, as they were last year.
ITV bosses are said to be fuming at the changes, despite the fact Simon's phone calls will still be filmed.
A TV insider told the paper: “His problems and drama off screen are now starting to impact on his TV shows. He even called the UK judges personally to apologise for messing them around.
“ITV bosses are furious at the last-minute change. Simon’s return was going to be a big coup for them, but now that he’s just doing exactly the same as he did last year, it’s no big draw for the viewers. It’s really disappointing.”
Open Image Modal
'X Factor' judges Louis, Sharon, Gary and Nicole
In a change to the usual format, the judges are being told their categories before bootcamp, which begins filming at Wembley Arena at the weekend.
Lauren Silverman's mother has also spoken out in defence of her daughter, following accusations of 'gold-digging' behaviour.
Open Image Modal
Simon Cowell and Lauren Silverman
She told The Sun: "Lauren is not a gold-digger. She came from a very comfortable family and never lacked for anything.
"Her and Simon are very fond of each other. She tells me great things about him and has always spoken very highly of him. I trust Simon to do the right thing by my daughter," she said.
Open Image Modal
Lauren Silverman: 9 Facts In 90 Seconds
Lauren Silverman: 9 Facts In 90 Seconds(01 of09)
Open Image Modal
1. Nobody seems to know whether she's called Lauren or Laura. Or both. So let's get this one cleared up once and for all. It's definitely LAUREN. We think. (credit:Getty Images)
Lauren Silverman: 9 Facts In 90 Seconds(02 of09)
Open Image Modal
2. She's 36. That's 17 years younger than Simon Cowell so yes, he is indeed *adopts nana voice* old enough to be her father, tsk tsk etc, etc. (credit:WENN)
Lauren Silverman: 9 Facts In 90 Seconds(03 of09)
Open Image Modal
3. She is still technically married to one of Simon's bezzies, property developer Andrew Silverman. But according to Us magazine, the estranged couple have been 'unhappy for some time and a divorce is in the works'. Phew. (credit:Splash)
Lauren Silverman: 9 Facts In 90 Seconds(04 of09)
Open Image Modal
4. She's been friends with Mr. Cowell for some time having holidayed with him in St Barts in August last year (along with with hubby Andrew). She also joined Sinitta and Si on holiday in 2011. (credit:Splash)
Lauren Silverman: 9 Facts In 90 Seconds(05 of09)
Open Image Modal
5. She's a socialite. You know, like Paris Hilton. (credit:WENN)
Lauren Silverman: 9 Facts In 90 Seconds(06 of09)
Open Image Modal
6. She's a dead ringer for Simon's ex fianceé, Mezhgan Hussainy - so much so that even we couldn't initially tell them apart when they all holidayed together last year. Can you? (Mezhgan's in the centre of this pic, Lauren's on the right) (credit:Splash)
Lauren Silverman: 9 Facts In 90 Seconds(07 of09)
Open Image Modal
7. Just like Simon, she loves a ciggie, although we're hoping now she's preggers she's kicked the filthy habit. (credit:WENN)
Lauren Silverman: 9 Facts In 90 Seconds(08 of09)
Open Image Modal
8. She pretty much knew all there was to know about Simon Cowell before allegedly rubbing belly buttons with him. We know this because she was at the launch party for his *coughs* unauthorised biography, 'Sweet Revenge: The Intimate Life of Simon Cowell' in April 2012. (credit:Getty Images)
Lauren Silverman: 9 Facts In 90 Seconds(09 of09)
Open Image Modal
9. She's not to be confused with Laura Silverman (right) who is the actress sister of US comedian Sarah Silverman (left). (credit:Getty Images)