The music mogul was due to make an appearance on the UK show, summoning the judges - Gary Barlow, Nicole Scherzinger, Sharon Osbourne and Louis Walsh - aboard his yacht in the South of France to reveal which categories they will be mentoring in the new series.
Simon Cowell aboard his yacht earlier this year
However, following the claims he had got his friend's wife Lauren Silverman pregnant, he has cancelled the plans at the last minute, according to The Mirror.
The paper reports that the judges were due to fly out on Thursday, but will now instead be told their categories by Simon over the phone, as they were last year.
ITV bosses are said to be fuming at the changes, despite the fact Simon's phone calls will still be filmed.
A TV insider told the paper: “His problems and drama off screen are now starting to impact on his TV shows. He even called the UK judges personally to apologise for messing them around.
“ITV bosses are furious at the last-minute change. Simon’s return was going to be a big coup for them, but now that he’s just doing exactly the same as he did last year, it’s no big draw for the viewers. It’s really disappointing.”
'X Factor' judges Louis, Sharon, Gary and Nicole
In a change to the usual format, the judges are being told their categories before bootcamp, which begins filming at Wembley Arena at the weekend.
Simon jetted off to Europe for a month-long jaunt last week, following a week of revelations about his private life.
Lauren Silverman's mother has also spoken out in defence of her daughter, following accusations of 'gold-digging' behaviour.
Simon Cowell and Lauren Silverman
She told The Sun: "Lauren is not a gold-digger. She came from a very comfortable family and never lacked for anything.
"Her and Simon are very fond of each other. She tells me great things about him and has always spoken very highly of him. I trust Simon to do the right thing by my daughter," she said.