There’s no word yet on whether another duo will take over, or if Simon Cowell and the ITV bosses will opt for another solo presenter.
Will Rylan, Emma or Sinitta (!) land the role?
Rochelle and Marvin Humes are also in the running for the job, according to William Hill, while Sinitta and Jedward are outsiders.
But who deserves the job? Check out all the odds and cast your vote in the poll below…
Olly and Caroline separately confirmed that they won't be returning to 'X Factor' on Monday.
Olly's statement came first as he told The Sun: “This was an incredibly hard decision to make and one I didn’t take lightly as I’ve really enjoyed co-hosting The X Factor.
“It’s been one big adventure and I loved every minute of it. But my heart belongs to music and I want to concentrate 100 per cent on making my next album and getting back to performing.”
Caroline then shared her decision, stating: "I have had a brilliant time working on 'The X Factor' over the last few years, and hosting the main show was just fantastic, I made some amazing friends."