The presenter - who, in an ironic twist of fate, is set to take over hosting ‘The X Factor’ later this year - has told fans of what actually happened when she lost her job on the ITV2 spin-off show in 2014.
"They were really nice about it, but I knew I had to leave.”
The reigning ‘Strictly Come Dancing’ champion added: "Yes, I felt really upset, but every career has ups and downs. You have to have a thick skin. And sometimes if you want to have a little cry, you do it on your own, and you get rid of it.
"Then ‘Strictly’ came along and it was like, 'One door shuts, another one opens."'
She is set to reunite with Olly when the new series of ‘X Factor’ begins later this year, taking over hosting duties from Dermot O’Leary as part of a massive shake up of the show.
Sarah Jane took over from Caroline but has since been axed
Sarah Jane also fell victim to the axe, having hosted just one series of ‘The Xtra Factor’.
Caroline and Olly will reunite to host 'X Factor'
However, Caroline - who is currently fronting ITV2’s ‘Love Island’ - backed Robbie Williams to take one of the seats, telling The Mirror: “I’d love Robbie. He’d be the dream. He loves the show. I just don’t know if he would do it.
“He’s been brilliant at judges’ houses. I want Snoop Dogg! How good would that be?”