Despite previously seeming to be a dead cert to return, the former Pussycat Doll is battling to pick between the show and her music career, doubting that she is able to juggle both.
A £1m offer is said to be on the table to secure Nicole for a second series, but with her album due out in November, she will need to be available for promo and is worried she will not be able to fully commit to 'X Factor'.
A pal of the star told The Sun she had said: “I’m so torn right now. My heart wants me to come back and do the show as I had such a great time last year, but my album is coming out and I also need to focus on that.
“I don’t think I can make it work. By trying to juggle the two, one or both will suffer.”
She also cited fans reacting badly to former judge Kelly Rowland when she jetted back and forth to LA and missed a live show due to illness her preventing from flying back in 2011, with Nicole adding that she didn't want that to happen to her.
However, those close to the star are urging her to sign up for the show, according to the paper.
It news comes as Simon Cowell has promised to revamp 'every aspect of the show' following last year's falling viewing figures.
After already having revealed that the original judging room auditions were returning, he told the Radio Times: "The format is ten years old and any format that has lasted that long, you have to change the format because the inevitable result is if you don't [the ratings] will go down.
"Every aspect of the show [will be changed] in some particular way. It will look like a slightly different show to one you have seen before.
"My job is to try and make the show better than it was two or three years ago."