The Aussie singer is said to be hurt after she was branded Simon's "toy" in a new biography, which also claims they shared several nights of passion while they filmed the ITV1 talent show.
"Dannii and Simon had a close relationship but he's shown his true colours by referring to her in this way."
Open Image Modal
Dannii also fears the new allegations, made by journalist Tom Bower in Sweet Revenge: The Intimate Life Of Simon Cowell, will hurt her career.
The insider added: "Simon's made her out as if she's some puppy that does what he says whenever he clicks his fingers. Anyone who knows Dannii knows she is not like that."
Simon, who was interviewed for the book, took to his Twitter page on Saturday to comment on the scandal, writing: "Just seen the TV ad for the unauthorised book on me. Yikes!"
But his spokesperson Max Clifford appeared to confirm the rumours on Sunday after revealing: "Simon was on his own at the time and she was on her own too.
"There was a natural chemistry between them. It's not something he wanted to come out now but it has - there is nothing unusual about it at all."