He has confirmed that the pair have parted ways, after Louise's brother allegedly caught her boyfriend with another woman.
"Their relationship was so hard to graft. I think he did have this kind of control over her. I think that it’s the best for everyone that they’re not together anymore," he added.
But Spencer didn't look too bothered about the split as he headed out with co-star Hugo Taylor and his former 'I'm A Celebrity...' campmate Ashley Roberts on Monday night.
The trio headed to London's Groucho before continuing the night at private members club Soho House.
Spencer has also come under fire from Francis Boulle, who was the one who broke it to Louise that Spencer had done the dirty on her.
He told Star magazine his former friend has "some growing up to do".
"Obviously everyone has their own flaws to work on, so I think he's got some growing up to do in some areas. You know, he thinks he can get away with things when he shouldn't. I just think people need to learn," he said.
Knowing what Spencer is like, we doubt it is long before he hits back...