The glamour model announced earlier this week she and her boyfriend - who met at the Oscars in February last year - had "mutually" decided to end their relationship but would remain friends.
But Penna is reported to have told an Argentinian magazine: "I left her simply because she is crazy. Katie's brain is not wired right, or the wires don't connect properly.
"I don't know what's wrong with her. I was thinking it over recently and one day decided to pack my bags and leave.
"There was nothing else I could do. She was driving me insane," he told Caras magazine.
Despite an earlier brief split, Penna added there was no going back, reportedly saying: "Katie is totally crazy."
In a statement on her website this week, Price said the split was amicable: "We will always feel deep affection for each other, but sometimes relationships just run their course ... we've mutually decided to call it a day, but we really wish each other nothing but the very best and will remain best of friends."
She recently denied reports she sent flirty messages to X Factor contestant Dan Ferrari-Lane from the group District 3.