In an interview with the BBC's Nick Robinson the former MP for Morley and Outwood signaled there had been divisions at the very top of the party over its economic policy, but maintained he backed Miliband "100%".
"In the end he didn't persuade people he could be the prime minister but I didn't persuade people I could be the chancellor either," he admitted to Robinson on Friday.
Balls made the comments in his first TV interview since the election
Answering questions about his surprise ousting from the Commons, Mr Balls refused to be drawn on whether he would seek a seat in Westminster again, saying he was "not dashing back" but he would "never say never".
"Politics is a brutal business," he said. "It's tough.
"In a democracy the people decide and however tough it is for us individual politicians - especially if you’re fighting a marginal seat - you have to accept that you can be here today gone tomorrow and that is democracy.
"Although it was hard, I was a symbol of the vibrancy of our democracy."
On his wife Yvette Cooper, who is running for the Labour leadership, Mr Balls said she would have his backing.
"I’m going to be supporting and voting for Yvette, of course but I’m not going to play any part in her campaign."
Balls is married to Shadow Home Secretary Yvette Cooper
"I think she’s brilliant and people will have a chance to see what she is, what she stands for and what she can do in the coming weeks.
"I’ve got the opportunity at the moment to stand back while she’s busy to do more to help the rest of the family.
"In politics you have to take your calling when it comes and that’s what she’s doing."
Asked whether the British public would see Mr Balls on the upcoming series of 'Strictly Come Dancing', the former Education Secretary said: "The people who succeeded on Strictly tend to be half my age and have played international sport or been at stage school, so I’m not sure I’m quite equipped for it".