Ed Balls has insisted he is not "hankering" for a return to Parliament, adding: "The right thing to say is never say never."
The former shadow chancellor said a comeback is "unlikely" as he has no plans to stand in a by-election, while a public service role would appeal to him.
He also said he does not believe he is missing the House of Commons, noting he was pleased to have served as an MP for 10 years before his defeat to Conservative Andrea Jenkyns at May's general election.
On the chances of Yvette Cooper - who he married in 1998 - becoming the next Labour leader, Mr Balls said he was unsure if she would win despite her credentials.
He added he had "dropped the ball on the parenting" at Lord's while watching the second Ashes Test by not bringing sun cream for the couple's son Joe.
Speaking to Jonathan Agnew on BBC Radio's Test Match Special ahead of England's 405-run defeat by Australia, Mr Balls said of his political future: "The right thing to say is never say never because who knows what's going to happen.
"But the reality is I think it's quite unlikely because I'm now out, I'm not going to fight a by-election. It'd have to be quite unusual circumstances for me to want to come back in 2020.
"I'd like to be involved in public service. I'd like to do something good in my life. I've got 20 more years but there's other ways to do that than being elected, and I spent 10 years with the privilege of being elected and 20 years overall since I first went in to work in government.
"So, it's a different chapter and it'll probably be in a different realm. But that's okay. I don't know what."
Mr Balls said it was right for him to "get out of the way" and "make sure the home is sorted out and look after the kids" while Ms Cooper fought for the Labour leadership and continued as an MP.
He said: "It's her time. She's brilliant. I don't know if she's going to win but she's very, very good and as a Labour Party member I'll vote for her.
"The right thing for me to do is to let her get on with it. She's her own person, she'll decide how to do it and so in our lives it's a different chapter for us - for her and for me."
Asked how his son was getting on at the cricket, Mr Balls said during the lunch interval: "I think fine. It's quite sunny out there.
"We've been enjoying the sunshine. I think if Yvette was here she would have told me I should have brought the sun cream, so I'm going to have to go and buy some sunscreen for the afternoon."
He added: "I've dropped the ball on the parenting this morning but I'll be okay in the afternoon so that's fine.
"Anyway, I think he enjoyed his chocolate muffin."