It is "absurd" that thousands of prisoners remain in jail beyond their original terms despite the scrapping of a discredited sentencing initiative, Ken Clarke said.
The senior Tory abolished Imprisonment for Public Protection (IPP) indeterminate sentences as justice secretary in 2012 - calling them a "stain" on the justice system.
But many remain behind bars because of "ridiculous" rules requiring them to prove they do not pose a danger to the public, Mr Clarke told BBC Radio 4's Today as he renewed calls for their release.
Current Justice Secretary Michael Gove has ordered a review of the cases where criminals have served more than the original tariff imposed by the courts.
Mr Clarke spoke out after the programme highlighted the case of one such inmate.
James Ward was given an IPP with a minimum term of 10 months for arson after setting fire to his cell mattress towards the end of a year-long sentence.
But he is still locked up 10 years on with no release set.
His solicitor, Pippa Carruthers, told the BBC that subsequent fires, dirty protests, self-harm and episodes of barricading himself in are linked to his mental health.
"He becomes overwhelmed. He loses sight of what he needs to do to prove to a parole board that he is no longer a risk, and he acts destructively," she said.
Mr Ward's sister said they were certain he would eventually take his own life if he was kept inside.
In a series of letters written to Today, Mr Ward said prison was "not fit to accommodate people like me with mental health problems.
"It's made me worse. How can I change in a place like this? I wake up every morning scared of what the day may hold."
Mr Clarke told the programme: "It is quite absurd that there are people who might be there for the rest of their lives, in theory, who are serving a sentence which Parliament agreed to get rid of because it hadn't worked as anybody intended a few years ago.
"The trouble is this ridiculous burden on the Parole Board of saying they can only release people if it's proved to them that they're not really a danger to the public.
"No prisoner can prove that - you never know when people are going to lose their control, what's going to happen to them when they're released."
He went on: "You have a few thousand people still in our prisons with no idea when they are going to get out and a parole board that dare not let them out for fear of public attack if one of them does something serious when they said they were satisfied that they were safe. You can't be satisfied.
"So long as prisons are overcrowded slums, so long as you don't tackle mental health problems, you don't tackle drug abuse properly, you don't give people some basic education when they haven't got any and you don't prepare them for a job, you are actually toughening up some of these people and they are likely to be more of a risk."
IPPs - introduced by Labour in 2003 - were a symptom of a justice policy "determined for fear of the tabloid newspapers", he said.
It was originally estimated that 900 serious violent and sexual offenders would be subject to them but that number swelled to 6,000, some for relatively minor offences.
About 4,000 are still behind bars, 400 of whom have served more than five times the minimum term, the BBC said.
Announcing the review, Mr Gove said: "There are a significant number of IPP prisoners who are still in jail after having served their full tariff who need to be given hope that they can contribute positively to society in the future."