Wellingborough Prison will close by the end of this year with the loss of almost 600 prison places, Kenneth Clarke said today.
The Justice Secretary said closing the category C jail in Northamptonshire will save £10 million per year.
The prison population in England and Wales was 86,652 last Friday, with enough capacity for a further 3,500 inmates, the Prison Service said.
Along with the annual saving, closing the jail will also avoid costs of up to £50 million which would have been necessary to maintain the longer term viability of the prison, the Prison Service said.
It is expected staff will move to other jails and compulsory redundancies are not expected.
Mr Clarke said: "The public has the right to expect continuing improvement in the quality and efficiency of public services, without compromising public safety.
Clarke announced the prison's closure, which will see 600 cells lost
"Closing outdated and expensive prisons is an important step in our strategy to deliver a fit-for-purpose, modern custodial estate that can provide high quality, cost-effective and secure regimes that protect the public and reform prisoners.
"Closing this one prison alone will save millions of pounds for the taxpayer."
Two new prisons - the G4S-run Oakwood jail near Wolverhampton and the Serco-run Thameside prison in south-east London - opened earlier this year, creating about 2,500 extra places.
Juliet Lyon, director of the Prison Reform Trust, said: "Today's announcement of the closure of Wellingborough Prison signals a slowing down in the inexorable growth of the prison population over the past two decades.
"Each prison place costs nearly £40,000 a year and half of all prisoners reoffend within a year of release.
"Before we sink even more taxpayers' money into large private jails, it's time to invest in effective measures to cut crime that give a better return for the public purse."