A new watchdog with the power to close failing colleges will be created as ministers promised "tough action" to improve standards in further education.
The new FE commissioner will be able to recommend stripping colleges of their freedoms over staff and spending.
In some cases the commissioner, who will report directly to ministers, could advise replacing the governing body or the dissolution of the college.
Skills minister Matthew Hancock said it was "wholly unacceptable" that nearly 1.5 million learners were not receiving teaching rated as good by Ofsted inspectors.
Ofsted currently judges two-thirds of colleges to be good or outstanding but 4% are judged to be inadequate.
Hancock said: "While our further education sector offers world class provision, in the global race we must and will tackle poor performance.
"It is wholly unacceptable that nearly 1.5 million learners are not receiving teaching that is rated as good.
"Where colleges are failing learners we will be knocking on their doors and take swift and effective action. It is a dereliction of duty to let failing colleges teach young people. We will not fail in our duty to act.
"All providers should meet tough standards of rigour and responsiveness. Through these reforms we will be able to intervene without hesitation where they fall short."
The commissioner will intervene if a college has an inadequate rating, misses minimum standards of performance or has financial problems.
Hancock also announced a £214 million funding package for 47 colleges across England to "ensure they have first rate facilities".
Shadow further education minister Gordon Marsden said: "This shows ministers belatedly scrabbling to catch up with growing concerns about quality in skills provision and their undermining of FE colleges through funding cuts, axing face to face careers guidance and abolishing EMA.
"Labour has led calls for safeguarding rigour in skills provision across all areas, including colleges, and has urged the Government to act.
"We want to better safeguard quality in the skills system, including through a new gold standard qualification in our proposal for a TechBacc to help the forgotten 50% who don't go to university.
"When Parliament returns MPs will want to question Ministers in detail on their new proposals, which should have been announced while Parliament was sitting.
"Serious questions remain on the role of the proposed new FE Commissioner, including whether this will clash with the remit and powers of Ofsted, and on how accountable to Parliament the Commissioner will be."