Requirements for builders to include a quota of affordable homes in new housing developments should be waived, a government-commissioned report has recommended.
The move would allow developers to create more properties for let to boost the private rental market.
But critics warned the change would come at the expense of people desperately struggling to get on the property ladder.
It comes days after a right-wing think tank suggested councils should sell off expensive houses to fund new building developments, leading to claims the poor could be pushed out of wealthy areas
The review of the private rented sector by Sir Adrian Montague, chairman of private equity firm 3i, said councils should consider using powers to waive the requirement to build homes available for those on lower incomes to buy in order to increase the number of properties built to let.
It said: "Whilst desirability of affordable housing should not be ruled out, it should be weighed against the benefits already built into market rent developments, in the context of an accurate assessment of the economics of building homes to rent.
"In many cases, it will be appropriate for authorities to waive affordable housing requirements in relation to schemes for private rental, or to the private rental component of larger schemes also including an owner-occupier component."
The review also recommends setting up a task force to encourage build-to-let investment and the release of unused publicly-owned land for development.
Sir Adrian Montague: "It's clear we must encourage investment in the private rented sector, which has gone through a period of rapid growth and is now relied upon by millions of people.
"My review shows that the rental housing sector offers potential investment opportunities of interest to institutional investors.
"But real momentum has been inhibited by constraints affecting the supply of stock, the treatment of rented housing schemes under the planning system and the need to create confidence among investors.
"The recommendations in today's report are designed to challenge this, and remove the barriers that prevent the kind of investment that our private rented sector needs."
Housing minister Grant Shapps said the report offered a "blueprint" to expand the sector.
He said: "We're determined to encourage greater investment in the build-to-let market and boost the country's private rented sector, which plays an integral role in meeting the nation's housing needs and aspirations.
"In the past it's often been seen as the Cinderella of the housing market, but when over three million people rely on this sector for their home, this is clearly no longer the case.
"A major part of this is to attract and encourage new players to the market, while at the same time avoiding the excessive regulation that would force up rents and reduce choice for tenants.
"Sir Adrian Montague's findings offer both a blueprint for achieving this goal, and for setting the standards of accommodation that people should expect. I will be considering his recommendations very carefully."
But the report's recommendations were challenged by industry groups, councils, charities and Labour.
David Orr, chief executive of National Housing Federation, said: "While we agree that there needs to be more private market rented housing, this should not be at the expense of affordable homes.
"The one doesn't replace the other as both are needed for a fully functional housing market.
"Therefore we don't think private rental developments should normally be exempt from affordable housing contributions."
Shelter's chief executive Campbell Robb said the report "misses a trick" by "offering nothing for the millions of people already in the sector, paying sky-high rents and living under constant threat of eviction or further rent rises".
He added: "Despite the increase in the number of people renting, the long term aspiration for many remains to be a homeowner.
"While more rented housing could help ease pressure on this over-heated market, it's vital that this does not come at the expense of building affordable homes for people across the country already struggling to get on the property ladder in their local area.
"Young people who are working hard and saving towards a secure and affordable home need to know that the Government is on their side."
Mike Jones, chairman of the Local Government Association's environment and housing board, said: "Any strategy to boost the number of new rental homes should not come at the expense of new affordable housing, and councils will in consultation with their residents always seek to ensure a suitable balance between the need for private rental property and new affordable homes."
Shadow housing minister Jack Dromey said: "While many of the measures recommended in this report are sensible, for instance on the use of public land, on attracting investment and on standards in the private rented sector, we are not convinced that the answer to the crisis created by this Government is to further water down affordable housing requirements that councils place on developers.
"As rents hit a record high in July, many families are already paying the price of the Tory-led Government's failure to build enough affordable homes - the Government should be acting to address this problem, not looking for ways to water down existing legislation which could make the problem worse.
"Huge cuts to Government investment in housing, a lack of liquidity in the finance markets, the failure of banks to lend to homebuyers, and stagnating demand are the real hurdles to viability, not the cost of providing much needed affordable housing."