Asylum seekers must be more fairly dispersed around the country, MPs and council leaders from the areas with the highest concentrations have told the Home Office.
Politicians from some of the 10 councils in the country with the largest ratios of asylum seekers to residents said the situation, at a time of cuts to public services, was unacceptable.
Those top 10 were all in Northern England, Scotland or Wales, in places already struggling with unemployment deprivation.
It followed a policy started in 2000 to disperse asylum seekers away from London and the prosperous South East.
Outspoken Rochdale MP Simon Danczuk was furious that the Government was "dumping" asylum seekers in towns like his, causing a strain on schools and health services.
He said: "Rochdale is a cohesive community. We are quite diverse, but this is really upsetting the apple cart and it is creating tension.
"You have groups of asylum seekers, who through no fault of their own have nothing to do because they are not allowed to work, wandering around the town centre, not speaking English."
Dave Budd, the elected mayor of Middlesbrough, which is the only local authority to have surpassed the Government's guideline of 1:200 asylum seekers to residents, said it had "far exceeded its fair allocation".
The town, where one in every 173 residents was an asylum seeker living in dispersed accommodation, was at the centre of the red doors controversy last week.
It emerged some asylum seekers felt stigmatised because their accommodation provided by Jomast, sub-contracted by services giant G4S, had red doors.
That made some feel they were easily picked out by racists and vandals. As a result, it was agreed the front doors would be repainted a variety of colours.
Stuart Monk of Jomast, and Peter Neden and John Whitwam of G4S, will be quizzed about the controversy today by the Commons Home Affairs Committee.
Geraint Davies, Labour MP for Swansea West, which is sixth in the national list, was also unhappy with the way asylum seekers were dispersed.
He said the Government should not exploit Swansea's "warm welcome" by "over-burdening us and not providing the resources that are needed to support asylum seekers".
A spokesman for Newport City Council, eighth on the list also mentioned Cardiff, which is ninth, saying their multi-cultural societies meant asylum seekers were welcome, but that those diverse communities also had issues with deprivation.
The council was working with the Home Office to reduce the impact of having concentrations of asylum seekers in particular areas, he said.
The issue is a particularly sensitive one. A contractor in Cardiff providing accommodation for asylum seekers came under fire this week for giving some clients wristbands which they had to wear to get meals.
In Glasgow, third on the national list, the Scottish Refugee Council did not say the city was overburdened, but did call for a national review of the housing contract, saying it had "grave concerns" about how asylum seekers were housed.
It said Middlesbrough's red doors issue was an example of the Home Office's "poor oversight" of how private contractors house vulnerable people.
Councillor Mike Connolly, leader of Bury Council and lead member in Greater Manchester on asylum seekers, said they have made the point "very clearly to government that we consider the high concentration of asylum seekers - relative to other parts of the country - currently accommodated across our area to be unacceptable".
Greater Manchester covers Rochdale in second position, Bolton in fifth and Oldham in 10th on the national list.
The Home Office said it will work with councils that raise concerns about asylum dispersal.
A spokesman said: "The UK has a proud history of granting asylum to those who need our protection and we are committed to providing safe and secure accommodation while applications are considered.
"Asylum seekers are housed where there is appropriate accommodation available. Agreements between the Government and participating local authorities are voluntary and have been in place since 2000. We review these arrangements regularly and all asylum intake has to be approved by the local authority involved.
"We work closely with local authorities to ensure that the impact of asylum dispersals is considered and acted upon.
"We will work with any local authority that raises concerns about asylum dispersal."