Most people want drastic curbs on the rights of people from India, Australia and Canada to vote in UK elections.
Currently, Commonwealth immigrants are entitled to register on the electoral roll as soon as they arrive in this country.
But a poll has found that 60% of people think they should have to wait until they qualify for British citizenship.
Last week, campaign group Migration Watch UK warned that by the next general election in 2015 there could be more than one million such voters.
Currently, citizens from those 54 states can register on the electoral roll upon arrival in the UK as long as they have an address.
Sir Andrew Green, Chairman of Migration Watch UK said "Public opinion is rightly very strong on this.
"The situation is both very unfair and undermines the value of citizenship. The Government's response so far has been pathetic.
"When Parliament has considered it in the past MPs had no idea of the scale that this issue had reached. Now that the scale has become known, this hang-over from the Empire must be tackled and soon."
The poll of 1,919 adults, which was carried out online on August 29 and 30, revealed that 30% were in favour of the right to vote, and 9% did not know. The percentages have been rounded.
Last week, Migration Watch UK claimed that more than half a million votes in the next election could be cast by Commonwealth nationals without British citizenship.
According to 2011 census data there are 960,000 Commonwealth citizens who do not have British citizenship living in England and Wales but who have the right to vote.
This total includes only those who, in 2011, were over the age of 15 and who therefore would be eligible to vote in 2015, the campaigners said.