More than 5,000 foreign criminals who should have been deported remain in the UK, the government has been warned.

A report published on Thursday by John Vine, the independent chief inspector of the UK Border Agency (UKBA), found that there had been a "sharp rise" in the number of foreign national prisoners whom the Agency had not deported.

"These cases cannot be ignored and represent a growing cost to the taxpayer," he said. "More must be done to actively manage these cases."

The study found that 3,775 former foreign national prisoners had been released from custody, as there was no prospect of them being deported and were living in the community.

More than 1,600 foreign national prisoners remained in detention under immigration powers, having completed their sentence.

And 12 people are known to be missing after being released by mistake.

Between 2007 and 2010, a total of 20,360 foreign national prisoners were deported from the UK. In 2010, 5,235 foreign national prisoners were deported.

Vine said the UKBA needed to reduce the number of deportation decisions that were overturned on appeal because they fell foul of the Human Rights Act.

Between February 2010 and January 2011, 425 foreign national prisoners won their appeal against the Agency’s decision to deport them

Vine warned against a "cycle of appeals, at considerable cost to the taxpayer even though the outcome will eventually be that the foreign national prisoner is entitled to remain in the UK".