More than 21,000 British passports were lost or stolen abroad last year, costing tourists hundreds of pounds just to get home, according to the Foreign Office.
Consulates around the world had to issue more than 30,000 emergency travel documents for Britons stuck overseas.
A new passport is £72.50, and an emergency document is £100, meaning holidaymakers will have spent around £4.5 million in replacing their lost items and getting back to the UK.
James Freedman, 51, who was appointed as Britain's first fraud ambassador, said con-men are "always evolving the tricks" they use to target holidaymakers.
He added: "Only carry what you really need and keep cash and other valuables in a secure pocket or bag. Above all, trust your instincts and be aware of anyone invading your personal space."
The Foreign Office advise making photocopies of passports for identification, locking it in a hotel safe, and ensuring it is not visible when carrying it.