Unfortunately, xenophobia is part of the Brexit debate. To some it seems that the EU hasn't brought them much in the way of palpable advantages, but instead is directly responsible for filling-up their neighbourhoods with foreigners, mainly from eastern European countries.
However, never in history has there been a period of time without migration. No nation ever was an island, to paraphrase John Donne. Not even the British Isles for that matter. Yes, EU immigration to the UK has risen in the past years. But as the below graphic based on data by the Office for National Statistics shows, the influx might have peaked already.
Check out other charts on migration issues by Statista here
In addition, once you get to know the individual, the actual Polish, the Bulgarian, the Romanian, you will very likely get along just fine. You might even take a liking to what they bring to the table, literally. Curry wasn't always a staple food in the UK and late-night Kebab fixes were unheard of. Console yourself with the fact that what seems foreign today might turn out to become heritage tomorrow.
Foreigners for sure bring change. But things, and people, need to be on the move for history to happen. As in many other cases, going into retreat is not an option and won't benefit anybody in the long-run. You're not being overrun by conquerors swords in hands, as we're not talking about Romans, Vikings or Normans here. To say it with Franklin D. Roosevelt, there's nothing to fear but fear itself.
Nota bene: Most EU immigrants to the UK actually come from Western Europe, as the graphic below illustrates.
Check out other charts on migration issues by Statista here