Take Ukip member David Hadley for example.
And he’s not alone.
On Tuesday evening even the Leave.EU campaign group was trying to get in on the action with this little video.
Starting off with a couple of soundbites of Brexit scepticism, it goes on to list the medals won by Team GB so far.
Among them is cyclist Callum Skinner who won gold in the men’s team sprint and silver in the individual sprint (which Leave.EU managed to get wrong).
Skinner’s response? Thanks but no thanks.
His response went down a storm, even with fellow Olympians.
Of course, it may actually have more to do with extra funding and the insanely hard work put in by athletes, coaches and all those whose roles support Team GB.
UK Sport, the government agency that allocates Exchequer and National Lottery money to Olympic and Paralympic sport, believes the success is no fluke.
UK Sport’s director of performance Simon Timson said:
“This is not happening by chance: this is success by design. It’s a result of 18 years of consistent, coherent, targeted National Lottery investment and the partnership we have forged with the British Olympic Association.”