Bridgen was suspended by Tory bosses after making the comment in a now-deleted Twitter post.
It said: “As one consultant cardiologist said to me this is the biggest crime against humanity since the Holocaust.”
His tweet caused a furious online backlash, including from some Tory MPs.
Simon Hart, the Conservative chief whip, said: “Andrew Bridgen has crossed a line, causing great offence in the process.
“Misinformation about the vaccine causes harm and costs lives. I am therefore removing the whip from Andrew Bridgen with immediate effect, pending a formal investigation.”
Appearing on GB News today, Adam Holloway, the Conservative MP for Gravesham, said people were entitled to air controversial views.
He said: “What has happened to freedom of speech? We actually can’t say what we think.
″We talk about diversity and inclusion and all this stuff. Well, why can’t people have different views, even if they’re crazy views and they’re completely wrong?
″For Andrew Bridgen to lose the whip is another example of the Tory Party going down this, to me, frighteningly totalitarian, woke way of looking at the world.
“I don’t agree with Andrew on the vaccine stuff, but if he wants to say this or anything else, I think people should be allowed to.
“It is really worrying now that we’re all having to self-censor. If people are crazy, I want to know that they’re crazy.”
In a video statement posted online last night, Bridgen denied being racist and said his tweet was “in no way anti-semitic”.
He said: “I’m disappointed that the chief whip, Simon Hart, with the support of the prime minister, has chosen to suspend me as a member of the Conservative parliamentary party.
“My tweet of the 11th of January was in no way anti-Semitic. Indeed, it alluded to the Holocaust being the most heinous crime against humanity in living memory.
“Of course, if anyone is genuinely offended by my use of such imagery, then I apologise for any offence caused.
“I wholeheartedly refute any suggestions that I am racist and currently I’m speaking to a legal team who will commence action against those who have led the call suggesting that I am.”
He went on to criticise the government for “actively look(ing) to remove MPs” who question the safety of Covid vaccines and said he had received “huge support” for his claims.
Bridgen is currently serving a five-day suspension from the House of Commons for breaking lobbying rules.