Allen wrote on Twitter: "One thing is for sure though Boris will not be negotiating with the working classes interests in mind. We must #stopboris."
The tweet came after the pop singer started the day by calling out Jeremy Hunt in an earlier message where she labelled the health secretary an "insidious prick".
Allen's attack on Johnson came after she appeared to annoy her fans by changing her tune on what side of the EU referendum debate she supported.
She wrote: "Never thought I'd say this, possibly because project fear worked on me. I'm now quite excited about #Brexit and shift on political climate.
"I'm not saying I wouldn't be happy if we'd remained, not at all. But if in time it levels the playing field , tbc 1/2.
"And helps to narrow the gap between rich and poor, that can be no bad thing. The people have spoken, some for good reason. I stand by them."
Later in the day Allen threw her support behind Labour leader Jeremy Corbyn as reports emerged of him being urged to resign after appearing at the London Pride parade.