John Bercow has a tendency to "shout off" but the campaign to oust him is "undignified", a senior Tory MP has said.
A motion of no confidence in the Commons Speaker was tabled as MPs left for the February recess, but with Parliament returning on Monday his supporters and critics will seek to bolster their positions.
Veteran Conservative Sir Nicholas Soames said the situation was "unattractive" and insisted he would not support either campaign.
Pressure on Mr Bercow has mounted since his outspoken comments essentially banning US President Donald Trump from addressing MPs and peers in Westminster Hall during his forthcoming state visit.
Opposition to his position increased after a video emerged of him telling students that he voted Remain in the EU referendum.
Sir Nicholas told ITV's Peston on Sunday: "I think John Bercow has, in many ways, been a really very good Speaker. But he does tend to shout off on occasions when it would be better if he didn't.
"I will take no part in any campaign, one way or the other, and I think it's really undignified, unattractive. The Speaker's office is one of the most important in the country."
He added that he had received emails from both sides "and I have replied no to both of them".
Labour former leadership contender Liz Kendall told the programme Mr Bercow had her backing and claimed some Tories had a vendetta against him.
"I don't think any of this is about what he said about the state visit, I think there is a vendetta here," she said. "There are many Conservatives who just do not like him because he has made this journey from the far-right to being more liberal and he has been a radical, reforming speaker in terms of changing the traditions.
"I think that's what's really underneath it and by having this motion of no confidence I think they may have made a mistake and people will rally around the Speaker. I certainly will."
Last week it emerged Mr Bercow has received more than 4,000 letters and emails about his decision on Mr Trump.
The vast majority of the communications (3,227) were supportive of Mr Bercow's attack on the US president, while 854 opposed his position, figures released in response to a freedom of information request reveal.
Tory former minister James Duddridge, who tabled the "no confidence" motion in Mr Bercow, has claimed that no Cabinet ministers are likely to support the Speaker in any vote.
Downing Street has said Mr Bercow's future is a "matter for MPs", in a sign that ministers will not be whipped into a position in any vote of no confidence.