Grieve blamed the defeated Ukip candidate who stood against him in the 2017 general election for organising the move.
On Friday night, Beaconsfield Conservatives voted by 182 to 131 against a motion of confidence in Grieve.
The local constituency association chairman Jackson Ng said he would be speaking to his fellow officers and executive council “as soon as possible” about what they would do next.
Conservative chairman Brandon Lewis, however, said Grieve remains a Tory MP and the vote had no formal standing under party rules.
He told the BBC Radio 4 Today programme: “Part of the strength of our party is that we are a broad church. Dominic is somebody who has contributed to Parliament, is a clear strong Conservative, and he is an asset to the party.”
Grieve said the confidence motion was moved by the constituency party officers after an attempt to table a motion of no confidence was ruled out of order.
He said the move against him had been organised by Jon Conway, who had been readmitted to the party after standing for Ukip in Beaconsfield in the last election.
Grieve told Sky News: “At the meeting there were a very large number of people turned up, around 100 of whom I had certainly never seen or met before in my years as a Member of Parliament.
“There is clear evidence that there was an orchestrated campaign by my Ukip opponent in 2017, who has since joined the association, with the express intention of trying to come along and defeating the motion.
“It was a slightly rowdy meeting, although the chair was able to keep it under reasonable control.
“But certainly it wasn’t the type of Conservative Party meeting that I’m used to attending.”
Conway insisted he had been only one person at the meeting and that it was Grieve who was out of step with the party on Brexit.
He told Sky News: “I’m one person… who voted no confidence in Dominic last night.
“He’s never really believed in the idea of it (Brexit) and he’s become very outspoken. But he’s the rebel, not us.”
In a statement, Ng acknowledged there had been a “robust” discussion and said he was announcing the result with a “heavy heart”.
South Buckinghamshire, which includes Beaconsfield, narrowly voted for Leave in the 2016 referendum by a majority of 570 votes.
Tory MPs rallied in support of Grieve, including Boris Johnson – despite their sharply differing views on Brexit.
He tweeted: “Sad to hear about Dominic Grieve. We disagree about EU but he is a good man and a true Conservative.”
Tom Tugendhat, chairman of the Commons Foreign Affairs Committee, said the vote was “deeply disappointing”.
He said: “Few in Parliament have contributed as much to public life as Dominic Grieve.
“He has served our country dutifully for decades. He deserves thanks and praise, not this. Our party and country would be poorer.”
Former minister Alistair Burt said it was an “extraordinary decision”, adding: “A quality colleague, brave enough to challenge, but fundamentally a Conservative through and through. Where on earth are we heading with situations like this?”