It's been four years since the sight of smokers huddling together outside pubs in order to enjoy a cigarette became the norm as a result of the illiberal ban on smoking in enclosed public spaces being introduced in England. Far from being satisfied with this infringement on peoples' freedom to smoke, anti-smoking campaigners are still making vociferous calls to extend the ban, with councillors from a small Buckinghamshire town proposing it becomes the first place in the country to impose a blanket ban on smoking on its streets.

It's remarkable how the rhetoric used by anti-smoking campaigners shifts. Unable to use the dangers of passive smoking - which was one of the primary reasons given for banning smoking in pubs - as a particularly convincing argument for restricting smoking outdoors, they instead hunt for other reasons to clamp down on people's ability to smoke.

A whole barrage of arguments have been tried and tested: people smoking in public are effectively advertising their habit to children, cigarette butts are covered in diseases, outdoor heaters outside pubs lead to climate change, smoking areas outside pubs lead to congestion on pavements and noise pollution. Campaigners even want to ban smoking outdoors simply on the grounds that it's smelly (What next? People with BO being unable to leave their houses?) In a move that shows just how puritanical campaigners are, some are even so offended by the sight of cigarettes they want to ban the smoking of e-Cigarettes, which produce relatively harmless, odourless steam.

Certainly councillors in the market town of Stony Stratford in Buckinghamshire are marshalling many of these arguments to call for a bylaw that would clamp down on smoking in public places. The Daily Mail quotes local councillor Paul Bartlett, as wanting to implement the ban because: 'Why should people have the freedom to smoke in my face, pass on diseases and spoil the environment? ... Smokers then get their butt, which is full of saliva, and chuck it on the floor. It costs millions to clear street rubbish, and goodness knows what a child could pick up from them.'

Following this dodgy logic, would Bartlett also favour bans on chewing gum and drinking from bottles, cans and cups which contain people's saliva in case they chucked them on the floor? Why single out smoking? Surely, if littering is such a problem, then the solution should be to tackle that rather than trying to ban the sources of rubbish.

Bartlett's contempt for the idea that people should have the 'freedom' to engage in a habit that he dislikes is also of great concern. Who does he think he is to deprive other people of their own freedom to choose? Certainly if people blow smoke in his face, he should feel perfectly free to challenge them about it. (As a non-smoker myself, I find that smokers are usually extremely polite if you ask them nicely to blow their smoke in a different direction.) But such irritants should be dealt with informally, not through legislating to ban smoking outside completely.

It's worrying that there has been little furore over the proposed ban in Stony Stratford, despite the fact the Independent seemed hard-pressed to find a single person who supported such a ban during a recent visit to the town. It suggests that, while public opinion isn't in favour of such a blanket ban - as it wasn't before the smoking ban in enclosed public spaces was implemented - policies pushed forward by those with authoritarian tendencies, such as Councillor Bartlett, may be met with a reluctant shrug rather than strong opposition.

Which is why it was heartening to attend an event at the House of Commons organised by the Save Our Pubs and Clubs campaign as it lobbied MPs at last week. Four years on from the smoking ban, you might expect campaigners to be dispirited, but from the fighting talk of Simon Clark, director of Forest (the Freedom Organisation for the Right to Enjoy Smoking Tobacco), alongside artist David Hockey and others, it's clear there is plenty of fight left in the pro-smoking lobby.

Even if, like me, you don't smoke, there are important reasons to oppose the smoking ban and attempts to further expand it. The smoking ban is just one example of an increasing trend by the state to interfere in our private choices and restrict the freedom of public spaces. We are perfectly able to make decisions for ourselves and negotiate relationships without the need for laws to do this for us. People should be free to smoke in public spaces, but the government's nasty habit of creeping petty authoritarianism needs to be stubbed out completely.

Follow Patrick Hayes on Twitter: www.twitter.com/P_Hayes