19/08/2014 13:49 BST | Updated 19/10/2014 06:59 BST

Three Pimps and a Lockout

I hadn't seen Waldorf for quite a time, but there he was, in the middle of a grey and humid Soho Sunday afternoon, loitering just outside the frozen yogurt shop which stands out like a pink belisha beacon against the surrounding drabness.

The aficionados of this allegedly healthy dessert, who pack the shop from the time it first opens in the morning to the time it closes late at night, probably wouldn't give a second glance to this small, glum, white-haired and white-moustachioed fellow in grubby beige trousers and a tan leather jacket of a certain vintage. But Waldorf, along with his associate Statler (who I've always presumed is his brother, though outside of them both being very small men of very few words, and both being in their late 60s or early 70s, there's no concrete evidence of a filial connection) is carefully and discretely plying his trade as a pimp of long-standing in the neighbourhood.

Waldorf and Statler (known collectively, of course, as the Muppets) are among the last visible survivors of the days when the Maltese Mafia reputedly ran the sex trade in Soho. I suspect discretion is a large part of their survival. I've probably exchanged half a dozen words with Waldorf over the years, despite frequently seeing him strolling ever-so-slowly down my alleyway. Statler, always clad in a long overcoat and flat cap whatever the weather, hands thrust deep into his pockets, eyes darting in all directions, is more conversational, though strictly of the "How are you boss?" variety.

You don't very often see pimps visibly plying their trade in my neighbourhood these days, which is both a sign of Westminster Council's success in its relentless campaign to crack down on the sex trade, and, perhaps, an affirmation of what the local girls proclaimed loudly and with varying degrees of success in court, after the dodgy police raids of last December.

Supposedly launched to crack down on the re-sale of stolen goods that the Met said was fuelling the drug business in Soho, the raids ultimately seemed more about an attempt to shut down a large number of apartments used by prostitutes. There was no mention of the resale of stolen goods when court orders were sought against 18 flats, the police relying instead on breaches of the Sexual Offences Act.

Despite the flimsiness of the Met's claim that there was a Mr Big "controlling" them, a number of the girls were forced out on the streets - a mad, bad and dangerous for them to know outcome - while only a handful won the right to return.

In a written judgement after two girls on Brewer Street appealed, the Recorder said: "[The Met] says that, taking the evidence together, it is clear that, on the balance of probabilities, someone was controlling the activities of the appellants....We disagree. In our view, the furthest the evidence goes is to show that the appellants used the first and second floor flats by arrangement with other sex workers at mutually convenient and agreed times. That does not constitute control."

I have no doubt that many women are forced into the sex trade and there are some pretty unsavoury characters out there making money from their appalling situation. But I know the working girls in my neighbourhood, and, like many local residents, my suspicion is that the raids and attempts to shut down the sex trade in Soho seem to have far more to do with Westminster Council's implacable opposition to the business, alongside private efforts to "gentrify" the area and make it more like Covent Garden or, God help us, Mayfair.

Still, the last remaining pimps circle the neighbourhood like some strange, endangered species.

Much more voluble than the Muppets is Danny, the sharpest dressed man on the streets of Soho (or at least, the streets around me.) Always laughing and joking and greeting you with a handshake and a smile, Danny steers clear of the drug trade, and looks after the ladies in their 50s and 60s who continue to ply their trade in plain site outside my front door. If I have an undue amount of affection for him, it's in part because he has ridden to my rescue on more than one occasion.

The front door of our building is the original one from the 19th century, and one of its quirks is that it defies the efforts of any entry phone system to make it close. So we need to rely on people closing it by hand and using the mortice lock.

Occasionally, a visitor or a cleaner will leave the key in the lock after they've entered, locking everyone else out from the building. Noticing my distress one day as I shouted up from the street to try and get someone to come down and take the key out, Danny promptly rounded up a small platoon of working girls and punters to stand outside and yell "Take the key out, you dozy sod" to the unfortunate person who'd committed the foul deed - a community service Danny has managed to repeat for me on at least another two occasions.

As I said the first time it occurred, with an unlikely crowd of people throwing what came to feel like an impromptu street party of increasing volume, living on my alleyway is sometimes a bit like being in a Richard Curtis film, albeit one populated with as many black people as white people.

But the days of Danny coming to the rescue are, I fear, dwindling. Perhaps it's time to change the locks - or even the entire front door - in Village Soho.