30/09/2014 17:58 BST | Updated 30/09/2014 18:59 BST

Listen To IDS Dodge Mansion Tax Question With An Eric Pickles Fat Jibe

LEON NEAL via Getty Images
British Communities and Local Government Secretary Eric Pickles arrives in Downing Street, central London, on August 20, 2014, ahead of a meeting to discuss the ongoing crisis involving the Islamic State radical group. British Prime Minister David Cameron broke off his holiday on August 20 for talks on the threat posed by Islamic State jihadists following the 'shocking and depraved' apparent beheading of US journalist James Foley. AFP PHOTO / LEON NEAL (Photo credit should read LEON NE

Iain Duncan Smith has avoided saying what he thinks about hitting rich people's properties with a "mansion tax" by joking that he does not want his cabinet colleague Eric Pickles to sit on him.

Speaking on Tuesday night, Duncan Smith said he was "not in a position" to comment on hitting high-value properties with a mansion tax, which Labour has suggested implementing on houses worth more than £2 million.

"If I started talking about taxes on buildings, I'd have Eric Pickles sitting on me, and that'd be unpleasant," he quipped.

The work and pensions secretary made the jibe at a fringe event hosted by the Centre for Social Justice at the Conservatives' last party conference before the general election in Birmingham after being asked by Tory commentator Tim Montgomerie about whether more taxes should be levied on the wealthy.

Duncan Smith is not the first Tory cabinet minister to poke fun at Pickles' weight; George Osborne did likewise last year.

"My Rt Hon friend the communities secretary has set an example to all his colleagues by reducing the size of his department by 12% and abolishing 12 quangos," the Chancellor told MPs, adding to cheers all around: "He is the model of lean government."

David Cameron has also indulged, previously calling Pickles "the big man on the side of the people" and joking about him running.

Duncan Smith did not just joke about Pickles in his answer tonight, telling Montgomerie that he had been consistently arguing for "balance and fairness" in how the coalition cuts were implemented.

This comes after Osborne unveiled how the Tories would make an estimated £3 billion out of a planned £25 billion of cuts by freezing welfare for working Britons - a move that was condemned as targeting "the poor and the vulnerable".

Duncan Smith insisted: "I believe that by the time we get to the next election that I will be able to stand on a platform where I am able to say that we have not balanced the books on the backs of the poorest."

"Even now people who are earning more are paying more than they ever did in the lifetime of the last government. It's about what you do with the people who are the wealth creators, who innovate, who settle here create jobs and give jobs to people."