The Chancellor insisted the coalition will keep the public finances under tight control as he gave details of a shake-up of green levies that could see £50 shaved off energy bills.
Speaking on the BBC's Andrew Marr Show, Osborne said his Autumn Statement on Thursday would stick to the task of delivering a "responsible recovery".
Despite signs of a sharp upturn in growth, he said there were still "lots of risks" for the economy and increasing borrowing would be "disastrous".
Asked how £1,000 grants for homebuyers to improve energy efficiency and cuts to environmental levies on bills would be funded, Osborne replied: "The money will come from additional taxes that we will raise from dealing with tax avoidance."
Chief Secretary to the Treasury Danny Alexander said he hoped the coalition's proposal would persuade Labour to drop its "barmy idea to con the public" with a 20-month energy bill freeze should it win the 2015 general election.
He explained the new measures paid for by "tax dodgers" would also "not sacrifice a single gram of carbon" that the coalition was already aiming to save.
Speaking on Pienaar's Politics on BBC Radio 5Live, Alexander said: "We are just doing it differently. So, rather than saying that bill payers should pay all these costs for other people to insulate their homes, we're saying instead that tax dodgers should pay that because we're going to use the taxpayer to pick up some of those costs and use that money to give people extra financial incentives.
"So, if you like, we're helping to take money off people's bills through the energy companies and we're paying people to take action to cut their own bills further."
But shadow chancellor Ed Balls claimed the coalition's new energy announcement would "not last 24 hours", adding the lack of answers from Deputy Prime Minister Nick Clegg and Prime Minister David Cameron made them look "a bit naked today".
He told 5Live: "I think by Thursday George Osborne will have to come up with something more but this is not going to be good enough."
Balls added: "What's happened today is David Cameron and Nick Clegg, in a rather bizarre joint article in The Sun, have decided they're going to try and engage Ed Miliband on Labour's territory - pointing out the cost of living crisis, the failure of action on energy prices, the fact that most people are worse off not better off compared to 2010.
"But they've raised the issue and they've not got an answer and therefore... I think David Cameron and Nick Clegg both look together a bit naked today."
To laughter, Balls added: "That was not intended to be more than a metaphor."
The package of changes to green levies will see the the Energy Company Obligation (ECO) scheme halved by giving the "Big Six" power firms two years longer to hit targets.
Other policy charges will be funded from general taxation in future.
EDF has welcomed the move - expected to trim average bills by £50 a year - and indicated that it was not now likely to hike prices again before 2015.
In addition, anyone buying a home will be eligible for the £1,000 grant for energy efficiency measures, such as installing insulation or replacing the boiler. The sum could be even higher if the property needs a great deal of work.
Osborne dismissed the idea that energy companies will pocket the reduction in government levies without bringing down bills.
"We are absolutely insistent that this is passed on... I am pretty clear with you that it is going to happen," he said.
He refused to give details of the tax avoidance crackdown but said people were wrong to be sceptical about whether such action really raised revenue.
"This government has taken step after step and the amount of tax we collect from people who were previously avoiding their tax goes up by billions of pounds over this parliament," he said.
The chancellor attacked Labour's pledge of a 20-month freeze on energy prices if they win the election - which has been dominating the political agenda since Ed Miliband announced it in September.
"We are doing it in the way that government can do it, which is controlling the costs that families incur because of government policies," he said.
"We are also doing it in the way that is not going to damage the environment or in any way reduce our commitment to dealing with climate change."
He went on: "It is all about providing people with carrots not sticks and I think that is the right way for this country to go green.
"By taking these additional measures we can afford to help the vast majority of people who do pay their taxes, have expensive electricity and gas bills and therefore want to have relief from the government."
Liberal Democrat president Tim Farron told BBC One's Sunday Politics: "The notion we should stop insulating the homes of elderly people or indeed stop investing in British manufacturing to the green industry is something that I absolutely, resolutely oppose.
"But I'm very pleased the funding is still going to be made available for all that and instead come from general taxation."
Told the Lib Dems are in coalition with a Tory party looking to drop its green credentials as fast as possible, Farron said: "I was as despondent as many reasonable people when David Cameron's panicked reaction to the issue in the last month or so was to decide to ditch all the green stuff, and it's been a job and a half over the last month to make sure we hold him very much to those sort of pledges and the green core of this Government.
"That is why we're not scrapping the investment, we're just making sure it's funded from general taxation.
Osborne said: "The economic plan is working and the recovery is under way.
"In the Autumn Statement I will say the job is not yet done because we have got to make sure we go on taking the difficult decisions to secure the recovery.
"We also want a responsible recovery. We want to learn from the mistakes of the past and not see a re-emergence of those problems in the financial system that brought this country to its knees."
Osborne rejected suggestions of tension with the Bank of England over the need to cool the housing market down.
Last week, governor Mark Carney announced that from next month the Bank's Funding for Lending scheme will no longer be available to support mortgages, but will instead be targeted at business lending.
"On housing, in particular, what the Bank of England says - what I say - is that there is not a housing bubble at the moment but we want to make sure one doesn't develop," Osborne said.
"I think you see us working in concert - the Bank of England and the Treasury - to make sure that we do not repeat the mistakes of the past."
Balls told the Andrew Marr Show: "I set two tests for George - one, would he stop bills rising? Secondly, would the energy companies pay?
"If David Cameron and Nick Clegg are right, he fails both tests. I would say to George, while the Prime Minister is in China, get back to the drawing board, come up with a policy, we've had lots of u-turns already, do another one, freeze the bills, take Ed Miliband's policy - that would be wise."
Questioned if Labour's policy could result in energy companies driving up bills before and after a price freeze, Balls said: "They are the Government. I think they should get a grip.
"Why do David Cameron and George Osborne run scared in the face of energy companies who have been putting up bills month by month, year by year?
"If energy companies came together as a cartel to push up prices to try and pre-empt a government action that would be a total abuse of market power and any sensible regulator would step in and stop it happening. I don't think they (energy companies) are going to do that although they are putting their bills up now."
Balls has often been seen making a flat-lining gesture with his hand to taunt the Chancellor over the lack of growth in the economy.
Asked if he had a new gesture given the improvements in the economic situation, Balls said: "If I had known flat-lining would last three years I might have thought twice about doing the gesture in the first place."
The shadow chancellor also dismissed claims Labour's future financial plans, which are said to be close to French President Francois Hollande's, would result in the UK enduring similar struggles as the French.
He said: "France is in the single currency, the euro, which rightly back in 2001 we decided not to join - a very, very good decision.
"America and Germany though - Germany in the euro, America not - are both well above where they were in 2007."
Told the UK is doing better than those two countries at the moment, Balls said: "But we are still 2.5% below our pre-crisis peak. We've got so much to catch up and it's got to translate to working people."