Competitive team sports will be made compulsory for all primary age children, David Cameron said on Saturday.
His comments came after he criticised schools for holding Indian dance classes.
The Prime Minister, who is under fire for scrapping a target for pupils to do two hours of sport per week, said the new requirement would be included in the revised national curriculum.
School sport has been thrust into the spotlight by Great Britain's success at the London Olympics, amid concerns that the momentum could be lost unless youngsters are offered more opportunities.
Critics have called for the Labour-imposed target to be reinstated, with London Mayor Boris Johnson saying he wanted to see pupils emulating the two hours a day he enjoyed at Eton.
But Mr Cameron dismissed the change, complaining yesterday that many schools were meeting the target "by doing things like Indian dance or whatever, that you and I probably wouldn't think of as sport".
Setting out his plans to make sure the London 2012 "Inspire a Generation" slogan is met, he said he wanted to end an "all must have prizes" culture and force pupils to think about beating their personal bests.
The most recent Government survey of primary schools found more than 10,000 primary schools had fewer than half of their pupils competing against other schools three or more times a year and 1,950 had none at all.
A new draft PE curriculum, to be published in the autumn, will make it compulsory to take part in what Downing Street called "recognised and recognisable sports" such as football, hockey and netball.
It will also prescribe "team outdoor and adventurous activity".
Mr Cameron said: "The idea of an Olympics legacy has been built into the DNA of London 2012 from the very beginning. Now the London Olympics has been a great success, we need to use the inspiration of the Games to get children playing sport more regularly.
"I want to use the example of competitive sport at the Olympics to lead a revival of competitive sport in primary schools. We need to end the 'all must have prizes' culture and get children playing and enjoying competitive sports from a young age, linking them up with sports clubs so they can pursue their dreams.
"That's why the new national curriculum in the autumn will include a requirement for primary schools to provide competitive sport."
Mr Cameron, who has said two hours per week is insufficient, has come under fire from teaching unions for suggesting the targets had led to a tick box mentality where some teachers did the minimum required.
Shadow education secretary Stephen Twigg said: "If David Cameron supports primary school sport, why did he abolish Schools Sports Partnerships which allowed primary schools access to coaches, staff and facilities to do competitive sport?
"This announcement doesn't look like a thought-through plan - there are no details of how this will be supported or funded and no plan for secondary schools.
"Instead of seeking to blame teachers and divide the country, Labour wants to build a consensus with a cross-party, 10-year plan for school sport."
Youth Sport Trust chair Baroness Sue Campbell welcomed the move but cautioned that teachers would need extra support if it was to be effective.
She said: "We welcome the commitment from the Prime Minister to put physical education and sport back at the heart of primary school life.
"Competition is a key factor in the development of young people and provides an opportunity to learn important life skills.
"However, it is important that young people are taught and coached in the basic skills of the various sports so that they have the confidence and competence to enjoy and succeed in competition.
"In primary schools there are no specialist teachers of PE and sport so it will be critical to provide training and support for teachers if we are to maximise this opportunity."
Alan Pascoe, who won a silver medal at the 1972 Olympics and was vice-chair of the successful 2012 bid, welcomed the announcement.
"We must not lose this fantastic opportunity to live up to our promise to create a legacy of inspiration and that has got to start at primary school if we are to continue and improve our level of competing at the highest level.
"We respect and revere great talent, whether it's in business, science, music, technology or the medical field and sport has to sit right alongside these as an aspiration for the younger generation.
"I'm glad the Government recognises that this is a priority right now, while we are riding this wave of pride in our sportsmen and women and I hope the private sector will also support the initiative.
This is not just about competitive sport, it's about creating a more active and fitter generation which will also make us a healthier nation."
But Labour former sports minister Gerry Sutcliffe said it was the wrong approach.
"He wants to compel schools to ensure that all pupils play football, rugby and netball, but you can't impose this from above without any consultation.
"You've got to do it by building up local sporting partnerships - the very partnerships that the coalition has been destroying."