A survey of eight to 10-year-olds has found they are spending just 20 minutes a day being active when the recommended figure is an hour.
By the age of eight, girls are already becoming more sedentary than boys - a phenomenon previously noted in secondary school - and it could be due to a belief that sport is not "cool".
Female role models such as Olympic hopeful Jessica Ennis could help the problem.
In work published in the open access journal PLoS ONE, more than 500 eight to 10-year-olds wore activity monitors which gave analysts an accurate picture of how little time children spent exercising.
They were monitored for a range of actions from moving around, climbing stairs to running, playing games and skipping.
It revealed children spent only 4% of awake time in moderate to vigorous intensity physical activity, about 20 minutes per day, while the recommended amount for health is 60 minutes per day.
At the age of eight, girls were already less active than boys, which surprised researchers as the change happened earlier than previously thought.
Scroll down to read inspiring fitness tips...
Older fathers tended to have less active children - possibly as a result of having more senior roles at work and less free time, or perhaps as they had a traditional, less hands-on view of parenting.
Children who took part in sports clubs outside of school were significantly more active than those who did not.
Newcastle University's Dr Mark Pearce, who led the study funded by the National Prevention Research Initiative, said: "Given the importance of physical activity in maintaining good health, we know we need to get our kids more active.
"What we hadn't known until now is how young we need to be catching them, or the reasons that lay behind their lack of activity.
"Already at the age of eight, we are seeing girls being less active than boys.
"This is something which we know then gets worse as they approach their teenage years.
"One of the important things is that most girls don't see sport as cool. We need to be tackling these issues earlier by encouraging girls to exercise, by providing a wider range of opportunities than are currently on offer and by ensuring they see positive female role models, particularly in the media."
As to why the children of older fathers were found to be less active, Dr Pearce said: "We think there may be a variety of explanations for this such as older fathers reaching more senior posts and having to work longer hours or maybe seeing themselves in a more traditional role so spend less time in active play with their children."
Professor John Reilly from the University of Strathclyde, one of the researchers involved in this study said: "There is an urgent need for interventions, at home and at school, which will help primary school children become more physically active."
Dr Pearce said the Olympics could be a great advert for female sport.
"It would be a fantastic legacy for the Games," he said.
"Hopefully they can inspire children and encourage young girls to be more active."
One unexpected outcome of the research was a finding that children whose television-watching was restricted tended to be less active.
This may be because parents who do not allow children to watch much television do not want them to play sport either - if, for example, they were particularly keen on maximising study time.
It could be that those youngsters are not exposed to TV sport so are less interested, or it may be a statistical quirk, Dr Pearce said.
Inspiring ways to get active, by leading fitness coach, Tim Weeks...