Another heartbreaking reality was that those who do want to keep fit are choosing to exercise in their sheds, hidden away, out of fear of being laughed at.
The report comes after Public Health England revealed that the number of women achieving recommended levels of physical activity was far lower than men - 31% of females engage in sport once a week compared to 40.1% of men.
The report, which has been collated by the Commons’ Health Select Committee, labels “fear of judgement” as a key factor when it comes to why women's fitness levels are below par.
Kay Thomson from Sport England said: "Three quarters of women want to become more active but something is stopping them - fear of judgement.
"Judgement about appearance when exercising, ability to be active, confidence to turn up to a session, or feeling guilty about going to be physically active or doing something when you should have been spending more time with your family."
One woman who was interviewed for the report, Julie Creffield, says that lack of information is a key factor which prevented her from keeping fit.
She revealed: "When I looked online for information, there was lots about weight loss and running but nothing about running just as an overweight person, the psychological aspects of that and how tough it is when you are constantly shouted at, laughed at and clothes in fitness stores don’t fit you.
"It feels like the whole sport is not geared up for you."
Figures released by Public Health England state that the percentage of adults achieving recommended levels of physical activity has increased in the last fifteen years from 32% to 43% among men, and from 21% to 32% among women.
Creffield, who is a motivational speaker and campaigns to keep women fit, said that you do not see many overweight people exercising because they often do it in secret due to embarrassment.
"I have women who tell me they run on a treadmill in their shed because they just don’t want to be seen in public," she said. "But that is part of the problem. Because we don’t see many overweight women exercising in public, other women don’t think that exercise is for them."
"They think it is for all the slim people that they always see out in the parks."
She added that larger women aren't able to get hold of sports kits which fit them properly, which presents another barrier: "No woman wants to dress in men’s clothing to go out for a run when there is already the risk of being laughed at."
The Government now hopes to address these barriers and issues by releasing a programme on diet and physical activity which works to examine how women, those with disabilities and overweight people, can be encouraged and supported to be more active.
Sport England's This Girl Can campaign is also helping to get women moving by showing "real women" working out - in a bid to help others summon up the courage to get active.