New research as part of the Talking TDS campaign, found that nearly half of men would rather discuss financial matters than loss of libido. Meanwhile just four in ten guys are prepared to speak with their partner about their physical and mental health.
And in a culture where illness could be taken as a sign of weakness or loss of masculinity - should we really be surprised?
In an exclusive interview with HuffPost UK Lifestyle, Dr Christian Jessen puts it quite frankly: "Not being able to perform in the bedroom could literally be killing men."
Here, he lifts the lid on why men need to admit defeat more often.
Afraid Of The Unknown
"With men, what tends to happen is that (unlike women) they don't have a great need to access healthcare for the majority of their early and middle-age," says Dr Jessen.
"Women are often going in about pregnancy, breast checks, smear checks and contraception, so they're very used to talking about that sort of thing. Whereas men often don't have to see the doctor until they hit 50."
"The problem then, and the sorts of things that go wrong for men at that age, are those things that could be seen as acutely embarrassing - so erectile problems or maybe prostate problems, which are often very lifestyle related."
What's more, he adds, is that erectile dysfunction can often be a pre-cursor of more serious, underlying health problems such as heart disease or diabetes.
Lack Of Knowledge
Another issue is lack of education surrounding the subject, with 43% of men being unaware of what TDS actually is.
“Loss of libido is the symptom most associated with testosterone deficiency yet men often think it’s just a part of ageing," reveals Dr Jessen.
Other signs and symptoms include tiredness, depression and erectile dysfunction.
Dr David Edwards, a GP with a special interest in male and female sexual dysfunction, adds: "TDS can impact on quality of life both emotionally and physically and can cause wider problems if left untreated.
"However, it can be diagnosed and managed.”
Male pride is probably one of the main factors behind why men don't seek help.
"The embarrassment about not being able to perform in the bedroom could be literally killing men. I know it's not a new subject, but you can't say it enough," explains Dr Jessen.
"The reticence to come forward might be stopping them from getting something like diabetes sorted out early. And it's imperative that they get over that."
Traditions And Beliefs: 'Men Should Be Alright'
Guys might also shy away from seeking medical help because of traditional norms and stereotypes surrounding their gender. In particular, the belief that you can be unwell and remain masculine.
"This idea that men look after the family and provide for them - I know it's a bit of an old-fashioned view but nevertheless, I still think a lot of men strongly believe it," reveals Dr Jessen.
Sadly, the turning point for a lot of men experiencing erectile issues is often late in life. So what drives them to ask for help?
Dr Jessen believes that many divorced men find themselves single again and often want to give their sex life an overhaul - and then they remember there are issues downstairs.
"What often drives men in [to doctor's surgeries], is they find themselves out on the dating scene again and when things aren't working properly, that can be a very powerful driving force to get them into the doctor's."
"I'm going to be a bit cruel here but it's often true," says Dr Jessen. "It tends to be your fat, middle-aged businessman who's boozing a bit too much, working a bit too hard and their own health is very low down on the list of important things to worry about."
"Suddenly they're single and they'd quite like to meet someone, and erectile problems then become an issue."
Story continues below...
Despite preconceptions surrounding sexual health, a lot of the time treatment is nothing more than a few simple lifestyle changes to get their health back on track.
"It can be a powerful driving force to get people losing weight, doing some exercise and changing their diet."
"I've spoken to people who have come in about erectile problems and have revealed that they wake up in the night with chest pains or terrible cramps in their chest.
"And you think: 'That sounds really quite serious'. But it's the sexual problem that's more pressing to a man than crushing chest pains."
He adds that the bottom line is that a lot of major health issues can be prevented with simple lifestyle changes.
"It's really important to realise that a lot of these diseases are not just bad luck or inevitable. You are in control quite a lot more than you think."
"Most men react when something goes wrong, they don't react before something goes wrong. In an ideal world, it would be great to help stop you getting sick in the first place, instead of treating you once you're sick."
The new research was conducted as part of the new Talking TDS awareness campaign sponsored by Besins Healthcare UK Ltd.