More needs to be done to address the psychological problems associated with being ill, a new report states.
Health officials also need to address the physical issues that come as a result of mental health problems, according to The King's Fund.
The authors of the new report argue that integrating physical and mental health care can both save the health service money and improve care for patients.
The charity's latest report states that there are high rates of mental health conditions among people with long-term physical health problems such as cancer, diabetes or heart disease - which could be costing the NHS in England £8 billion each year.
And poor management of "medically unexplained symptoms" - such as persistent pain or tiredness - costs the health service around £3 billion annually.
Meanwhile people with severe mental illness have significantly reduced life expectancies than the general population - largely attributable to poor physical health, the authors said.
There are "a range of barriers" preventing integrated care including separate budgets for physical and mental health care and cultural barriers between organisations, the report adds.
The national policy of achieving "parity of esteem" between mental and physical health care is welcome, but more can be done, the authors said.
Chris Naylor, senior fellow at The King's Fund said: "Traditionally physical and mental health have operated as distinct, separate systems in terms of both treatment and funding. That is no longer affordable financially or acceptable clinically.
"The Government has set the goal of parity of esteem, meaning that mental health care should be 'as good as' physical health care. We argue that there is an even greater prize at stake - that mental health care should be delivered 'as part of' an integrated approach to health."
Commenting on the report, Philippa Lowe, chair of charity Rethink Mental Illness, said: "The way mental illness impacts on someone's physical health has been badly neglected. I completely agree that it's not just about mental health care being 'as good as' physical health care, but they need to be treated as two sides of the same coin, as highlighted in this report. That's the only way we will see real change."
Luciana Berger, Labour's shadow minister for mental health, said: "Labour has been arguing for years that we should treat the person as a whole, including their mental health, physical health and social care needs, rather than responding to one symptom at a time.
"That makes sense for the patient, and the success of their treatment. Today's report highlights that it makes financial sense too.
"Tory ministers must stop hiding behind their rhetoric and act swiftly on these findings to give mental health the focus it demands."
Stephen Dalton, chief executive of the Mental Health Network, said: "Health outcomes can be greatly improved by bringing mental and physical care closer together. We need to see the mental and physical health care of individuals as two sides of the same coin and which require equal attention, equal funding and coordination at the point of delivery.
"There is now a significant body of evidence that signposts the need for Government to invest in mental health services after years of neglect."
A Department of Health spokeswoman said: "We are absolutely committed to making sure physical and mental health are treated equally and the interaction between the two is recognised.
"Only recently, the NHS published its plan to revolutionise mental health care with an additional £1 billion invested by 2020 - helping one million people with mental illness access new high quality services, including offering health checks to people with severe mental health problems so they have their physical health needs met."