Researchers who studied 1,447 women treated for breast cancer found that 8% of them had undergone a double mastectomy.
But 70% of these women did not meet the medically approved criteria for losing both breasts - a family history of breast or ovarian cancer, or BRCA 1 or BRCA 2 gene mutations.
They had a very low risk of developing cancer in the healthy breast, the US scientists said.
Study leader Dr Sarah Hawley, from the University of Michigan, said: "Women appear to be using worry over cancer recurrence to choose contralateral prophylactic mastectomy.
"This does not make sense, because having a non-affected breast removed will not reduce the risk of recurrence in the affected breast.
"For women who do not have a strong family history or a genetic finding, we would argue it's probably not appropriate to get the unaffected breast removed."
Story continues below the slideshow:
The research, published in the journal JAMA Surgery, also found that 18% of the women studied had considered a double mastectomy.
Women with a risky family history and BRCA mutations make up around 10% of all those diagnosed with breast cancer.
They are considered to be at high risk of a new cancer developing in the unaffected breast.
Without these indications, women who develop a tumour in one breast are very unlikely to acquire a second cancer in the other.
Many of the women in the study who elected to have a double mastectomy were candidates for breast-conserving surgery to remove lumps, the researchers said.
Overall, about three-quarters of the patients reported being very worried about their cancer.
Those who chose to have both breasts removed were significantly more likely to be concerned.
A double mastectomy is a major operation that can be associated with complications and difficult recovery, the authors pointed out.
Most of the participants who had the procedure underwent breast reconstruction as well.
The study found that women with higher education levels and who had been given a magnetic resonance imaging (MRI) scan before surgery were more likely to choose a double mastectomy.
The new research coincides with former Dancing With The Stars host Samantha Harris making public her decision to have a double mastectomy following a diagnosis of breast cancer.
She was praised by Baroness Delyth Morgan, chief executive of the Breast Cancer Campaign charity, who said: "Samantha's openness in sharing her experience serves as a stark reminder of the impact of a breast cancer diagnosis. It also raises awareness of the fact that breast cancer can occur at any age, although 80% of the 50,000 cases diagnosed in the UK each year occur in women over 50.
"Being breast aware can be crucial to early diagnosis, increasing chances of survival. It is important women report any unusual changes to their GP and consider attending routine breast screening when invited."
Ms Harris discovered a lump in her right breast during a routine self-examination.
Commenting on the study, Lady Morgan said: "This study was conducted in the US which has a very different healthcare system than the UK, where data on why mastectomies take place is not broken down in this way.
"Whether a woman in the UK is advised to have a double mastectomy when they are diagnosed with breast cancer in one breast will depend on the individual case, such as whether they have a family history of the disease.
"Women who are concerned about getting breast cancer in their other breast should speak to their healthcare team who will be able to provide them with support and information about the risks of this occurring and their individual treatment options.
"More research is needed into what triggers the growth of breast tumours and makes the disease progress for each patient. We hope that with more research by 2025 better and more personalised treatment options will become available for women with breast cancer, giving them the best possible chances for survival."
Sally Greenbrook, from the charity Breakthrough Breast Cancer said: "Double mastectomies are not routinely offered to women with breast cancer in the UK unless they have a family history of the disease or a high risk of recurrence as outside of these groups there is no evidence to suggest that this would be of benefit.
"This study provides yet more evidence to support the need for improved ways to establish a clear picture of an individual's risk of recurrence, enabling clinicians to better help their patients make decisions as they move through treatment and ongoing monitoring.
"Our dedicated team of scientists are working to make personalised medicine the standard for breast cancer treatment. By gaining a full understanding of every type of breast cancer, what causes them to begin and by mapping how they grow and spread, we will be able to match up each and every patient with a precise treatment plan based on the most effective ways of beating their particular breast cancer - with the fewest side effects possible."