Breast Cancer Care said the lack of specialist nurses was impacting on patient care.
New cases of breast cancer in England have risen 18%, from 38,153 in 2003 to 44,831 in 2013. Wales and Scotland have also seen rises of around 20%.
Meanwhile, the number of specialist breast cancer nurses has remained around 430 across England since records began in 2007.
The increasingly ageing population plus rising obesity and higher levels of alcohol intake are fuelling the rise in cases.
Breast Cancer Care said specialist nurses are vital for giving patients support from diagnosis to recovery.
Patients with access to a specialist nurse report significantly better treatment and care in the NHS cancer patient experience survey.
Samia al Qadhi, chief executive of Breast Cancer Care, said: "Breast cancer nurses do a fantastic job but they are under more and more pressure to provide the same quality of care with much less time, more responsibilities and many more patients.
"We welcome the Cancer Strategy recommendation that every cancer patient should have access to a specialist nurse, but the next step is how we make that a reality.
"We know NHS England's budgets are tight, but as the number of breast cancer cases rises, action is needed to address this now."
Breast cancer is the most common cancer in women and is also the fastest rising cancer in women after lung cancer.
The number of people living with breast cancer is set to more than double from 691,000 at present to 1.7 million by 2040.
Although 78% of women with breast cancer live for 10 years or more after diagnosis, there are still almost 12,000 deaths a year from the disease.
A spokeswoman for NHS England said: "Decisions on numbers of posts available for clinical nurse specialists are the responsibility of individual hospitals, and overall hospital nurse staffing levels have been rising.
"The NHS is working with partners such as Health Education England to plan for future workforce needs, and to implement the independent cancer taskforce strategy.
"In the meantime, it's great news that breast cancer survival rates are now the highest they have ever been."