More than 470 cancer doctors have written to Jeremy Hunt to say imposing a new contract will have "negative effects" on cancer care in England.
The letter, which has been signed by consultants and junior doctors working in oncology, said the Health Secretary's decision to impose a contract would only damage the recruitment of cancer doctors and would do nothing to retain doctors already in the NHS.
It warned that the new contract "will lead to worse outcomes for our cancer patients" and will widen the gap that already exists on cancer survival between the UK and other countries.
It said: "The announcement of the imposition of a new junior doctors' contract will have considerable negative effects on the future of cancer care in England.
"The Independent Cancer Taskforce's report published last year stated that 'in some parts of the country, workforce deficits in oncology mean that it is not possible to deliver optimal treatment safely'.
"This damning indictment of the current state of cancer services is a warning that should be heeded. Furthermore the UK already falls behind many European countries in oncology doctor:patient ratios and outcomes are significantly worse than our European counterparts.
"This situation can only worsen in the context of a new contract that will actively discourage recruitment and retention of the clinical and medical oncology workforce."
The letter said the changes being brought in by the Government - following months of dispute with the British Medical Association (BMA) - will have an adverse impact on the training opportunities for oncology trainees.
It added: "Clinical and medical oncologists already offer 24/7 access to acute oncology services. The existing contract is not an obstacle to providing this care. However the increased out-of-hours work will reduce the cancer-specific training that junior doctors will be exposed to leading to less experienced consultants of the future."
The letter said doctors working in cancer would also receive far less pay for working out-of-hours than at present.
It said it was "inconceivable" that the gap between the UK and other countries when it comes to cancer could be closed "without an oncology workforce replete with well-trained and enthusiastic specialists".
A poll of 600 junior doctors and consultants carried out by the Association of Cancer Physicians (ACP) and the Faculty of Clinical Oncology of the Royal College of Radiologists (RCR) found that 95% agreed the new contract would adversely affect outcomes for cancer patients.
Some 52% of oncology trainees surveyed said they would now "look for a job outside the NHS".
And 95% said they feared training in oncology would be less attractive under the new contract.
Professor Johnathan Joffe, chairman of the ACP, said: "The new junior doctors' contract will not deliver better care for patients including those suffering from cancer.
"The fundamental issues of understaffing and underfunding of NHS oncology services and the workforce remain unaddressed.
"With the imposition of this contract the ability to maintain and develop the workforce in non-surgical oncology is of particular concern, while the existing gap in oncological staffing levels and cancer patient outcomes between England and other developed nations will only get worse."
Dr David Bloomfield, from the Royal College of Radiologists, added: "One in two of the population will be diagnosed with cancer in their lifetime. We are short of cancer specialists now and this is going to get worse as it is, as we do not have enough training numbers to deal with future demand for cancer doctors."