The Government has reminded hospitals to offer concessions on parking fees to carers as MPs debate the issue.
Social Care Minister, Alistair Burt, said carers were "unsung heroes" but said it was up to individual NHS trusts to decide how they structure their parking charges.
The Government is not in favour of legislating for free parking to carers, arguing it would not be fair to the other groups who also deserve concessions.
MPs are looking to change the law to force bosses to exempt carers from paying to park at NHS hospitals, walk-in centres, GP practices and private hospitals.
The current system gives hospitals in England discretionary powers to grant exemptions. Parking charges were abolished at hospitals in Scotland and Wales in 2008.
Julie Cooper, Labour MP for Burnley, who is leading the bid to change the rules, believes the current system is "hit and miss" and in need of reform.
Introducing her Hospital Parking Charges (Exemption for Carers) Bill to the Commons for its second reading, she told MPs the charges are "one extra penalty" carers do not need.
Ms Cooper told the House she decided to campaign on the issue after she cared for her own mother when she was in hospital.
"Each night when I left tired and distressed I queued up to pay for my parking," she said.
"At that time it was costing me £40 a week.
"On one of those days driving out of the car park, it occurred to me that I was lucky because I could afford to pay this charge and I went on to reflect on the matter and I thought what about those people who can't afford to pay.
"Not those who would rather not pay to park, those who can't afford.
"I was distressed worrying about my mum but I thought how much more distressing it must be for those in financial hardship - financial hardship made worse by hospital car parking charges."
In a statement, Mr Burt said hospitals should look at the discounts they can offer.
Non-binding principles published by the Government in 2014 say carers should be offered discounts or free parking.
These should be made available for disabled people, frequent outpatient attenders, visitors with relatives who are gravely ill or their carers, and the relatives or carers of people having an extended stay in hospital.
Mr Burt said: "Carers are the unsung heroes of our society, which is why they are recognised in many ways by Government.
"But it's right that the NHS has the autonomy to make decisions that best suit their local circumstances and community interests.
"The principles provide clear direction and leadership. A one-size-fits-all central policy is not appropriate for car parking.
"However we understand the cost burden on carers that might have to make frequent visits so we encourage hospitals to look at the discounts they can offer."
Katherine Murphy, chief executive of the Patients Association, said: "The exorbitant fees that are often charged by hospitals for car parking can cause a great amount of distress to patients and their families, who very often cannot afford to pay such prices.
"While we welcome this attempt to remove charges for carers, the reality is charging for parking in hospitals is a tax on the sick.
"Whilst we acknowledge that hospitals have to generate revenue, this should not be done at the expense of patients."
George McNamara, head of policy at the Alzheimer's Society, said: "Unpaid dementia carers prop up our health and social care system.
"This silent army of husbands, wives, sons and daughters spend 1.3 billion hours a year providing care - saving the state an estimated £11.6 billion.
"It's totally unacceptable that they should have to worry about footing excessive hospital car parking charges on top of this."