Two-thirds of women feel embarrassed breastfeeding in front of strangers, while 44% feel awkward feeding in front of friends, a survey suggests.
One in five also feel embarrassed feeding their baby in front of their partner and 45% worry about feeding while their parents are around.
A quarter of 500 women polled for Public Health England (PHE) - 83% of whom breastfed - also did not want to breastfeed in front of their other children.
Nearly six out of 10 (59%) felt awkward breastfeeding in front of their partner's family while almost half (49%) also had a problem feeding in front of their own siblings and wider family.
But half of women surveyed said celebrities who breastfeed had inspired them to breastfeed themselves.
Figures show that 73% of new mothers start breastfeeding, but this drops to 44% when the baby is six to eight-weeks-old. Rates have not really changed in the last five years.
The Start4Life campaign from PHE has launched a new chatbot through Facebook messenger so that new mothers can seek support with breastfeeding 24 hours a day.
Women can open messenger and search for Start4Life BreastFeeding Friend or visit m.me/Start4LifeBreastFeedingFriend.
Viv Bennett, chief nurse at PHE, said: "Breastfeeding, while natural, is something that all mums and their babies learn by doing.
"Mums tell us that after the first few weeks breastfeeding becomes easier, so proper support is crucial at this time, which is where our breastfeeding friend is designed to help.
"We can all help women feel comfortable breastfeeding their baby wherever they are.
"Creating a wider culture of encouragement and support will help make a mother's experience all the more positive."
Jacque Gerrard, director for England at the Royal College of Midwives, said: "Getting infant feeding right will help give new-born babies the best possible start in life.
"Women need all the support they can get, particularly first time mothers.
"It is important that midwives and maternity support workers continue to promote breastfeeding.
"Any initiative that goes towards helping mothers start and sustain breastfeeding for longer is positive as we know the health benefits from being breastfed last a lifetime."