David Cameron has called for a review of the way the prison system treats pregnant women after figures showed 100 babies spent time living behind bars last year.
The Prime Minister said it was "absolutely terrible" to think of infants spending months or years within prison walls and the Government will look at alternatives to custody for women with babies.
He also confirmed that a pilot scheme to track offenders by satellite will launch later this year.
Mr Cameron, who will give a landmark speech on prisons policy tomorrow, said: "It is absolutely terrible to think that some babies are spending the earliest months – even years – of their lives behind bars.
“Prison staff do their best to make these environments pleasant. Some units even have special sensory rooms, so that babies can see colours, sights and sound – even nature – that they wouldn't ever otherwise see inside the grey walls of a jail.
“There are actually women in these prisons who were born in the same prison 20 years earlier, and then have ended up there later as criminals themselves.
“Think of the damage done to the life chances of these children."
At present, when a pregnant woman gives birth while serving a custodial sentence, she is usually transferred to a prison’s mother and baby unit (MBU) for an initial period.
MBU policy requires that these units should be available to accommodate babies up until the age of 18 months.
Mr Cameron said: "It’s time to think seriously about whether this is the right approach.
“We’ve got to break this cycle. So I want us to find alternative ways of dealing with women prisoners with babies, including tagging, problem-solving courts and alternative resettlement units.”
Downing Street said alternatives to custody would not be appropriate in every case and in some circumstances it could be in the best interests of the child for both mother and baby to be together in prison.
The Government also announced that a major pilot of satellite tracking technology will launch later this year.
The hi-tech tagging system could be used to track the movement of non-violent offenders, potentially changing how prisoners are released on licence or the way community sentences are enforced.
The technology will be rolled-our right across the country by 2020.