Family court judge Patricia Pearl said the 18-month-old would benefit from starting a new life in South Africa after details emerged of the father's infidelity when his wife was four months pregnant.
The judge said the mother 'had every reason not to trust the father'.
At the London hearing, Judge Pearl ordered that no-one could be identified but said the woman was a radiologist and the man worked in customer relations.
She told how the man and woman were both from South Africa and had been together since they were teenagers.
They had moved to the UK a decade ago and married in 2008. In late 2012 the woman had become pregnant - and had announced the news on Christmas Day.
In the spring of 2013 the man had started a relationship with a woman at work, who had two children.
The couple had separated about two weeks after their baby was born and the judge commented that the father had made some 'poor decisions'.But she they were not matters which had weighed against him when she was deciding what was best for the toddler.
The judge concluded that the woman would have more support and a better chance of career advancement in her South African homeland.
She said the man could afford to visit South Africa and concluded that the little boy would gain more than he would lose.