In the academic year starting September 2013, 30 children aged four or under were expelled and 70 were suspended for a fixed period.
These toddlers were displaying aggressive behaviour such as biting other children and hitting nursery staff.
The figures were released by the Department for Education in response to a Parliamentary Question by Liberal Democrat leader Tim Farron.
"Teaching is facing a recruitment crisis, and violence in the classroom is part of the reason why," Farron said, according to The Mirror.
"We need to tackle the root causes of this crisis earlier on and prevent problem behaviour developing."
The figures uncovered were part of the Department for Education’s Permanent and Fixed-term Exclusions Report for 2013-14.
In all, 4,950 pupils were excluded or suspended from primary and secondary schools during the period examined.
Violent behaviour among other pupils and teachers was cited as the reason for 44.3% of exclusions.
Amanda Gummer, child psychologist and founder of Fundamentally Children said there may be many reasons why children are exhibiting violent behaviour.
"There could be undiagnosed conditions such as ADD or autism in young children that may not have been picked up and won't be supported," she told The Huffington Post UK.
"But there's also the issue of teachers being stretched and under-resourced in these classes.
"If you look at Ofsted ratios of the number of adults per children, the groups jump from under ones to under threes, then under eights."
Gummer explained this could mean four-year-olds have the same amount of adult supervision as eight-year-olds which could be very challenging for teachers.
"There also may be parenting issues where parents are reluctant to provide those consistent strong boundaries that children need between home and school," she added.
"If children aren't used to certain behaviours at home then applying them to the class is different.
"There is also the issue of home-school communication where really, the school needs to offer parents advice right from before they start school to help parents understand how they can help their children thrive."
Ultimately Gummer said if young children are being suspended or expelled, something needs to be done to prevent it from continuing throughout their education.
"There should be fairly comprehensive intervention for that child otherwise the danger is these patterns won't get addressed," she said.
"The teacher has to balance the needs of the individual with the needs of the class."
Siobhan Freegard, founder of video parenting site Channel Mum said we must remember toddlers simply don't have the language to express their frustrations so will take it out in other ways.
"Even the best behaved children occasionally nip or kick - it certainly doesn't mean they should be excluded from an educational setting which will help socialise them," she told HuffPost UK.
"For children this young who are persistently violent, there is always an underlying cause.
"They could be suffering abuse or neglect at home, or their parent or carer may not be able to cope.
"Again, exclusion isn't the answer and could well make the problem worse. Instead nurseries and schools should report serious concerns to the relevant authorities and have support mechanisms in place to help children overcome their anger and grow up into youngsters better able to cope."
A Department for Education spokesman said, according to the Daily Mail: "Permanent exclusions of pupils below the age of 11 remain very rare.
"We are determined that every child feels safe at school and is able to work and study hard without disruption. We are absolutely clear that no teacher should have to work in fear of violence or harassment."