Speaking in a behind-the-scenes clip from the DVD release of 'This Is It', the pair admitted they used to "be at each other's throats" during their time on the ITV talent show back in 2010.
"We used to be at each other's throats on 'The X Factor'. He took it so seriously," Louis said.
Liam added: "We were two completely different characters. He was really loud and bubbly.
"There was one point where I literally wanted to whack him."
WATCH the video above for more.
One Direction's film 'This Is Us' is released on DVD and Blu Ray on 19 December.