“I was quite honoured to be honest,” he told Capital FM about the rumours. “It's a big job. Would I say no? I don't know. I haven't been offered anything.”
However, despite being up for the job, he added it would depend on what his boss Simon Cowell would have to say about him judging The X Factor's biggest rival.
“To be honest, I don't think it's my decision,” he admitted. “I think it will be more down to Simon and The X Factor's decision if they let me do it.”
'The Voice' coaches
He added: “I don't think I want Simon Cowell as an enemy – so I don't think that's probably going to happen.
"If there was an offer on the table, you'd definitely consider it.”
Olly isn't a stranger to TV, having hosted two series of 'The Xtra Factor' with Caroline Flack.
Suggested For You
Get top stories and blog posts emailed to me each day. Newsletters may offer personalized content or advertisements. Learn more