Fans of the hit BBC Two show have picked up on the pair's flirty exchanges, but Paul has branded the claims 'rubbish' and revealed he's more sweet on baker Kimberley Wilson.
He told the Radio Times: "Personally I think (fellow contestant) Kimberley's far prettier. With all the love in the world, Ruby's not my type," he told the Radio Times.
"But she's a great baker, one of the most talented bakers who has walked in that tent."
The judge - who left his wife of 15 years amid an attempt to launch his career in the US and rumours of a growing friendship with his US co-star Marcela Valladolid - added: "My job is to judge what goes on the plate. Always has been, always will be.
"I don't judge the person - Mary will back me up on this - because it's not one judge doing this. It's two. And we both say the same thing."
Earlier this week, Ruby also denied she had the hots for the grey-haired chef.
"Not my thing, you know, for other people maybe that’s what they go for."
She added: "He’s a genuinely nice guy, he really is, and actually I quite like his style of baking as well because he’s really into bread and it’s something I enjoy.
"He has to play the bad cop on screen, but off screen he’s lovely."
Meanwhile, it has been announced that 'Great British Bake Off' is set to move from BBC Two to BBC One for its fifth series next year, after its huge ratings success.
Controller of BBC One, Charlotte Moore, said: I’ve championed 'The Great British Bake Off' from the very start and believe the time is absolutely right to bring the show to an even broader audience on BBC One.
"I’ve watched the series grow over the last few years and earn a special place in the nation’s hearts. It’s inspiring and warm storytelling at its best which celebrates the huge talent of enthusiastic and passionate bakers from all over the country.
"It’s been fantastic to watch it flourish on BBC Two and I can assure viewers I will continue to cherish it on BBC One,” she added.