It was previously reported Gary was quitting to concentrate on a new album and tour with Take That in 2014.
Speaking at the premiere of One Direction's film 'This Is Us', Dermot told reporters: "If Simon comes back, the dynamic would be very different. Gary wouldn't be there - he'll be leaving at the end of this year.
"Simon is a great judge so we'd welcome him back, but Gary's been such a great head judge as well."
Simon has been linked with a return to the show, after ITV bosses pleaded with him to make a comeback as part of new contract negotiations to keep the show with the broadcaster.
Gary's departure means there will be at least three vacant seats on the judging panel for the 11th run of the show next year, as Sharon Osbourne has made it clear that her comeback is for one year only, while Louis Walsh has also confirmed he will be departing at the end of the new series.
However, the Irish judge revealed earlier this week he would reconsider his resignation if Simon was to return to the panel.
He told the Radio Times: "I've been doing it ten years. It's an awful long time to be on the show. I don't want to be pushed. I'd prefer to leave than be pushed. To leave on my terms.
"Now if Cowell came back, I might come back. The two of us, we would not stop laughing. If Simon called me up and said... but, oh well. Let's see what happens."
Series 1-3 (2004-2006)
The original line-up: Sharon Osbourne, Simon Cowell and Louis Walsh
Series 4 (2007)
All change: Dannii Minogue joins Simon Cowell and Sharon Osbourne, with Brian Freidman replacing Louis Walsh.
Series 4 (2007)
The return of the Walsh: Louis makes a comeback after Brian Freidman is axed from the panel after only a few days of auditions.
Series 5-7 (2008-2010)
The Cheryl era: Cheryl Cole replaces Sharon Osbourne, joining Simon, Dannii and Louis.
Series 8 (2011)
A mass exodus: Gary Barlow, Tulisa and Kelly Rowland join the only original remaining judge, Louis Walsh.
Series 9 (2012)
Schermazing: Nicole Sherzinger replaces Kelly Rowland, joining Gary, Tulisa and Louis.
Series 10 (2013)
She's BACK! Sharon Osbourne rejoins the panel after five years, replacing Tulisa and joining Louis, Gary and Nicole.
Also on HuffPost: