And bosses at the Beeb already have an idea of who they want to replace him in the role, reports The Sun.
Matt - who will begin filming the 50th anniversary episode next month - is understood to not want to renew his contract, which expires in November, so he can focus on cracking Hollywood.
Speaking on Jonathan Ross's show last week, Matt said: “I am very happy doing ['Doctor Who']. I do the anniversary special, then the Christmas special. At the moment it’s 2013 and we will see what 2014 holds.”
Matt's star has continued to rise since signing up to play the Time Lord in 2009 for £200,000 a year. It is thought he now earns five times that amount.
He has just finished filming 'How To Catch A Monster' with Ryan Gosling, which is due for release next year and also has his eye on a role in the next James Bond film.
Last night, the BBC said in a statement: “Sorry folks but even we don’t know what’s going to happen at Christmas. It’s not been written yet! But Matt loves the show and is to start filming the unmissable 50th anniversary, and the new series starting on Easter Saturday.”