The blame of course cannot be entirely put onto the acting of Matt Smith. A major let down has been the writing. I am not one of these people who felt the stories were too complicated and the show needed to dumb down, if I wanted dumb/waste of time I'd watch The Only Way Is Essex.
What is far more important is that the Doctor remains a British archetype rather than conform to any preconceived physical form. He has been many shades of British eccentric; a tea drinking, jelly baby munching dandy, draped in cricket whites, tweeds and Edwardian velvet.