Team Sky's leader in 2012, Wiggins barely hid his disgust when Froome launched an infamous unplanned attack on stage 11 to the Alpine ski station of La Toussuire four kilometres from the end.
Tension continues to linger between Wiggins and Froome
According to Yates, Wiggins sent him a text message which said: "I think it would be better for everyone if I went home."
Yates, in an extract from his new book carried in The Times, said: "He (Wiggins) was upset and felt like Froomey had stabbed him in the back after the discussion we'd had before the stage (where it had been agreed Froome would attack only in the last 500 metres).
"He couldn't understand why he'd gone back on the agreement, especially when everything was going so well.
"There were only four kilometres left at that point and Brad went on to win by three minutes, so the maths say that he would still have been comfortably on top in Paris.
"However, his mental state was always fragile, and that psychological blow could have been a knockout one."
Yates left Team Sky in October last year after speaking with team principal Dave Brailsford as part of his policy of zero tolerance to doping.
The 53-year-old denies ever doping despite being on the same teams as the disgraced Lance Armstrong and Bjarne Riis.
Yates added: "Dave said to me, 'I am under pressure and having to answer questions about you. I said: 'OK I'll walk. I am not interested. I don't like the fact that people think I am some sort of baddie. I can't be dealing with that. I'll walk.' End of story."