In a shock victory, the former Shadow Justice Secretary got 58.9% of the vote, compared to the 41.1 per cent received by Dame Tessa.
The former Culture, Media and Sport Secretary, who served under Tony Blair from 2001 to 2007, was predicted to win but appears to have suffered from a surge of left-wing activists who joined the party to vote for Mr Corbyn in the leadership race.
Dame Tessa congratulated Mr Khan but left the event in tears.
Mr Corbyn is expected to be crowned Labour leader tomorrow in a further sign that the Blairite wing of the party is out of favour among activists.
The Tooting MP, one of the the 35 Labour MPs to nominate Mr Corbyn for the party leadership, also saw off Diane Abbott and Tottenham MP David Lammy to win.
Despite nominating the Islington North MP, Mr Khan voted for Shadow Health Secretary Andy Burnham.
In his victory speech, Mr Khan said he was “overwhelmed and deeply humbled” by the nomination.
He later said that he was not 'tribal' and said that his three successive victories in the Tory-facing marginal seat of Tooting proved he could win Conservative votes.
Asked by The Huffington Post UK how important the backing of Ken Livingstone and supporters of Mr Corbyn had been, he replied: "I've been blessed by having endorsements from all sorts of great people. Ken Livingstone, Margaret Hodge, Oona King, Michael Cashman.
"That's been my campaign, how I've done my politics throughout. I'm not tribal. If you see people supporting me, it's people across the Labour spectrum. It's people who aren't even politicians. Bill Bailey, the very funny comic, was backing me, that wonderful boxer Amir Khan.
"What I'm pleased about is I've won every single section of the Labour vote: members, supporters, affiliated and registered. From day one, my campaign has been one where I've tried to create a movement.
"At each stage we've made sure we understand the rules of the game and win the selection. You've seen the youngsters involved in our campaign, never before involved in politics."
But Mr Khan said that he would also be prepared to disagree with a Corbyn leadership.
"I nominated Jeremy, but when I nominated him I made it clear I wouldn't be voting for him. I will work with whoever the Labour leader is. The number one issue for me is making sure I'm London's advocate. Sometimes that may mean there being tension between me and the Labour leadership. Other times we will be on the same side."
There was a 76.5 per cent turnout in the election, with 87,854 votes cast from 114,839 eligible voters.
Of the 26,291 votes cast by 'registered supporters' in the final round - those who joined Labour for £3 since the General Election - 17,179 were allocated to Mr Khan.
Dame Tessa was in the lead of votes from full members until the final round, when Mr Khan overtook her in that category as well - 24,983 for Mr Khan and 24,019 for Dame Tessa.
It has been reported that Mr Khan will stand down from his Tooting seat in order to focus on winning the 2016 Mayor election.
With a majority of just 2,842, the Tories will see the by-election as an opportunity to put the new Labour leader under immediate electoral pressure.
Mr Khan, an MP since 2005, will be hoping to win back control of the capital from the Tories, who won the Mayoralty in 2008 through Boris Johnson.
Mr Johnson, who won a second four-year term in 2012, is not contesting a third election, and it is likely that Richmond Park MP Zac Goldsmith will be the Tory candidate next year.
Liz Kendall yesterday seemed to serve the last rites on Blairism. But Khan defeat of Jowell today sees coffin laid into the earth.
— Paul Waugh (@paulwaugh) September 11, 2015
Labour's previous London Mayor, Ken Livingstone, is also a close ally of Mr Corbyn.
In 2000, Mr Livingstone won the Mayoralty as an Independent after Tony Blair blocked him becoming Labour’s official candidate.
By 2004, he was back in the Labour fold, with Mr Blair admitting his prediction that Mr Livingstone would be a disaster “has not turned out to be right.”
Mr Khan was born in the capital to Pakistani immigrants, and went on to become a human rights lawyer.
In 2008 he was only the second ever Muslim to serve in the Government when he was appointed Communities Minister.