The Labour leader has urged the Prime Minister not to "shrink" from the public contest.
Scottish National Party leader Nicola Sturgeon and the Liberal Democrats' Tim Farron would sign up if the Prime Minister accepted the challenge, the Independent was told.
Jeremy Corbyn has challenged the Prime Minister to a televised annual debate
Yet a senior Downing Street source dismissed Corbyn's suggestion as a "desperate attempt by Labour to distract voters from the deep divisions that have left the party in turmoil".
"People are entitled to know more about their political leaders and to have their government held to account by the elected opposition in every way possible," Corbyn said.
"It is crucial that the Prime Minister and government are held to account, both inside and outside Parliament, throughout their period in office – not just at election time."
Corbyn, who puts questions from members of the public to Cameron every week in Parliament, said: "Democracy relies on the participation of the people. No political leader should shrink from the chance to engage more fully with the public and to test their arguments in debate.
"It is clear that televised debates can engage more people in politics, so we should seize the opportunity to hold them more regularly."
TV debates were first held at the 2010 election, when Cameron took on then prime minister Gordon Brown and Lib Dem leader Nick Clegg in three debates, which helped propel Clegg into public prominence.
Cameron's reservations resulted in tortured negotiations ahead of the 2015 election, which produced a head-to-head debate with Labour leader Ed Miliband, a seven-way clash also including the Lib Dems, Green Party, Ukip, SNP and Plaid Cymru and one between opposition leaders.
Sturgeon's performances in the election debates were very popular with voters.
Farron lambasted Cameron for being "two-faced about this issue for years".
A spokeswoman for the SNP said: "The televised debates earlier this year brought the election campaign alive and were far, far better for having a full range of participants – unlike in 2010 – which properly reflected the diverse range of political choices."
It is thought any annual debate would have to adopt the wider format including all major parties.
The call for a debate came after research by Leeds University found the debates increased viewers’ interest in politics by 30%.
A No 10 source said: "We haven't been approached about this proposal and would need to see more details. But the PM is happy to be held to account every week at Prime Minister’s Questions by MPs."