He also said the polls should "never have been that close" and joked he would sue the polling companies for giving him nervous "stomach ulcers" in the run up to the vote last week.
The neutrality of the queen over Scottish independence became an issue in the runup to last week's vote.
After it was reported she was pro-unionist, she told people near Balmoral, her Scottish estate, to "think very carefully" before voting.
Cameron made the comments in New York to Michael Bloomberg, the city's former mayor, unaware he was being recorded.
"The definition of relief is being the prime minister of the United Kingdom and ringing the Queen and saying 'It's alright, it's okay'. That was something," he said.
"She purred down the line."
What he says next is inaudible for a few seconds, before he added: "But it should never have been that close.
"It wasn't in the end, but there was a time in the middle of the campaign when it felt…"
After another few seconds of the conversation being inaudible, he said: "I've said I want to find these polling companies and I want to sue them for my stomach ulcers because of what they put me through.
"It was very nervous moments."
The incident triggered an immediate backlash on Twitter, where people variously mocked him while former Labour spin doctor Alistair Campbell called him an "idiot" and a journalist said an apology was "sure to follow".
Taxi for Cameron. The Queen will not be amused but I am http://t.co/wn8jcEaFhO
— Kevin Maguire (@Kevin_Maguire) September 23, 2014
David Cameron saying the Queen purred down the phone when he told her it was a No vote sounds like 50 Shades of Eton.
— Scott Wilson (@HeartofFire) September 23, 2014
David Cameron overheard saying the Queen 'purred' when he told her the Scotland result. So she doesn't have a telly, then?
— malc mcgookin (@malcmcgookin) September 23, 2014
A Buckingham Palace spokesman would not confirm or deny whether the Queen purrs when she is happy, Sky News reports.
The palace also said it would not comment on conversations between the prime minister and the Queen.
Journalists are pouring over the tape and anticipating a whirlwind of public damage similar to when Gordon Brown was picked up on a hot mic calling a Labour voter "a bigoted woman".