The Prime Minister has been rocked by a double blow, with Mr Reckless' switch to Nigel Farage's "people's army" and the resignation of Government minister Brooks Newmark over a sex scandal.
Mr Cameron insisted that only a Tory government could deliver a referendum on membership of the European Union and said the defection of Mr Reckless was "frustrating".
On BBC1's Andrew Marr Show he insisted he would renegotiate the UK's relationship with Brussels before the referendum promised by the end of 2017, with a focus on changing the immigration rules.
"If I don't achieve that it will be for the British public to decide whether to stay in or get out," he said.
But he added: "I have said this all my political life: if I thought that it wasn't in Britain's interest to be in the European Union, I wouldn't argue for us to be in it."
The Tory gathering in Birmingham has been rocked by the scandal involving Mr Newmark, who resigned after reportedly sending explicit pictures of himself online to an undercover newspaper reporter in a tabloid sting, and the loss of a second MP to Ukip.
Rochester and Strood MP Mr Reckless followed Clacton MP Douglas Carswell to Ukip, in a sign of the unease within Tory ranks.
Mr Cameron said he had "not specifically" been aware of Mr Reckless' intentions but "he very rarely votes for the Government and has made his views known".
The Prime Minister acknowledged that Mr Reckless' actions made a Conservative government "less likely" after 2015.
He added: "These things are frustrating and, frankly, they are counter-productive and rather senseless.
"If you want to have a European referendum, if you want to have immigration controlled, if you want to get the deficit down, if you want to build a stronger Britain that we can be proud of, there is only one option and that is to have a Conservative government after the next election."
The choice was between the Tories and Labour to form the next government, he said, "so to act in a way that makes the Conservative government less likely is senseless and counter-productive".
Mr Cameron rejected the suggestion he was "not a proper Conservative" and added: "Douglas Carswell and Mark Reckless are both people who, unlike me, they want to leave the European Union no matter what.
"I don't agree with that. I think we should have a real go at reform to get a better deal for Britain. But then there is this promise - and it is a complete and clear promise - of an in/out referendum by the end of 2017.
"So, even if you don't agree with my renegotiation strategy, I'm the only Prime Minister who is going to give you the chance to have a vote to stay in or get out of the European Union."
Mr Reckless insisted he was trying to do the "right thing" by his constituents and accused Cameron of breaking his election promises.
"I made a lot of promises to my constituents and I want to keep those promises on immigration, on the deficit, on political reform. The Prime Minister isn't keeping those promises, I want to do so and that's why I'm moving to Ukip," he told BBC1's Sunday Politics.
"We made all these promises in 2010 as Conservatives and they have been broken. David Cameron has had his chance, he hasn't kept his promises, I want to keep mine."
Mr Reckless was confronted with a message he left on Tory Party chairman Grant Shapps's voicemail the night before he defected in which he said he would be "very happy" to take part in a Conservative campaign event the next day.
"I sound a little more hesitant on that call than I usually do," he said, adding, "You can't discuss these things in advance. You have to make a decision and you have to move at a particular point in time."
He said that in standing down as an MP in order to contest the seat in as a Ukip candidate in a by-election, he was putting his political career "on the line".
"I'm being open, I'm being honest, I'm giving people a say, I'm trying to do the right thing by my constituents.
"Whatever the risk is to me personally, I think that's the right thing to do."