The Conservative Party conference began under a dark cloud after Mr Reckless announced he was defecting to Nigel Farage's party, triggering another by-election.
He followed backbencher Douglas Carswell, who defected and is on-course to win a by-election in his Clacton constituency to become Ukip's first elected MP.
At the conference, George Osborne is expected to bring forward an announcement about cutting a death tax on pensions to boost the mood and regain the initiative.
Writing in his weekly column in The Daily Telegraph, Mr Johnson said Conservative MPs deserting their party would simply risk handing power to Labour at next year's general election - when Mr Johnson is standing for parliament again.
He said he believed reports that more MPs were preparing to follow defector Mark Reckless and jump ship were "fanciful".
"Let me try to reason with any in my party, the folk who want to launch themselves headlong into the silage, because I genuinely think that they (if they indeed exist) must be utterly nuts," he wrote.
"There are only two people who are in a position to take the keys to Downing Street next May, and they are David Cameron or Ed Miliband.
"I know that you disagree vehemently with most of what Ed Miliband says and does - and so I must urge you not to allow the disaster of a Miliband premiership.
"If you really want to let this country sleepwalk into a Labour government, then that is your prerogative. You can close your eyes and let it happen.
"You kip if you want to; the rest of us are going to fight and win."
Though the Tories are behind in the polls and could lose in key marginals, Mr Johnson insisted his party could catch up with Labour as "people look harder" at what is being offered".
He wrote: "To put it another way, I was much further behind Labour in London in 2007, and went on to win the mayoralty in 2008.
"Tories tend to close the gap in the last few months as people look harder at what is being offered, and I believe in the next few months and years the news about this country and its prospects are going to get better and better – if we can keep Labour out."
Mr Johnson is mooted as potential leader of the party, if he is, as is likely, elected to parliament in 2015, but he defended Mr Cameron and endorsed repeated claims the prime minister has made of being the only party leader who will renegotiate Britain's membership of the EU or deliver a referendum if he cannot.
He wrote: "There is only one prime minister in the current EU who is capable of collating that support and turning it into a new treaty, and only one man who is going to be able to give the people of this country the vote on the EU that we have been deprived of for the past 40 years, and that man is David Cameron.
"I know this point is trite, and that you have now heard it a million times, but it happens to be true."