The Labour leader said it was "one of the perils" of talking for more than an hour without notes, but insisted that the whole address was focused on Britain's economic plight and he had touched on immigration too.
Chancellor George Osborne was among those who were quick to criticise Miliband, saying it was "extraordinary" that he did not mention the deficit. Tory MP Henry Smith said the Labour leader had "no plan" for the economy.
Ed Miliband didn't mention the deficit once. Extraordinary. If you can't fix the economy you can't fund the NHS
— George Osborne (@George_Osborne) September 23, 2014
Unite union general secretary Len McCluskey said Mr Miliband's failure to talk about the economy had been a "glaring omission".
Asked if he had forgotten parts of his 65-minute no notes speech, Miliband told ITV1's Good Morning Britain: "Absolutely yes.
"It's not really about memorising the speech. What I try and do is I try and write a speech and then I use it as the basis for what I want to say to the country.
"In a way I could just stand there and read out a speech that's been prepared earlier. I like it as a way of engaging with people. And, of course, it's one of the perils of it that there are bits that get left out, bits that get added in. It sort of comes with the territory."
Miliband said: "The speech was all about the economy. The speech was all about how we change our economy so it works for working people again."
Miliband insisted he had talked "very clearly" about Labour's plans on immigration as well.
"I didn't pluck a number out of the air for what should happen to immigration, but, you know, the prime minister has done that," he added. "He made a promise, he made a false promise to get immigration into the tens of thousands and it didn't happen."
In the original version of the Labour leader's speech, he had been set to warn his party "there won't be money to spend after the next election."
He was going to tell activists: "Labour’s plan is based on a tough new approach. Eliminating the deficit as soon as possible in the next parliament. Getting the national debt falling. And no proposals for additional borrowing. We will get the deficit down."
He was also going to tell his party that immigrants "have a responsibility to learn English and earn their way", adding: "And employers have a responsibility not to exploit migrant workers and undercut wages."
Miliband's deficit omission will be politically awkward as shadow chancellor Ed Balls warned members that he would have to make "unpopular decisions" and pledging "fiscal responsibility in the national interest".
Business groups expressed alarm at Miliband's lack of "economic narrative". John Cridland, head of the Confederation of British Industry, told Radio 4's Today Programme: "I think what was missing in the speech will worry business."
He added: “There really wasn’t an economic narrative. We’d heard it the day before from Ed Balls, but I think you expect to hear it from the Leader of the Opposition: how do we create the wealth that we then distribute – that was missing and I think it should have been [included] in an hour-long speech.”
In an attempt to bolster Labour's economic credentials, Balls said the party would be "straight" with the public by admitting that it would make real-terms cuts, and that he would "not spend money we cannot afford".
The remarks Ed Miliband decided not to deliver...
The Tories put out an online poster accusing Miliband of avoiding "hard truths" by forgetting to talk about the deficit.
Miliband gives an 80 min conference speech, the last before a G.E. & FORGETS to mention the deficit & immigration! pic.twitter.com/XVARpLaAym
— Kris Hopkins (@khopkinsmp) September 23, 2014
Seems Miliband forget passage in his speech that says how he’d reduce the deficit. Speaks more eloquently than anything he actually said.
— David Jones (@DavidJonesMP) September 24, 2014
"If there is one thing we learned from story time with Ed is that this episode of Jackanory is not going to end well," Ukip leader Nigel Farage wrote on Twitter. Daily Mail journalist Andrew Pierce quipped: "Did he forget to tell his brother he was running for leadership?"
So Miliband forgot to mention the deficit which Labour created. Did he forget to tell his brother he was running for leadership
— Andrew Pierce (@toryboypierce) September 23, 2014
It sounded like a normal teleprompted political speech. And if he forgot bits it's *worse* than doing a normal teleprompted political speech
— Andrea Mann (@AndreaMann) September 24, 2014
Others on Twitter did not seem so fussed about Miliband's speech, with one user calling the issue "trivial".
— Victoria Hunt (@tori_tweets) September 24, 2014
— william morton (@bykerbill) September 24, 2014
Labour has been trying to repair its credibility on the economy as senior figures worry that the party's failure at the 2010 general election to talk about cuts led voters to believe it was not serious enough about the deficit.
Balls and Miliband have also regularly trailed George Osborne and David Cameron in polls over who is most trusted to run the UK economy, with a recent poll putting the lead enjoyed by the Tories on this at 25%.