Miliband, who now lives in New York as the head of the International Rescue Committee aid charity, told an audience of 2,000 business leaders in London today that David Cameron's upcoming in/out EU referendum was a "risky and perilous venture".
And Mandelson said those advocating Brexit needed to "get real" and that continued EU membership was a "no brainer".
The former Labour cabinet ministers were both speaking Institute of Directors (IoD) conference at the Royal Albert Hall.
A ComRes poll for the Daily Mail published today showed the “Remain” side in the EU debate has a 19 point lead over those saying “Leave”.
Miliband said: "There is a case for an improved EU. I don’t see cause for believing that Britain can somehow be the Singapore of the North Atlantic that is going to live outside of the European continent."
"I am a great believer that we actually win arguments around the table rather than outside the room," he added. "It's far better to be in the room writing the rules than at the end of an mail or end of a fax being sent the rules."
The former foreign secretary said his "heart", "gut" and "brain" all told him that Britain was better off remaining inside the EU.
Miliband, who used much of his speech to address humanitarian aid issues, said the refugee crisis "makes the case for an improved EU" while "the last thing it argues for is a fragmented EU".
He said: "I hope that the next year or two years, depending on how long it takes, is a time for clarification and myth busting as well as for argument and debate."
In the wake of Labour's defeat, Miliband sharply criticised his brother Ed for having "turned the page backwards" with his leadership of the party. However he avoided discussing internal-Labour politics during his speech today.
Also speaking at today's event, Lord Mandelson said as a "county of only 64 million people" it would be a mistake for Britain to quit the union.
He said while the UK would be "notionally more independent" if we came out, the country would in practise be "more isolated and less influential".
Mandelson said: "I know anti-Europeans say we could have all the benefits and more outside the EU because the others need us more than we need them. I think we need to get real.
"Only 6% of their exports in Europe come to Britain. Compared to nearly 50% of ours to them. That is not a great starting point for negotiation. If we exited and asked for our trade privileges back it would be on their terms not ours."
The referendum, the former EU commissioner and business secretary said, was a "simple choice between a stronger economy and a Britain stronger in the world" verses "the risks and uncertainty of leaving and loss of influence that would come with our exit"
He added: "To my mind it’s a no brainer."
Mandelson was taking part in an EU debate with Lord Lawson. The former Tory chancellor, who has taken on a leading role among Conservative eurosceptics, said he believed the referendum would be a "very close run thing".
He accused Mandelson of having an "astonishing" lack of "self-confidence" in Britain's ability to function outside of the the EU. "I would like the EU totally reformed, the chances of getting hat are very slim indeed," he said.
Asked whether he still supported Britain joining the single currency, Mandelson replied: "Heavens no."
"My view is that moment came and went and it's not going to come back. I supported the government's policy at the time which was to keep the option open, but the option closed. And given everything that is happened since I don’t see the British public reopening it that’s mu view," he said.
Simon Walker, the director general of the IoD, warned the prime minister today that if the referendum was delayed until 2017 voters could choose to leave simply to "whack" the government.