On Wednesday, the Peep Show star announced he was withdrawing from the competition.
The actor and comedian, who underwent major heart surgery two years ago, said that Strictly “and its demanding schedule” had begun to take their toll on his recovery.
Speaking at the Henley Literary Festival last week, Robert opened up about his experience of being on the show.
“It is brutal. It is an absolutely weird show,” he said.
“It is not like it is an unpleasant environment, but they are long days and it is a long time to be nice.”
Robert, who was partnered with Dianne Buswell, admitted that trying to learn the dance moves in such a short space of time often left him on the brink of tears.
“Last week I could not get it at all and I was almost weeping as it is so intense,” he said.
“It’s like, ‘I have got to do this on Saturday. I am never going to do this bit’. Dianne simplified it for me.”
Robert also spoke about the “weirdness” of constantly having cameras around, even during training.
“You turn up and there is this reality to it,” he said.
“There is always a camera and there is a camera in the rehearsal room. But I trust the people as they are only there to make you look good, but it is a bit weird having this thing of cameras all the time.”
In a statement following his Strictly exit, Robert said he had “bitten off more than I could chew”.
He said: “I’m extremely sorry to announce that I’m withdrawing from Strictly Come Dancing due to ill health.
“Two years ago I had open heart surgery and although I believed I was fit enough to take on Strictly and its demanding schedule, it became clear that I had bitten off more than I could chew for this stage in my recovery.”
Strictly continues on Saturday night at 7pm on BBC One.