James, who left Strictly in 2013, reckons that “no one would miss” Craig if he was given the boot.
“The show doesn’t need Craig. He should go,” James told The Daily Star.
“If Craig was gone they wouldn’t notice any difference at all. The viewers wouldn’t miss him either. No-one would miss him.”
James added that Strictly has continued to be a huge success since the departure of Len Goodman in 2016.
“Len was a much bigger personality than Craig and the show is still huge without him,” he said.
“So it would still be a big success. Strictly will always be a juggernaut of a show no matter who you have on there. The format just works.”
James’ comments come after Craig revealed he had been asked to tone down his scathing comments by BBC bosses.
But the long-serving judge says he’d rather quit the dance show than go easier on the celebrity contestants.
He told The Sun: “Every year I get told to tone it down and I don’t listen. Of course I won’t be toning it down this year.
“I’m going to be just like I’ve been for 16 years. I’m not changing for anyone, if that’s the case I’d rather not judge it. It will never work darling, I’ve got too much of a big mouth.”
He was also forced to apologise to reigning Strictly champ Stacey Dooley for comments he made about her at the recording of this year’s launch show.
Although Craig’s joke about Stacey and her dance partner – and now boyfriend Kevin Clifton - never made it to air, they were caught on tape.
“Stacey was dragged up from the dance gutter, of course, if you sleep with your dance partner that helps,” he was reported to have said.
Craig later admitted he made a “huge error” in his quip and issued an apology to Stacey, saying: “I’m deeply sorry for the offence caused to Kevin and Stacey and everyone who works on the show.
“I recognise that what I said was hurtful, cruel and incredibly disrespectful. It was a huge error of judgment that I very much regret.”
“Did I (say sorry) or was that the BBC?” he said. “Oh yes I did. It was very grovel-ly. It doesn’t even sound like me. You’d think I’d done something really out of order.”
He continued: “A jokey remark was treated like it was the start of World War Three. I guess if some people are quite sensitive then they need the apology. So I just apologised, it’s quite simple.
“I thought it was quite funny, but it wasn’t taken that way.”