The debonair Sherlock actor had to punch the screen hardman who has starred in Bronson, Warrior and plays villain Bane in the new Batman film, and admits it may be his only chance to win a fight against Hardy.
Cumberbatch said: "It was very enjoyable.
"He wanted me to hit him harder and I had to point out that I needed my hands for the rest of the day. Because he can take a bit of punishing Tom. And it's probably, let's face it, the only time I'm going to get to really have a proper pop at him, in his current form. So it was good fun."
The 35-year-old British actor is currently filming a Tom Stoppard adaptation of Ford Madox Ford's novel Parade's End with Rebecca Hall and another of his Tinker Tailor Soldier Spy co-stars Stephen Graham.
Cumberbatch revealed: "We made a pact to work together again after we first met on this (Tinker Tailor) and we got on very well just on our brief day together. I adore the man.
"He's playing my best friend in an adaptation of Parade's End. It's fantastic. Susannah White's directing it for HBO and BBC and that's what I've just come from so I'm a bit cross-eyed with it."
Cumberbatch will also be heading to New Zealand in January to play the voice of Sauron and Smaug in The Hobbit opposite his Sherlock co-star Martin Freeman.
He said he was looking forward to it but added: "I understand I might not be seeing him because he's invisible. That's all I'm going to say about that."