The Deputy Prime Minister said he understood why the supermarket apologised to 26-year-old Jo Clarke, who was ordered to end her conversation by the woman staff member.
But he admitted that it drove him "round the bend" when people constantly had their "noses glued" in smartphones and iPads.
Mr Clegg was asked about the incident in Crayford, south east London, during his weekly LBC radio phone-in.
"I strongly suspect I have spoken on the phone in a queue. I like to think I probably wouldn't do that at the front of a check-in queue," the Liberal Democrat leader replied.
"If the customer was not responding, then the check-out woman couldn't do her job and she's perfectly entitled to ask 'Well, do you want it or not?'.
"I have a sneaking sympathy for her. But I understand Sainsbury's have to be on the side of their customers."
Mr Clegg said he and wife Miriam try to avoid using phones and iPads at the dinner table with their three sons - although he conceded it was not a "rigid" rule.
"I have sat in numerous meetings where people don't look each other in the eye - they drop in and out of the conversation," he said.
"It drives me round the bend. I have an old-fashioned view that people are supposed to talk to each other. They don't - they just mumble at each other."
Sainsbury's was forced to apologise on Wednesday after the incident, although the assistant has received some support from those who have read about her stance.
Miss Clarke said: "I was standing at the foot of the till waiting to bag my shopping up, yet the lady on the checkout was just staring at me, when I stopped my conversation and said 'Is everything OK?'.
"'She said 'I will not check your shopping out until you get off your mobile phone'.
"I ended my call swiftly and said to the lady on the checkout 'Apologies, I didn't realise that it was Sainsbury's policy that you are unable to use your phone at the checkout', and she said 'Well, you learn something new every day'."
Miss Clarke, from Crayford, said she checked with the customer service desk on her way out of the store on June 24, and was told it was not company policy.
A Sainsbury's spokeswoman said: ""We have apologised to Ms Clarke. It isn't our policy to not serve customers who are using a mobile phone."