A number of glaring defensive errors contributed to another heavy Spurs loss against a top four side, despite an encouraging first 45 minutes.
Samuel Eto'o intercepted Jan Vertonghen's backpass in the 56th minute to put Chelsea ahead and Eden Hazard doubled the lead four minutes later after Younés Kaboul was sent off for supposedly upending Eto'o in the area.
Sherwood was appointed Spurs boss after the club's 5-1 home thrashing at the hands of Liverpool in December, and Manchester City won by the same scoreline at White Hart Lane in January. Sherwood was left demoralised by the latest capitulation.
"Too often. Too often. And again today, against the big sides," Sherwood said of Tottenham's late collapse which saw Demba Ba strike in the 88th and 89th minute. "It's very disappointing to see.
"It's difficult, 2-0 down with 10 men at Chelsea, you're not gonna win the game, but you expect to see a little bit more gut, a little bit more pride. I don't want to be the only one who shouts at them, I think they need to dig each other out every now and again.
"First half, we were alright, nice and solid enough, created a few chances of our own," Sherwood added. "And I said to the boys at half-time the longer we're in this game the better it is for us because Chelsea will come out and we'll be able to catch them on the counter-attack. But it all went Pete Tong after the goal. I'm more disappointed at the capitulation."
On the controversial penalty decision which effectively settled Chelsea's win, Sherwood suggested Tottenham could appeal Kaboul's dismissal.
"Anyone can slip up and make a mistake for the goal, and I appreciate the referee has got a very tough job and he's got to make that call but it went against us, that one," he added.
"I think it's a soft decision because I think it's not a penalty, so the sending off comes with it. We'll have to look at it a few more times. It's a shame the referee doesn't get that opportunity."
Sherwood admitted he had "no idea" if he would make wholesale changes to a squad likely to miss out on Champions League football again, providing he is still in charge of Tottenham next season. Holland coach Louis van Gaal has been vocal in his desire to manage in the Premier League and claimed Spurs had approached him earlier in the season.
The former Ajax, Barcelona and Bayern Munich manager will leave his role with the Dutch squad after the World Cup, which could cut Sherwood's 18-month contract prematurely short.
Sherwood's selection was described by former Tottenham striker Gary Lineker as a "lucky dip", as several players started out of position in a peculiar line-up which sorely missed the creative Christian Eriksen.
Tottenham managed just two shots on target throughout the game and José Mourinho claimed the visitors "had control but didn't scare us".
"I've got no reaction to it," Sherwood responded when Lineker's tweet was brought up, along with Kyle Walker's role on the right wing.
"He done alright first-half, didn't he? You can't start picking holes now. Gary's right about the second-half performance. Someone slips over and the referee gives them a penalty. All of a sudden, Gary's right."
Sherwood's anger was briefly quelled when Tottenham's central defensive issues were mentioned, since Michael Dawson, who came off injured, could join Kaboul on the sidelines for Arsenal's visit next week.
"Where do you play mate?" he asked.