The elderly man was shut out on a second-floor balcony while babysitting the little boy in Munich.
He reportedly launched into a series of 'gymnastic exercises' in an attempt to keep warm while trying to persuade the toddler to open the door.
A spokesman for the Munich fire department told The Local that they didn't quite know why the man didn't yell to passersbys for assistance.
"I don't understand why he didn't shout out for help, into the street or courtyard or something. I'm sure I would be shouting a lot if I were stuck out in the cold for that long," the fire department worker said.
"We are not sure whether the little boy thought he was playing a game, or whether he was unable to open the door."
The pensioner was said to be wearing slippers and lounge clothes when he popped out on to the balcony to have a cigarette. He eventually got the little boy to pass him a phone through a slightly open window and called for help. When the child's parents returned, they found the door key stuck in the lock from the inside, and had to call the fire brigade to let them in.
The grandfather was admitted to hospital suffering from the effects of cold.