And there is a very real – and sad – possibility that the grandparents might not stick around long enough to see their grandkids realise their potential.
This cheery (!!) thought struck me over Christmas when we visited our families. Both sets live 200-plus miles away and so visits are few and far between, which makes the time that we do re-unite all the more special.
But our latest trip felt all the more poignant, because it coincided with the second anniversary of my mother's death, my dad's 75th birthday and my father-in-law's 86th birthday on January 1st.
Spending time with my father-in-law felt extra important because as he sat at the dinner table next to his youngest grandchild – my son – I found myself becoming teary-eyed. My boy is five years old.
It is very unlikely his grandfather will be around when he gets to that curious stage of adulthood – you know, when you wished you'd asked your grandparents more questions about where they came from and what they'd experienced.
I looked at the wise old face of my father-in-law and thought back to all the things he'd been through.
He was born in Middlesborough – a thriving industrial town in those days – but lost his dad when he was nine and was sent away to a charity-funded boarding school near London during the Second World War because his mum thought it was for the best.
When he left school, he moved back up north and got a job as a bank clerk, rising to become branch manager, where he met his wife, now my children's grandmother, 10 years younger than him.
Together, they raised two girls, who have gone on to have five children between them. He's passionate about cricket and football, and in his heyday was actively involved in his local community, though the Rotary Club and Round Table.
When he retired from the bank, he and Grandma went round the world, visiting New Zealand, China, Kenya and South Africa. Now he busies himself tending to his garden and giving short-shrift to cold-callers who try to flog dodgy goods and services to the unsuspecting elderly.
His bones creak a little and his hearing is extremely poor, but he is – fortunately – very fit and razor sharp. But at 86, I can't help having dark thoughts about how long he's got left, especially given the fact that my mother died two years ago at the age of 74.
I'd love him to be around to see his grandchildren become teenagers. Even better if they became the kind of young adults who were as interested in the past as I never was until I hit my mid-20s. By which time, sadly, it was too late: I'd lost both my grandfathers by then.
As we sat around the house on my father-in-law's birthday, and his grandkids - all under 14 years old - wished him a Happy New Year as they presented him with a cake and cuddles, I couldn't help thinking: "How many more times will we do this?"
For the stroke of midnight that brings in each New Year is also the stroke that chalks up another year on my father-in-law's time on the planet.
When my wife mentioned the 81 age gap between the oldest and youngest members of the family, my mother-in-law piped up with typical chirpy optimism.
"Don't think of yourself as being 81 years older than Sam," she said to her husband. "Think of yourself as being 15 years younger than John."
"Who's John?" I asked.
"The organist at our church. He's just turned 100 – and he's as fit as a fiddle."
See you next year, then, father-in-law!