Officials told Associated Press that embers in a bag of discarded ashes started the blaze in Stamford, Connecticut that killed ten-year-old Lilly and seven-year-old twins Grace and Sarah Badger and their grandparents, Lomer and Pauline Johnson.
They believe they were removed from the fireplace to make it safe for Father Christmas to come down the chimney.
The girls' mother, Madonna Badger, escaped the fire along with a friend, Michael Borcina. They were both treated at a local hospital and released.
Fire officials have said the ashes were placed in or outside an entryway, near the rubbish.
The victims died of smoke inhalation.
One of the grandparents, Lomer Johnson, also suffered a blunt head and neck trauma, which resulted from a fall or being hit by an object.
One of the girls, found dead just inside a window, had been placed on a pile of books, apparently so her grandfather could reach in and grab her after he jumped out.
Instead, authorities say, he fell through the roof. Stamford police are helping fire officials investigate the blaze.