The 77-year-old actress - who co-created the original series about an Edwardian London house and its owners and servants in the 1970s with fellow actress Jean Marsh - played Lady Holland in the BBC revival which aired last Christmas.
But Dame Eileen, who has been nominated for an Emmy Award for best supporting actress, is reportedly unhappy with the direction the scripts for the new series are taking, and has pulled out, meaning scripts have been rewritten to edit out her character.
The BBC said in a statement: "It's with much sadness that we say goodbye to her wonderful character, the straight speaking mother-in-law Lady Holland.
"However, we respect her decision and we will be announcing new star casting soon."
The new series, starring Marsh, Keeley Hawes, Ed Stoppard and Claire Foy is due to start filming in October for broadcast next year.
Writer Heidi Thomas said when the new series was announced: "The Holland family and their servants are set for an extraordinary year and I am thrilled to be their guardian."
Dame Eileen did not appear in the original show, but helped create it with Marsh, who is the only cast member to have appeared in both series.
Suggested For You
SUBSCRIBE AND FOLLOW
Get top stories and blog posts emailed to me each day. Newsletters may offer personalized content or advertisements.Learn more