Obama described Reverend Clementa Pinckney, pastor of Charleston's Emanuel African Methodist Episcopal Church, which was targeted in a "hate crime", as a "good man".
He described those killed as "good people, decent people, God-fearing people. People so full of life and so full of kindness, people who ran the race, persevered. People of great faith."
He told the families of those murdered: "The nation shares in your grief. Our pain cuts that much deeper because it happened in a church."
The president also used the eulogy to call on America to front up to its problem with guns.
He said that for too long America has "been blind to the unique mayhem that gun violence inflicts on this nation", and called for action on gun control, not just talk.
The removal of the confederate flag from the country's capital was also touched upon. Obama said that, while "this flag did not cause these murders", the flag represents more than just ancestral pride.
He said removing the flag from the state's capital would not be an act of "political correctness".
Obama said: "For many, black and white, that flag was a reminder of systemic oppression and racial subjugation. We see that now.
"By taking down that flag," he said, "we express God's grace."
Article continues below slideshow:
Obama said taking the flag down would be an "acknowledgment that the cause for which they fought, the cause of slavery was wrong".
Obama never mentioned the name of the 21-year-old charged over the murders, Dylann Roof, name, only referring to "the alleged killer" during his eulogy.
The president said: "Blinded by hatred the alleged killer couldn't see the grace surrounding Revered Pinckney and that bible group."
He added: "The alleged killer couldn't imagine how the city of Charleston... how the state of South Carolina... and the United States of America would respond, not merely with revulsion at his evil act but with big hearted generosity and more importantly with a thoughtful introspection and examination that we so rarely see in public life."
After his speech, Obama tweeted how much he admired the grace shown by the victims' families, using the hashtag 'HateWontWin'.
People were moved by Obama's rendition of Amazing Grace.
What a privilege to listen to the whole of Obama's speech tonight. Filled with grace. Then his astonishing rendition of Amazing Grace.— Jon Trickett (@jon_trickett) June 26, 2015
@POTUS My goodness... Your Amazing Grace brought tears to my eyes... In Amsterdam.. Watching tv... Amazing... Thank you...— Eric Corton (@EricCorton) June 26, 2015
Like him or not. @BarackObama gave a great inspiring speech at church in Charleston, SC. Power of God's grace. Amazing grace.— Larry Kudlow (@larry_kudlow) June 26, 2015
After his rendition of Amazing Grace, @BarackObama may end this week by winning a Grammy, too.— Scott Simon (@nprscottsimon) June 26, 2015