Following his visit to the scene in which he read messages left by members of the public, Griffin said the "mainstream media" and turned "the religion to blame into the victim".
"So wet in Woolwich that couldn't tell if the Union Flag at half mast on barracks is now new. One they had up earlier was tattered disgrace," he added.
Drummer Rigby, from Langley, in Middleton, Greater Manchester, was from 2nd Battalion, The Royal Regiment of Fusiliers, had served in Afghanistan and was attached to the regimental recruiting team when he was hacked to death in broad daylight on Wednesday afternoon in an apparent terror attack.
Two suspects were shot and arrested at the scene and remain in police custody.
Tell MAMA, the charity which monitors anti-Muslim attacks, has logged 83 new incidents of threats or violence reported by Muslims to its helpline in the last 24 hours.
Julie Siddiqi, of the Islamic Society of Britain, told the BBC Radio 4 Today programme: "We can't allow the voices of Nick Griffin and the far right to become louder than ours in the coming days. They will say what we have to say and it gives us even more incentive to speak out and come together and not allow people like that to divide us as a country.
"The people who did this act yesterday do not speak in my name, do not speak for my community or the rest of the country. We have to come out with the strongest condemnation, which is what I'm seeing this morning.
"All of the Muslim organisations have come out with the strongest possible terms to say there is absolutely no excuse whatsoever, no justification for anything like this.
"This is one of the most shocking things I have seen in recent years and to have the people of Woolwich have to experience that so close to them, I just feel is absolutely horrific."
Nick Griffin arrives to lay flowers close to the scene
Members of the English Defence League (EDL) clashed with police in Woolwich on Wednesday evening following the attack.
More than 50 EDL supporters gathered at Woolwich Arsenal train station, throwing bottles at police, before moving to a nearby pub. The group then marched to the town centre where they were surrounded by police. No arrests are being reported.
The tearful family of murdered soldier Lee Rigby said today "our hearts have been ripped apart". The soldier's mother Lyn Rigby, 46, sat red-eyed, unable to speak through tears, clutching a teddy her son had bought his child.
Related on HuffPost:
Suggested For You
SUBSCRIBE AND FOLLOW
Get top stories and blog posts emailed to me each day. Newsletters may offer personalized content or advertisements.Learn more