He said cyber bullying was a huge problem and that children need to be taught 'respect for other human beings'.
Mr Gove spoke out after Hannah Smith, 14, hanged herself after apparent abuse about her looks on the Ask.fm chatsite. In a speech at Conservative Party headquarters, he said: "When it comes to tackling bullying, it's not just fixing a technology, it's fixing what's in people's hearts.
"We have to ask why among some young people there is a desire to show a degree of viciousness and personal cruelty which is horrendous."
He called for schools to ensure they teach kids 'the right values'.
Latvia-based Ask.fm is introducing a button to report abuse, recruiting more monitoring staff and asking users to register.
But Mr Gove suggested that people who run sites used by cyber bullies should be named and shamed.
"One of the things we can do is use effective publicity to shame those who are allowing that sort of activity to exist on sites," he said.
"But I do think we need to ask a series of questions – not just about how this technology can be used to inflict suffering on vulnerable young people. We also need to ask about the motivation of those young people who are themselves using these sites."
Mr Gove added: "It is certainly the case that technology creates a new arena where bullying can take place but at its heart when it comes to tackling bullying, it's not just fixing a technology, it's fixing what's in people's hearts."
Campaigners claim that Ask.fm is more worrying than other similar sites because it allows people to post messages in total anonymity. Users are allowed to join when they are 13 and there is no option to block other users, only to report them once messages have been posted.