In an interview with Vox, the social network CEO spoke anecdotally of a moment when Facebook’s software had detected messages being sent through Messenger that were trying to incite harm in Myanmar and then as a result, the company had been able to intervene.
“I remember, one Saturday morning, I got a phone call and we detected that people were trying to spread sensational messages through — it was Facebook Messenger in this case — to each side of the conflict, basically telling the Muslims, “Hey, there’s about to be an uprising of the Buddhists, so make sure that you are armed and go to this place.” And then the same thing on the other side” he said.
“So that’s the kind of thing where I think it is clear that people were trying to use our tools in order to incite real harm. Now, in that case, our systems detect that that’s going on. We stop those messages from going through. But this is certainly something that we’re paying a lot of attention to.”
When asked by HuffPost how the company scans everyone’s messages it replied saying that it was using the same algorithm that’s found on Facebook and would only be scanning for images or links.
Facebook went on to confirm that it does not read text within your messages, and would only do so if a message had been flagged to a moderator.
The system has been designed to crack down on inappropriate or illegal imagery such as child exploitation and to also prevent the spread of fake news among communities.
“For example, on Messenger, when you send a photo, our automated systems scan it using photo matching technology to detect known child exploitation imagery or when you send a link, we scan it for malware/viruses. Facebook designed these automated tools so we can rapidly stop abusive behaviour on our platform.” explained the company in a statement.
Finally Facebook confirmed that its scanning technology was not used in any way to inform advertising.
Facebook is able to scan the content of your messages on Messenger because it doesn’t use end-to-end encryption by default. Instead Messenger has an opt-in setting called Secret Conversations that enables end-to-end encryption however you have to activate that for each message individually.
In case you’re wondering if any other companies might be scanning your messages, both WhatsApp, Apple’s iMessage and Signal all use end-to-end encryption by default so they couldn’t read your messages even if they wanted to.