Here's how we're using AI to help detect misinformation

Artificial Intelligence is a critical tool to help protect people from harmful content. It helps us scale the work of human experts, and proactively take action, before a problematic post or comment has a chance to harm people.

Facebook has implemented a range of policies and products to deal with misinformation on our platform. These include adding warnings and more context to content rated by third-party fact-checkers, reducing their distribution, and removing misinformation that may contribute to imminent harm. But to scale these efforts, we need to quickly spot new posts that may contain false claims and send them to independent fact-checkers — and then work to automatically catch new iterations, so fact-checkers can focus their time and expertise fact-checking new content.

From March 1st through Election Day, we displayed warnings on more than 180 million pieces of content viewed on Facebook by people in the US that were debunked by third-party fact checkers. Our AI tools both flag likely problems for review and automatically find new instances of previously identified misinformation. We’re making progress, but we know our systems are far from perfect.

As with hate speech, this poses difficult technical challenges. Two pieces of misinformation might contain the same claim but express it very differently, whether by rephrasing it, using a different image, or switching the format from graphic to text. And since current events change rapidly, especially in the run-up to an election, a new piece of misinformation might focus on something that wasn’t even in the headlines the day before.

To better apply warning labels at scale, we needed to develop new AI technologies to match near-duplications of known misinformation at scale. Building on SimSearchNet, Facebook has deployed SimSearchNet++, an improved image matching model that is trained using self-supervised learning to match variations of an image with a very high degree of precision and improved recall. It’s deployed as part of our end-to-end image indexing and matching system, which runs on images uploaded to Facebook and Instagram.

SimSearchNet++ is resilient to a wider variety of image manipulations, such as crops, blurs, and screenshots. This is particularly important with a visuals-first platform such as Instagram. SimSearchNet++’s distance metric is more predictive of matching, allowing us to predict more matches and do so more efficiently. For images with text, it is able to group matches at high precision using optical character recognition (OCR) verification. SimSearchNet++ improves recall while still maintaining extremely high precision, so it’s better able to find true instances of misinformation while triggering few false positives. It is also more effective at grouping collages of misinformation.

Another challenge in detecting misinformation at scale is that false claims can appear in countless variations over time. We’ve developed a set of systems to predict when two pieces of content convey the same meaning even though they look very different. (For example, they might have captions with completely different text that makes the same claim, such as “masks are dangerous” and “face coverings are not safe.” Or they might have different images that show the same subject.)

Moreover, identifying mutations is a highly contextual problem, given that two posts about similar concepts or entities can make very different claims. As you can see below, sometimes small changes in the content can be the difference between truth and misinformation.

We’re now introducing new AI systems to automatically detect new variations of content that independent fact-checkers have already debunked. Once AI has spotted these new variations, we can flag them in turn to our fact-checking partners for their review.

This system relies on several technologies, including ObjectDNA, which is based on Facebook AI research published earlier this year.

Unlike typical computer vision tools, which look at the entire image in order to understand the content, ObjectDNA focuses on specific key objects within the image while ignoring background clutter. This allows us to find reproductions of the claim that use pieces from an image we’ve already flagged, even if the overall pictures are very different from each other. Our AI also uses the LASER cross-language sentence-level embedding developed by Facebook AI researchers. This technique embeds multiple languages jointly in a single shared embedding space, allowing us to more accurately evaluate semantic similarity of sentences. It leverages both text and images, but also works for content that consists of only one or the other.

Deploying tools to detect deepfakes

We’re also taking steps now to make sure we’re prepared to deal with another type of misinformation: deepfakes. These videos, which use AI to show people doing and saying things they didn’t actually do or say, can be difficult for even a trained reviewer to spot. In addition to tasking our AI Red team to think ahead and anticipate potential problems, we’ve deployed a state-of-the-art deepfake detection model with eight deep neural networks, each using the EfficientNet backbone. It was trained on videos from a unique dataset commissioned by Facebook for the Deepfake Detection Challenge (DFDC). The DFDC is an open, collaborative initiative organized by Facebook and other industry leaders and academic experts. The dataset's 100K videos have been shared with other researchers to help them develop new tools to address the challenge of deepfakes.

In order to identify new deepfake videos that our systems haven’t seen before, we use a new data synthesis technique to update our models in near real time. When a new deepfake video is detected, we generate new, similar deepfake examples to serve as large-scale training data for our deepfake detection model. We call this method GAN of GANs (GoG) because it generates examples using generative adversarial networks (GANs), a machine learning architecture where two neural networks compete with each other to become more accurate. GoG lets us continuously fine-tune our system so it is more robust and generalized for dealing with potential future deepfakes.

These systems -- and the others described in the companion blog posts linked above -- are in production now to help us catch misinformation on our platform. As we have said in previous blog posts and as we tell ourselves every day, there’s much more work to do. But AI research is advancing rapidly and we are getting better and better and taking these new technologies and putting them to use quickly. We believe the new computer vision systems and language tools we’re developing in the lab today will help us do better at catching harmful content on our platforms tomorrow. It will take long-term investments and a coordinated effort from researchers, engineers, policy experts, and others across our company. But it will continue to be our priority and we’re confident we can continue to find new ways to use AI to better protect people.