The shift to generalized AI to better identify violating content

Addressing content that violates our Community Standards and Guidelines is one of the top priorities at Meta AI. Over the past five years, AI has become one of the most effective tools for reducing the prevalence of violating content, or the amount of violating content that people see on our platforms. AI systems have typically been single-purpose, each designed for a specific content type, language, and problem, like detectingmisinformation or flagging hate speech violations, and they require varying amounts of training data and different infrastructure. Groups of bespoke systems result in high compute resources and maintenance complexity, which slows the process of updating systems to quickly address new, evolving challenges. But today, one of the biggest challenges in our integrity work is to build not more bespoke AI systems but fewer, more powerful ones.

AI models that can combine signals across multiple systems help AI make new connections and improve content understanding. This also makes integrity systems more efficient by making better use of compute resources — which, crucially, allows us to respond more rapidly to new issues.

This year, we deployed a new cross-problem system that tackles three different but related violations: hate speech, bullying and harassment, and violence and incitement. It’s clear that these issues overlap -- bullying is often connected to violence and incitement, which can involve hate speech. By generalizing the AI across the three violations, our system has developed a broader understanding of all three separate problems, outperforming previous individual classifiers. This consolidation has helped reduce hate speech prevalence over the past six months, as reported in our Community Standards Enforcement Report. We use technology to reduce the prevalence of hate speech in several ways: It helps us proactively detect it, route it to our reviewers, and remove it when it violates our policies. We also saw a direct impact in how quickly we bring classifiers to new languages. While previous systems typically take months to create separate classifiers for each market, we replaced existing classifiers with our cross-problem systems in many markets within weeks, without needing additional hardware to run the new advanced models.

By replacing dozens of existing individual models with just a few consolidated ones, the system learns from training data across all problems, which helps cover gaps that a single-purpose model may have on its own. So, for example, if a model is good at detecting hate speech in Spanish but has less Spanish training data for bullying and harassment, our advances in multilingual understanding translate the content into a supported language to catch bullying in Spanish without having to train on it.

This work builds on our previous multimodal integrity system, which combines different systems across languages, modalities (like text, images, and videos), and violation types, to understand harmful content at a deeper level.

Of course, these types of generalized models were almost unimaginable just a few years ago. They’re only possible today thanks to recent AI research advancements that we’ve deployed into our production systems.

Something Went Wrong

We're having trouble playing this video.

Learn more

Historically, text, visual, and audio signals are processed through vastly different training data and architectures. To consume all signals simultaneously — and expand to multiple languages and multiple policy violations — our standardized PyTorch framework has been a crucial tool for transferring research to real-world systems. It’s what helped us develop Facebook AI Multimodal (FAIM) framework libraries, which enable multimodal content understanding across text, images, and video. We also created Whole Post Integrity Embeddings (WPIE), a service that’s been trained to identify violating content across content types, like posts, captions, and videos.

Large-capacity models, like Transformers, have been driving innovation in the AI space over the past few years. Theoretically, the larger the AI system, the more types of shared AI signals it can process. But larger models are both computationally intensive and rely on large labeled training data sets, which, in turn, can slow our progress. This is true in the broader AI industry but especially at Meta, where our integrity systems must scale to billions of posts per day. Usually, engineering efficiency gains come at the expense of prevalence reduction or technical innovation.

We’ve taken steps to democratize these innovations at Meta so that any integrity team can use advanced models with the most scalable, sustainable setup. Beyond FAIM and WPIE frameworks, Linformer has been important for deploying state-of-the-art models by quadrupling the length of text that our systems can process while shrinking the compute required. Our self-supervised learning research, such as XLM-R, is the go-to model for flagging harmful content across languages while reducing our dependence on resource-intensive fixed, labeled training data.

Our researchers and engineers are also currently exploring breakthrough work in a relatively new area of AI research called “few-shot” or “zero-shot” learning. This means building AI models that can learn to recognize something from just a small number of training examples, or even just a single example. It’s a very different approach from most of the AI systems in operation today, which require a lot of labeled training data before being able to learn a new task and operate reliably.

Increasingly, we’re collapsing tasks, domains, and languages into single, larger systems, which improves systems across multiple dimensions: increasing performance, decreasing system complexity, and boosting iteration speed.

More broadly, generalization is the path toward more intelligent AI systems that mimic the way humans learn. Rather than treating different tasks as completely separate, our brains can look at an object or piece of content and instantly make connections in ever-changing contexts. Teaching machines to do this well is one of the hardest and most important opportunities in AI.

As violating content continues to evolve and people look for new ways to evade our systems, we’ll continue our work to build more generalized AI systems that can adapt as needed to keep people safe on our platforms.