Inside the Lab: AI breakthroughs that will power the metaverse

Building for the metaverse is the most ambitious long-term project Meta has ever attempted, and the experiences we’re envisioning are impossible to deliver with the software and hardware that exists today. Getting there will require major advances in almost every technology we work with.

A common link between many of these advances, from ultra-realistic immersive visuals to miniaturized devices capable of high-performance computing, is that they will be enabled by breakthroughs in artificial intelligence (AI).

Meta’s AI labs are already making these breakthroughs, as part of a long-term effort to create foundational technologies to enable the next era of computing. Today we are showcasing some of this work, such as our Builder Bot demo, which enables people to generate or import things into a virtual world just by using voice commands. And we’re announcing new research advances and sharing more about where we see progress heading in 2022 and beyond. Here’s some of what we are sharing in today’s Meta AI: Inside the Lab event, which you can watch here:

• Driving inclusion through the power of speech and translation: No Language Left Behind is working to create a single system capable of translating between all written languages, breaking down barriers for nearly half the world’s population, who can't access the internet in their preferred or native language today. We also aim to create a Universal Speech Translator, an AI system that provides instantaneous speech-to-speech translation across all languages, even those that are mostly spoken.

• A next-generation AI model for chatting with virtual assistants: Project CAIRaoke is a groundbreaking new approach to conversational AI (the technology that powers chatbots and assistants) that could one day enable people to have more natural conversations and interactions with their devices.

• A new resource for understanding how AI systems work: If you've ever wondered how Instagram ranks content in your feed, our new tool can help. The prototype AI system card tool we've developed outlines the many AI models that comprise an AI system and can help people better understand how these systems operate.

• New ways to bring diverse talent into AI: The AI Learning Alliance is making coursework on machine learning topics open to everyone and creating a consortium of professors at universities with large populations of students from underrepresented groups; these professors will teach the curriculum.

• Meta’s Chief AI Scientist Yann LeCun sketches a vision for building human-level AI: LeCun proposes that the ability to learn “world models” — internal models of how the world works — may be the key.

• Open-sourcing high-performance AI for recommendations: TorchRec is our library for building state-of-the-art recommendation systems for the open source PyTorch machine learning framework. These recommendation systems power personalization across many of our products.

As we build for the metaverse, we’ll need AI to do much of the heavy lifting that makes next-generation computing experiences possible. This means continuing to break ground in areas like self-supervised learning, so we aren’t dependent on limited labeled data sets, and truly multimodal AI, so we can accurately interpret and predict the kind of interactions that will take place in persistent, virtual 3D spaces with lots of participants. It also includes our efforts in building the world’s most powerful AI supercomputer, to ensure that we have the compute resources needed to drive the future of AI research breakthroughs. We are at the beginning of the journey, and today’s advances will provide a snapshot of what’s possible through the power of AI and open science.

Watch the Meta AI Inside the Lab event here.