Wav2vec
AI at Meta develops an artificial intelligence system for self-supervised, high-performing speech processing.
THE OPPORTUNITY
Building breakthrough speech recognition with AI
Today, speech technology is only available for a small fraction of the thousands of languages spoken around the world because traditional systems need to be trained on large amounts of annotated speech audio with transcriptions. Obtaining that kind of data for every human language and dialect is almost impossible.
Wav2vec works around this limitation by requiring little to no transcribed data. The model uses self-supervision to push the boundaries by learning from unlabeled training data. This enables speech recognition systems for many more languages and dialects, such as Kyrgyz and Swahili, which don’t have a lot of transcribed speech audio. Self-supervision is the key to leveraging unannotated data and building better systems.
Wei-Ning Hsu
Research Scientist, AI at Meta
RESEARCHER THOUGHTS
I don't want speech technology to only be accessible to people who speak English and who speak English without an accent. I want this to be accessible to more people. I want to support all the languages in the world.
Alexei Baevski
Research Scientist, AI at Meta
RESEARCHER THOUGHTS
The grand motivation behind all of this is to build systems that can learn similarly to how humans do and make connections between different pieces of data. To build models that can learn really good representations by simply observing the world around them.
Michael Auli
Research Scientist, AI at Meta
RESEARCHER THOUGHTS
How do you access information if you don't know how to read or write? You can speak. There are nearly 7,000 languages in the world. We know how to do speech recognition decently for maybe a hundred, and really well for maybe 10. If we can lower the bar to build speech technology for many more languages, then we can make information more accessible.
Why It Matters
Our researchers understand how essential speech recognition can be in everyday life, from asking a virtual assistant what the weather will be, to telling a GPS app to find the nearest gas station. But for those who don’t speak English or do so with a heavy accent, it can be difficult to use speech recognition devices. The wav2vec team believes that technology like this should benefit all people and not only those who are fluent in one of the world’s most widely spoken languages. That’s why our researchers have open-sourced the code and pre-trained models with the hopes to scale impact and enable other researchers to build better speech recognition systems that are not dependent on annotated data.
How It Works
The wav2vec model is trained by predicting speech units for masked parts of speech audio. It learns basic units that are 25ms long to enable learning of high-level contextualized representations. This enables us to build speech recognition systems that can outperform the best semi-supervised methods, even with 100 times less labeled training data. Our model learns a set of speech units, which are shorter than phonemes, to describe the speech audio sequence. Since this set is finite, the units encourage the model to focus on the most important factors to represent the speech audio.
Wav2vec learns from recorded speech audio and unpaired text, lessening the need for transcriptions. The self-supervised model segments the voice recording into speech units that loosely correspond to individual sounds. Training a generative adversarial network consisting of a generator and a discriminator network helps the model learn to recognize the words in the audio recording. Reducing our dependence on annotated data by using self-supervised learning is an important part of expanding access to speech recognition tools to many more people around the world.
DID YOU KNOW
Without any labeled data at all, it can achieve 5.9% Word Error Rate
WAV2VEC TEAM
Alexei Baevski, Steffen Scheider, Wei-Ning Hsu, Alexis Conneau, Henry Zhou, Ronan Collobert, Abdelrahman Mohamed, Michael Auli
Resources
Other Impact Projects
New ways to store renewable energy with AI
Facebook AI and Carnegie Mellon University’s Department of Chemical Engineering have joined to collaborate on the Open Catalyst Project.
AI & MRIs: for a better patient experience
AI at Meta and NYU Langone Health have developed a way to use AI for MRIs that need only a quarter of the raw data traditionally required for a full MRI.
Build the future of AI with us
If you have the passion, drive and experience to play in the sandbox that is AI, we just might be a great fit for you. We’re always looking for great talent to help us make a meaningful impact in science and on the world with AI. Sound like your kind of challenge? Then we’d like to meet you.
Foundational models
Latest news
Foundational models
Meta © 2024