Research
NLP
Advances in Pre-Training Distributed Word Representations
May 7, 2018
Abstract
Many Natural Language Processing applications nowadays rely on pre-trained word representations estimated from large text corpora such as news collections, Wikipedia and Web Crawl. In this paper, we show how to train high-quality word vector representations by using a combination of known tricks that are however rarely used together. The main result of our work is the new set of publicly available pre-trained models that outperform the current state of the art by a large margin on a number of tasks.
AUTHORS
Research Topics
Related Publications
April 17, 2025
Human & Machine Intelligence
Conversational AI
Collaborative Reasoner: Self-improving Social Agents with Synthetic Conversations
Ansong Ni, Ruta Desai, Yang Li, Xinjie Lei, Dong Wang, Ramya Raghavendra, Gargi Ghosh, Daniel Li (FAIR), Asli Celikyilmaz
April 17, 2025
March 17, 2025
NLP
reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputs
Zhaofeng Wu, Michihiro Yasunaga, Andrew Cohen, Yoon Kim, Asli Celikyilmaz, Marjan Ghazvininejad
March 17, 2025
February 06, 2025
NLP
Brain-to-Text Decoding: A Non-invasive Approach via Typing
Jarod Levy, Mingfang (Lucy) Zhang, Svetlana Pinet, Jérémy Rapin, Hubert Jacob Banville, Stéphane d'Ascoli, Jean Remi King
February 06, 2025
February 06, 2025
NLP
From Thought to Action: How a Hierarchy of Neural Dynamics Supports Language Production
Mingfang (Lucy) Zhang, Jarod Levy, Stéphane d'Ascoli, Jérémy Rapin, F.-Xavier Alario, Pierre Bourdillon, Svetlana Pinet, Jean Remi King
February 06, 2025
April 30, 2018
NLP
Speech & Audio
Identifying Analogies Across Domains | Facebook AI Research
Yedid Hoshen, Lior Wolf
April 30, 2018
November 01, 2018
NLP
Computer Vision
Non-Adversarial Unsupervised Word Translation | Facebook AI Research
Yedid Hoshen, Lior Wolf
November 01, 2018
December 02, 2018
NLP
Computer Vision
One-Shot Unsupervised Cross Domain Translation | Facebook AI Research
Sagie Benaim, Lior Wolf
December 02, 2018
June 30, 2019
NLP
Variational Training for Large-Scale Noisy-OR Bayesian Networks | Facebook AI Research
Geng Ji, Dehua Cheng, Huazhong Ning, Changhe Yuan, Hanning Zhou, Liang Xiong, Erik B. Sudderth
June 30, 2019
Help Us Pioneer The Future of AI
We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.
Foundational models
Our approach
Latest news
Foundational models
Meta © 2025