Announcing new research awards in NLP and machine translation

Facebook AI is launching three new open calls for research proposals in the fields of natural language processing (NLP) and machine translation. We strongly believe open research will accelerate progress in these areas, and we look forward to collaborating with the academic community. These three requests for proposals (RFPs) focus on research that can help improve how ML can be applied in production — in particular, to make it easier to implement state-of-the-art NLP, to provide translation services in more languages, and to make NLP systems more flexible and more robust. Facebook AI’s previous RFPs have produced important research in dialogue systems, speech and audio classification, and deep learning platforms for research.

Awards for these RFPs will be made in amounts of up to $80,000 per proposal, for projects up to one year in duration, beginning in August 2019. The winning work will be shared openly with the AI community so that everyone can benefit from the researchers’ insights.

Computationally efficient natural language processing

Natural language processing has seen significant advances in recent years enabled by pretraining text representations on large amounts of unlabeled data. Unfortunately, this adds large computational requirements, both at training and at inference time. This computational cost is the biggest barrier to applying such models in practice and gaining wider adoption in the industry. Furthermore, with the rapid development of mobile devices and end-to-end encrypted communication, it is important to be able to run NLP models directly on those devices.

This RFP is focused on making NLP models more efficient both at training and testing time. We hope this work will enable production applications of such models at scale and potentially make them directly runnable on mobile devices without losing the quality of models run on servers.

Research topics should be relevant to computationally efficient NLP. Topics can include but are not limited to:

• Model compression techniques, such as quantization, knowledge distillation, model pruning, etc.

• More efficient, modular, sparse architectures.

• Universal representations that can be cached (i.e., that don’t need fine-tuning).

• Non-parametric (lookup table-based) approaches, which can be more memory demanding but are more computationally efficient.

For more details and to apply, visit the Computationally Efficient Natural Language Processing RFP page.

Neural machine translation for low-resource languages

Machine translation has recently made significant progress with a shift to neural models and rapid development of new architectures such as the transformer. While neural machine translation is effective with high-resource languages, it is not yet effective with low-resource languages.

To that end, Facebook AI invites the academic community to respond to an open call for research proposals in neural machine translation for low-resource languages. This is a follow-up to last year’s RFPs in the same area. Recipients of the awards will be expected to contribute to the field of low-resource machine translation through innovative approaches to obtain strongly performing models under low-resource training conditions.

Research topics should be relevant to low-resource machine translation, which includes but is not limited to:

• Unsupervised neural machine translation for low-resource language pairs.

• Semisupervised neural machine translation for low-resource language pairs.

• Pretraining methods leveraging monolingual data.

• Multilingual neural machine translation for low-resource languages.

In addition to launching this request for proposals, Facebook AI would like to encourage the academic community to continue to consider low-resource languages by participating in this year’s Workshop on Machine Translation (WMT) tasks, Machine Translation of News, and Parallel Corpus Filtering for Low Resource Conditions.

For more details and to apply, visit the Neural Machine Translation for Low-Resource Language RFP page.

Robust deep learning for natural language processing

While neural networks have achieved state-of-the-art results on various NLP tasks, one of the biggest open challenges is their robustness to changes in the input distribution and their ability to transfer to related tasks. Modern NLP systems interact with text from heterogeneous sources with distinct distributions, while the underlying linguistic regularities may be shared across tasks. This presents several interrelated challenges:

• From an application perspective, these models need to produce a robust output at test time given diverse inputs, even if such input distributions have never been observed at training time. For instance, content on the internet can be characterized by informal language with a long tail of variations in terms of lexical choice, spelling, style/genre, emerging vocabularies (slang, memes, etc.), and other linguistic phenomena.

• From the machine learning perspective, we need theoretical and empirical understanding of the intrinsic behaviors of neural networks used in NLP tasks both at training and inference time. For example, how to derive formal verification of a model’s robustness for a specific task? What training objectives and optimization methods can improve robustness to adversarial input at prediction time? Given that neural models are trained on large amounts of data from heterogeneous sources, how is model quality affected by noise and/or bias in training data? At inference time, what are unbiased and robust evaluation protocols to assess whether the model has improved linguistic generalization capability?

Facebook AI invites the academic community to propose novel and robust methods to address the aforementioned challenges. Research topics should be relevant to understanding and improving the robustness of neural NLP systems (machine translation, question answering, representation learning). Recipients will be invited to attend a workshop in Menlo Park, California, in August 2020.

In addition to launching this request for proposals, Facebook AI would like to encourage the academic community to participate in a new shared task on machine translation robustness at this year’s Workshop on Machine Translation (WMT).

For more details and to apply, visit the Robust Deep Learning for Natural Language Processing RFP page.

Applications for these research awards close Friday, May 31, at 11:59 p.m. PT. Questions can be emailed to academicrelations@fb.com .