BERGAMOT-LATTE Submissions for the WMT20 Quality Estimation Shared Task

October 01, 2020


This paper presents our submission to the WMT2020 Shared Task on Quality Estimation (QE)1. We participate in Task 1 and Task 2 focusing on sentence-level prediction. We explore (a) a black-box approach to QE based on pre-trained representations; and (b) glassbox approaches that leverage various indicators that can be extracted from the neural MT systems. In addition to training a feature-based regression model using glass-box quality indicators, we also test whether they can be used to predict MT quality directly with no supervision. We assess our systems in a multilingual setting and show that both types of approaches generalise well across languages. Our black-box QE models tied for the winning submission in four out of seven language pairs in Task 1, thus demonstrating very strong performance. The glass-box approaches also performed competitively, representing a lightweight alternative to the neural-based models.

Download the Paper


Written by

Vishrav Chaudhary

Lucia Specia

Paco Guzmán

Shuo Sun

Fred Blain

Lisa Yankovskaya

Marina Fomicheva

Mark Fishel



Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.