The Second Conversational Intelligence Challenge (ConvAI2)

December 12, 2019


We describe the setting and results of the ConvAI2 NeurIPS competition that aims to further the state-of-the-art in open-domain chatbots. Some key takeaways from the competition are: (i) pretrained Transformer variants are currently the best performing models on this task, (ii) but to improve performance on multi-turn conversations with humans, future systems must go beyond single word metrics like perplexity to measure the performance across sequences of utterances (conversations) – in terms of repetition, consistency and balance of dialogue acts (e.g. how many questions asked vs. answered).

Download the Paper


Written by

Emily Dinan

Alexander Miller

Arthur Szlam

Douwe Kiela

Jack Urbanek

Jason Weston

Joelle Pineau

Kurt Shuster

Ryan Lowe

Alan W Black

Alexander Rudnicky

Iulian Servan

Jason Williams

Mikhail Burtsev

Shrimai Prabhumoye

Valentin Malykh

Varvara Logacheva

Yoshua Bengio


The Springer Series on Challenges in Machine Learning

Related Publications

February 13, 2024



IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation

Luke Melas-Kyriazi, Iro Laina, Christian Rupprecht, Natalia Neverova, Andrea Vedaldi, Oran Gafni, Filippos Kokkinos

February 13, 2024

August 10, 2023



Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation

Samaneh Azadi, Akbar Shah, Devi Parikh, Sonal Gupta, Thomas Hayes

August 10, 2023

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.