Building richer, real-world datasets to push conversational research forward

January 23, 2020

Most dialogue research in AI is based on supervised learning from publicly available conversation datasets, which are static and can greatly differ in distribution from the environment in which an agent might be deployed. True progress in dialogue research requires real-world interactions directly between agents and people. That’s why we’ve released a new data collection and model evaluation tool called Beat the Bot, which is a game exclusively for researchers on Messenger.

Our goal with sharing this new tool is to provide researchers with high-signal data from live interactions instead of fixed language data. With user permission at the beginning of the game, the data collected will be open-sourced to facilitate new research directions for the entire community. We plan to continuously enhance this tool’s capabilities (for instance, adding image understanding) to help both improve our latest dialogue models and further explore dialogue research.