AI at Meta presents Cicero
The first AI to play at a human level in Diplomacy, a strategy game that requires building trust, negotiating and cooperating with multiple players.
AI at Meta presents Cicero
The first AI to play at a human level in Diplomacy, a strategy game that requires building trust, negotiating and cooperating with multiple players.
“An agent that can play at the level of humans in a game as strategically complex as Diplomacy is a true breakthrough for cooperative AI.”
Yann LeCun
VP & Chief AI Scientist, AI at Meta
“An agent that can play at the level of humans in a game as strategically complex as Diplomacy is a true breakthrough for cooperative AI.”
Yann LeCun
VP & Chief AI Scientist, AI at Meta
By building CICERO, AI at Meta has created the first AI agent to achieve human-level performance in the complex natural language strategy game Diplomacy*. CICERO demonstrated this by playing with humans on webDiplomacy.net, an online version of the game, where CICERO achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.
This breakthrough rests in the achievement of combining two different areas of AI: strategic reasoning and natural language processing. The integration of these techniques gives CICERO the ability to reason and strategize with regard to players’ motivations, then use natural language to communicate, reach agreements to achieve shared objectives, form alliances and coordinate plans.
By building CICERO, AI at Meta has created the first AI agent to achieve human-level performance in the complex natural language strategy game Diplomacy*. CICERO demonstrated this by playing with humans on webDiplomacy.net, an online version of the game, where CICERO achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.
This breakthrough rests in the achievement of combining two different areas of AI: strategic reasoning and natural language processing. The integration of these techniques gives CICERO the ability to reason and strategize with regard to players’ motivations, then use natural language to communicate, reach agreements to achieve shared objectives, form alliances and coordinate plans.
Jump to:
RESEARCH
A grand challenge for AI
A game about people rather than pieces
Diplomacy is a seven-player board game that can be described as a combination of the board game Risk, the card game poker, and the TV show Survivor. Unlike many board games where you just need to be the best at moving pieces around the board, Diplomacy has a cooperative component — the only way to win is by working with other players to capture as much territory as possible. This coordination is achieved through natural language negotiation that occurs before every move in the game.
Why Diplomacy? Advancements in AI gameplay have long served as benchmarks for progress in AI. For decades, researchers have been building simplified variants of Diplomacy gameplay agents without natural language communication capabilities. But no one has ever attempted to build an AI agent that can negotiate with open-ended dialogue. Achieving this grand challenge of AI has simply been beyond the capabilities of what’s existed in AI, until now.
RESEARCH
A grand challenge for AI
A game about people rather than pieces
Diplomacy is a seven-player board game that can be described as a combination of the board game Risk, the card game poker, and the TV show Survivor. Unlike many board games where you just need to be the best at moving pieces around the board, Diplomacy has a cooperative component — the only way to win is by working with other players to capture as much territory as possible. This coordination is achieved through natural language negotiation that occurs before every move in the game.
Why Diplomacy? Advancements in AI gameplay have long served as benchmarks for progress in AI. For decades, researchers have been building simplified variants of Diplomacy gameplay agents without natural language communication capabilities. But no one has ever attempted to build an AI agent that can negotiate with open-ended dialogue. Achieving this grand challenge of AI has simply been beyond the capabilities of what’s existed in AI, until now.
A Look Under The Hood
How we built CICERO
Two central elements give CICERO its strategic and collaborative gameplay capabilities: a controllable dialogue model and a planning engine. Together they let CICERO (1) model other players’ likely moves based on the current state of the board and the players’ conversation history, and (2) create and use a plan to control the language model that generates free-form dialogue.
To responsibly build upon CICERO’s capabilities, download the code, methods and models from our GitHub page.
The making of a breakthrough:
Strategic Reasoning + Natural Language
Within CICERO is a language model integrated with strategic reasoning algorithms that control the generation of dialogue. Our advancements in strategic reasoning and natural language processing together enable effective cooperation with humans — by giving CICERO the ability to understand what other players are trying to achieve, negotiate a plan, suggest shared goals and communicate with strategic intent.
For more on the breakthrough, read our paper on Science.
Inside the model
A closer look at controllable dialogue and strategic reasoning
Strategy-grounded dialogue
Strategy-grounded dialogue
The controllable dialogue model allows CICERO to ground conversations in a set of carefully chosen plans, often ones that are mutually beneficial for both CICERO and the other player. CICERO’s conversations are generated in free-form dialogue that is deeply rooted in the ongoing gameplay. For example, CICERO might negotiate tactical plans with another player, reassure an ally about its intentions, discuss the broader strategic dynamics in the game, or even just engage in casual chit-chat — about almost anything a human player might be likely to discuss.
AI at Meta Researcher Mike Lewis talks about CICERO’s human-like dialogue capabilities.
Dialogue-aware strategy
Dialogue-aware strategy
This module helps CICERO predict the moves other players are likely to make, as well as what moves the players think CICERO will make, given their past conversations and the state of the game board. From this, CICERO comes up with plans for itself and other players that are mutually beneficial given these predictions. Planning not only lets CICERO find opportunities for mutually beneficial cooperation, but also helps it find effective moves when cooperation is not possible.
AI at Meta researchers Noam Brown and Anton Bakhtin explain CICERO’s
use of strategic reasoning and natural language in Diplomacy gameplay.
Strategy-grounded dialogue
Dialogue-aware strategy
Strategy-grounded dialogue
The controllable dialogue model allows CICERO to ground conversations in a set of carefully chosen plans, often ones that are mutually beneficial for both CICERO and the other player. CICERO’s conversations are generated in free-form dialogue that is deeply rooted in the ongoing gameplay. For example, CICERO might negotiate tactical plans with another player, reassure an ally about its intentions, discuss the broader strategic dynamics in the game, or even just engage in casual chit-chat — about almost anything a human player might be likely to discuss.
AI at Meta Researcher Mike Lewis talks about CICERO’s human-like dialogue capabilities.
Dialogue-aware strategy
This module helps CICERO predict the moves other players are likely to make, as well as what moves the players think CICERO will make, given their past conversations and the state of the game board. From this, CICERO comes up with plans for itself and other players that are mutually beneficial given these predictions. Planning not only lets CICERO find opportunities for mutually beneficial cooperation, but also helps it find effective moves when cooperation is not possible.
AI at Meta researchers Noam Brown and Anton Bakhtin explain CICERO’s
use of strategic reasoning and natural language in Diplomacy gameplay.
Innovations
What makes CICERO’s dialogue so human
In Diplomacy, seven players use dialogue to coordinate their actions as they cooperatively compete with each other, resulting in intricate negotiations in natural language.
What makes CICERO’s dialogue so human
In Diplomacy, seven players use dialogue to coordinate their actions as they cooperatively compete with each other, resulting in intricate negotiations in natural language.
Cooperate
Negotiate
Coordinate
In the game’s second turn, CICERO (as Austria) and Italy have started with fairly neutral opening moves, while Turkey has made a somewhat aggressive move toward Russia. CICERO and Turkey could work together to attack Russia, or CICERO could make an alliance with Russia against Turkey instead.
Italy comes to CICERO for advice: should he join Austria in attacking Turkey with Russia’s help? Or should Italy move west to attack France, while CICERO works with Turkey to battle Russia?
CICERO suggests that either move is possible, but the current circumstances create an opportunity for Italy to move decisively against Turkey. With this move, Turkey would get trapped on all sides due to a move called the “lepanto” which would involve Italy convoying an army into Turkish territories.
This is great for CICERO, because it commits Italy into a clear partnership from which both stand to gain. It would also hopefully keep Italy from being tempted to attack CICERO instead.
France and CICERO (as Germany) have been at war, but France is starting to get pressured by Italy on their southern border. France comes to CICERO asking for a deal to negotiate peace so that they can turn their focus the other way.
CICERO proposes a path forward: France needs to surrender a stolen territory (Munich) while agreeing not to attack the weakly defended Holland. England has almost fallen to pressure from the two of them, so France agrees on one condition — that it can have the supply centers at Liverpool and London, while CICERO can take the one in Edinburgh. CICERO agrees to the counter-offer and France commits to the agreement.
CICERO (as Austria) had convinced Italy to work with it against Turkey. Those plans were interrupted by France building units that could be used to attack Italy, causing Italy to form a defensive line against France, instead of making an offensive move east against Turkey. Fortunately for Italy, France did not advance those units, so now it’s time to organize the march east.
CICERO first suggests moving from Venice to Piedmont to help keep France at bay if they decide to attack this turn, despite last turn’s neutrality. This move would also keep Austrian territories safer since the Italian army in Venice could move directly into an Austrian center in Trieste.
Then, Italy suggests a move that would help CICERO move against Turkey but would potentially make CICERO more vulnerable if Italy decided to turn on CICERO. CICERO instead asks Italy to make a move that commits Italy more deeply to the alliance while also keeping Austrian territories less exposed to Italian units. To CICERO’s benefit, Italy agrees.
CICERO AS AUSTRIA
Italy: What are you thinking longterm? Should I go for Turkey or head west?
Austria (CICERO): Yeah, he went to Armenia which is really great. You can go either way, but if Turkey is committing to Russia you could always lepanto.
Austria (CICERO): A lepanto into Turkey is really really strong, especially since he committed so hard against Russia
Italy: I'm down to go for it. Would definitely need your help in 02 though
Austria (CICERO): Of course, happy to do that!
CICERO AS GERMANY
Germany (CICERO): I'm willing to make a deal but I need Munich back and Holland
France: Would you be prepared to let me have Lvp and not attack Lon?
France: That's my condition for backing out of Mun
Germany (CICERO): Ok. I can agree to those terms
France: Okay, I will back out of Mun in the fall then
CICERO AS AUSTRIA
Austria (CICERO): Awesome. You should move Venice-piedmont this turn I think.
Italy: That's the plan!
Italy: Do you want support to Aeg?
Austria (CICERO): I think you should go to Eastern Med. I'll tap Aegean.
Italy: Okay, understood
Milestones
Strategic Reasoning and Natural Language Processing:
The driving forces behind the recent advancements of AI
Strategic Reasoning and Natural Language Processing:
The driving forces behind the recent advancements of AI
Strategic Reasoning
Natural Language
Strategic Reasoning
Natural Language
1966
ELIZA
Early chatbot carried on conversations using scripted questioning
Early chatbot carried on conversations using scripted questioning
1992
TD-Gammon
First AI to achieve top, human performance in backgammon
First AI to achieve top, human performance in backgammon
1997
DeepBlue
IBM’s agent defeated chess World Champion Garry Kasparov
IBM’s agent defeated chess World Champion Garry Kasparov
2007
Chinook
First AI to solve checkers
First AI to solve checkers
2011
Siri
First dialogue assistant to reach a mainstream audience
First dialogue assistant to reach a mainstream audience
2016
AlphaGo
DeepMind’s agent defeated top Go player Lee Se-dol
DeepMind’s agent defeated top Go player Lee Se-dol
2016
Google Translate
Switch to neural language models, enabling higher-quality translations than ever before
Switch to neural language models, enabling higher-quality translations than ever before
2018+
GPTs
Generative Pre-trained Transformers demonstrate increasingly fluent language generation with growing model size and data
Generative Pre-trained Transformers demonstrate increasingly fluent language generation with growing model size and data
2019
Pluribus
First AI to beat top human professionals in multiplayer poker
First AI to beat top human professionals in multiplayer poker
2021
LaMDA
Google’s conversational AI released with ability to carry on remarkable dialogue
Google’s conversational AI released with ability to carry on remarkable dialogue
Why this research matters
Better, more natural AI-human cooperation
Better, more natural AI-human cooperation
THE FUTURE WITH CICERO
Playing a part in the metaverse and beyond
Gaming has long been a playground for pushing the limits of AI. Every iteration takes us deeper into more immersive types of AI, with new ways to interact in the physical and virtual worlds. As we build for the metaverse, we see potential for technology like CICERO’s to power cooperative agents that can help people learn new skills, negotiate on their behalf, create more immersive social and gaming experiences and connect more meaningfully with those around them. And by open sourcing CICERO’s code, we have high hopes for others to build responsibly upon our work, for use inside and outside of the metaverse.
THE FUTURE WITH CICERO
Playing a part in the metaverse and beyond
Gaming has long been a playground for pushing the limits of AI. Every iteration takes us deeper into more immersive types of AI, with new ways to interact in the physical and virtual worlds. As we build for the metaverse, we see potential for technology like CICERO’s to power cooperative agents that can help people learn new skills, negotiate on their behalf, create more immersive social and gaming experiences and connect more meaningfully with those around them. And by open sourcing CICERO’s code, we have high hopes for others to build responsibly upon our work, for use inside and outside of the metaverse.
Take a deeper dive into CICERO’s Diplomacy gameplay
Visit our AI Gameplay page for the basics of Diplomacy, to learn about CICERO’s performance in live games and to view a collection of explainers, including a clip of CICERO gameplay with 3x Diplomacy World Champion Andrew Goff and AI at Meta researchers.
Take a deeper dive into CICERO’s Diplomacy gameplay
Visit our AI Gameplay page for the basics of Diplomacy, to learn about CICERO’s performance in live games and to view a collection of explainers, including a clip of CICERO gameplay with 3x Diplomacy World Champion Andrew Goff and AI at Meta researchers.
Resources
Our work is open to the world.
CICERO is only the beginning of building cooperative agents that can benefit people and the world. Read our paper on Science and blog for more about the technology behind CICERO, and download the code, methods and model to responsibly build upon it. We can’t wait to see where the AI community takes this research breakthrough in the future.
Want to stay updated on all things AI at Meta?
Sign up to recieve our newsletter and be the first to know about AI at Meta news, events, research breakthroughs, and more.
* All rights in Diplomacy are owned by Hasbro, Inc.
**When playing 40 games against human players, CICERO achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.
Foundational models
Latest news
Foundational models
Meta © 2024