CICERO: The first AI to play Diplomacy at a human level | AI at Meta

Meta presents CICERO — the first AI to achieve human-level performance in Diplomacy, a strategy game which requires building trust, negotiating and cooperating with multiple players. New techniques in strategic reasoning and natural language processing give CICERO the ability to understand players’ motivations and perspectives, make complex plans, reach agreements, form alliances and even win! When playing in live games, CICERO achieved more than 2X the average score of its human opponents.

#ArtificialIntelligence

--

Working on AI at Meta, we're bringing the world together by advancing AI, powering meaningful and safe experiences, and conducting open research.

[severinusdemonzambano]

While CICERO's achievements are laudable, the claims made in this video stretch the truth of what CICERO has achieved. They need to be addressed and contextualized, because Diplomacy is not a solvable game.

First, CICERO participated in "Blitz" Diplomacy, a variant in which players have (typically) 5 minutes to deliberate with one another before entering orders. It is much easier for a generative neural network to interface with humans for five minutes than the standard fifteen minutes in face-to-face players.

Secondly, conversations in "Blitz" matches are generally cabined to immediate tactical considerations. Strategic conversations, discussion and familiarity with variants, and rapport building are backgrounded in "Blitz" matches. Thus, CICERO's tensors likely contain insufficient quantities of data or tensor categories to capture the actual experience of Diplomacy.

Thirdly, in the paper published, the examples of CICERO's successful negotiation does not indicate the linguistic mastery that Meta believes it has achieved. Austria convincing Italy to deploy a Lepanto opening is not a hard ask to make - indeed, it is the most common opening by Italy. The negotiation done by CICERO to convince the player playing as Turkey does not reveal CICERO's mastery, but the Turkish player's strategic shortcomings.

Lastly, playing 40 "Blitz" matches against 82 players is not a fairly reliable sample size. How many of those players went AFK? Furthermore, the number of possible opponents (240, if CICERO participates in every game) is almost three times larger than the sample CICERO played against. CICERO's ability to gather information on 82 persons is much easier, many of whom are repeat players, is much easier than 240 different opponents. While CICERO's opponents were anonymous, I can tell with little difficulty who certain players are on Backstabbr, even if the game is anonymous. CICERO should be able to identify opponents based upon lexical quirks and tells. While it was able to succeed in "Blitz" Diplomacy, CICERO's success draws upon a limited pool of repeat opponents that are playing a variant that does not actually reflect how the game is played.

Meta's AI research, and their willingness to share algorithms publicly, is laudable. But what is lamentable, and frankly dishonest, is stating that CICERO is playing Diplomacy at the same level as a human player. Meta should make more modest claims that reflect what CICERO was actually able to achieve in a variant, which are impressive achievements. As a company who is still reeling from Francis Haugen's disclosures, it is unwise and puzzling that Meta would misrepresent CICERO's achievements in Diplomacy. There's great irony in the fact that in misrepresenting CICERO's achievements in Diplomacy, Meta has themselves committed an unforced strategic error - something that CICERO would never do.

[carlosquinones]

I would imagine the planning engine would plan its moves, then send keywords associated with each decision to the dialogue engine, which would use some sort of generator to expand on those keywords and send them to the player and blend them with the response from the player, then send that response back to the planning agent, which would decode the keywords from sentence to keywords in order to simplify its decision-making process, calculate an update and send that back to the dialogue engine, is what I'm thinking.

[AIMatej]

Very cool! I am curious to dive into the details in the science paper to see how you implemented the strategy component. It is this first step towards common sense for large language models?

[jackbauer]

This is what META should instead invest full On over the CRAPVERSE !

[eSportDjango]

LMAO can't wait for Cicero to get into a skunk works server and launch an ICBM

[ebujvnr]

I applied for llama model's access 7 days ago but still unable to get access. Anyone can tell me the reason?

[Adhil_parammel]

Please connect alpha zero to language, to teach chess/go

[StasKelvich]

Why all verticals (table, bottle, curtains) are not vertical on this video while horizontals are horizontal?

[ebercondrell]

The comments on this video are fantastic. I might sue for damages because of my whiplash.

[derek_davidson]

Finally! I can continue playing board games by myself

[bts__armyyyyyyy]

The new version of WhatsApp app meta-AI its not launched they released testing version but if ask information it shows correct but if we ask again it will display different information on same topic and also not accurate information I'm student we belive that ai information is accurate give an accurate information input
"

[andrewvirtue]

I can not find this game "Diplomacy" on steam or google play. Irritating.

[JohnDoe-otbm]

This video and audio is 100% artificial (CGI)

[TheNoblot]

comme le temps aspect la sagesse au profite de petit pour le bien d’autrui 🤔🪐
Bertrand Russell on Israel: 🤔🤑🤐🤔🌎🛡⚔🤔⚖🥇

[globus]

Yeah, now we're officially fucked :) Just yesterday was watching 3rd season of Westworld - .... and now this:)