Expert Diplomacy Player vs CICERO AI

CaptainMeme takes on Meta's new Diplomacy AI, CICERO! This video shows the entire game in real time, including all Press - yes, this AI negotiates!

There's a commentary track with me giving my thoughts on the game, but feel free to mute if you just want to watch the messaging.

Diplomacy is a negotiation-based board game set during World War One. 7 players each control one great power (England, France, Germany, Austria, Italy, Russia, and Turkey) and fight for control of Europe, using their diplomatic skill to make covert deals with one another and to backstab their friends.

[DiploStrats]

Feel free to ask any questions you want to, I'll answer those I can. Although bear in mind, I'm a Diplomacy player, not an AI researcher - my understanding of the bot itself is pretty limited.

[cordobes]

Nice, an AI that is really good at being deceptive to unwitting humans. What could go wrong?

[repmel]

Deep Blue beat Kasparov and Watson beat Ken Jennings, now Facebook knocks on Machiavelli's door.

[mighatesmakingusers]

"Germany stabbed me! Therefore I <checks map> invade England!"

I think this is a good example of the language model painting over discontinuities in the intents system. It knew it got stabbed. It knew that its best moves were in England. Those were separate intents and policies, but the language model added some bonus causality when told to generate all of the data at once.

[karthikkonath]

Love this video! It was awesome seeing CICERO evolve all the way from where it began to where it is now and to see you cover this in another amazing DiploStrats video!

[PikaPilot]

The AI is playing and communicating excellently. This is amazing to see!

[SealedSun]

That was super interesting. And I have never played a game of diplomacy in my life. Wish you had spent a moment commenting on the moves once they get revealed (what they signify/what they tell us etc.)

[Sploack]

This video could have 2 different titles depending on how it's viewed.
Just listening to the commentary: Expert Diplomacy Player plays a very high level game against incredibly strong Diplomacy AIs
Reading the press: Man argues with Alexa for an hour and a half trying to make her understand him while Alexa tries to convince him she's a human being. They both fail.

[DS-uozy]

That's so crazy how quickly AI evolves, thank you so much for sharing this :)

[ejayAD]

Breathtaking to watch and listen, thank you! :)
As well as being a Diplomacy player I'm very concerned about AI and AI alignment, so this is startling.
Few questions -
To be clear: the 6 other CICEROs are also interacting with each other via text to plan and negotiate? Would be interesting to see their chats, if available.

I'm really interested when the AI says things like "Didn't see it in time' - can't find the timestamp. Can we clarify if there is some kind of 'parsing time' that means that is legit, or in reality does it parse if it instantaneously and it perhaps using that as an excuse, having learned that human players do that from the dataset?

[sinfinite]

Great video, I’m interested in seeing more of CICERO from the perspective of an experienced player playing them!

[bradleygrace]

This is absolutely incredible! Given these are blitz games I wonder how you think the AI would do in a full length extended deadline game?

I personally believe given the tactical superiority they have, this format is actually well suited towards them and that they would struggle when humans have more time to think things through and build relationships but I'd love to be proven wrong and report to our robot overlords.

[gronkymug]

Thanks for this video. It is awesome! I really do hope the next gaming AI will paly Civ6/7 and that diplomacy will be like in Diplomacy game instead of being restricted to a few predefined dialog options.

[bradgasdia]

Fantastic video! I thought we were a few years away from press AI’s, but we’ll here we are. Most excited I’ve been about research in a long time.

Would be really cool to see a game commentary including observations on press from the bot, which I believe meta has made a few available.

[leovershel]

Absolutely insane game, amazing to see what modern AI is capable of. Great video!

[nikebless]

Do you know if the bot ever uses deceptive tactics in communication, e.g. where it would message Italy, "let's gang up on Germany, " and message Germany "let's gang up on Italy"?

[ravibetzig]

This was fantastic. As many others mentioned I am amazed at the quality of the press that the bot sends. I can see why it breaks down in a non-blitz format. I like how England goes along with the suggestion for you to support him into Den when he's down to his last dot as he doesn't hold the past fights you two have had over Scandinavia against you.

This game really turned for you when you guessed correctly about supporting Sev as opposed to War in that one phase... I think it was F05. That was massive and I think we are looking at an Italy 11 center board top or so otherwise.

The bots would run circles around me tactically for sure. I've been playing for some years but have never played much gunboat. Moves like Boh-Tyr in S08 that both G and T suggest to you are quite good and not something I would've seen.

[obnoxas]

Imagine if the 5 AI players all immediately teamed up and obliterated our human champion then agreed to a 5 way draw

Edit: 6 AI players. 6 way draw. I can't count.

[sinfinite]

Question about CICERO, in a 1 bot 6 human game, would you be able to figure out who CICERO is not by game play, but the way they spoke? Like if you started to talk about non Diplomacy things or speak in short hand would you be able to figure out who the bot is? Because from your Gunboat Diplo format, one of the criteria for the bot that you made was that it was indistinguishable from a human I was wondering if CICERO fulfilled these requirements in a Press Diplo game.

[AnglosArentHuman]

I'm legally obligated to ask before even watching, how does the AI respond if you just send absolute nonesense diplo?