THIS IS BOTH IMPRESSIVE AND DISTURBING: Meta researchers create AI that masters Diplomacy, tricking human players.
I’ve been playing Diplomacy since middle school, and there are no random elements like dice or cards — it’s all human interaction.
Even before Deep Blue beat Garry Kasparov at chess in 1997, board games were a useful measure of AI achievement. In 2015, another barrier fell when AlphaGo defeated Go master Lee Sedol. Both of those games follow a relatively clear set of analytical rules (although Go’s rules are typically simplified for computer AI).
But with Diplomacy, a large portion of the gameplay involves social skills. Players must show empathy, use natural language, and build relationships to win—a difficult task for a computer player. With this in mind, Meta asked, “Can we build more effective and flexible agents that can use language to negotiate, persuade, and work with people to achieve strategic goals similar to the way humans do?”
…
The resulting model mastered the intricacies of a complex game. “Cicero can deduce, for example, that later in the game it will need the support of one particular player,” says Meta, “and then craft a strategy to win that person’s favor—and even recognize the risks and opportunities that that player sees from their particular point of view.”
…
At the same time, this technology could be used to manipulate humans by impersonating people and tricking them in potentially dangerous ways, depending on the context. Along those lines, Meta hopes other researchers can build on its code “in a responsible manner.”
Meta claims to have taken steps to prevent Cicero from being abused (or from becoming abusive, I suppose), but the source code is also open-sourced on GitHub.
From the comments at Ars: “Unbelievable. Training a non-conscious AI to ruthlessly deceive, manipulate and enlist humans towards the achievement of an arbitrary value function no matter the cost, as one of the first AI functions to develop. It’s like these guys want a paperclip-optimising singularity apocalypse.”
And yet: META WORKS ON AN AI, INSTEAD PRODUCES a “random bullshit generator.”
Similar programs, totally different results.