Only days after AI demonstrated its seeming superiority over humans by thoroughly smashing a human champion at a board game, AI has laid a big proverbial egg, in form of an AI driven chatbot that seemingly went completely bonkers.
There is a a bit more than meets the eye, of course. For all its computational complexity, board games of complete information are mathematically simple. Following the rules inevitably leads to a simple set of outcomes: either there is a perfect strategy or there isn’t given the rules of the game, and if there is a perfect winning strategy, it can be mathematically calculated. Thus, chess, go, and tic-tac-toe are doomed to end in a permanent draw (this has been proven for Tic-Tac-Toe) or are pointless because one player will always win through that perfect strategy (this has been proven for checkers–the most complex game that has been computationally “solved” so far.) All these have been known for more than 100 years.
The distinguishing characteristic of the games of perfect information is that, given the choice by the other player, the best response is well-defined. What choice to make, then, is fairly simple and straightforward. Of course, chess programs supposedly often crashed in the past when multiple choices that are equally good were available, but a randomized tie-breaking mechanism is a fairly simple addendum.
Human interactions, riddled with nuances, conventions, and other subtle subtexts, are not so straightforward. If you will, the variance of the best response distribution conditional on a stream of events that has so far taken place is enormous. Humans, other than those who are autistic, are much better at learning how to navigate these nuances and make appropriate hedges that account for the risks involved. The Wired article that I linked above, on the other hand, inadvertently points out why the Microsoft AI screwed up so badly : “Mortensen says this is just how this kind of AI works. The system evaluates the weighted relationships of two sets of text—questions and answers, in a lot of these cases—and resolves what to say by picking the strongest relationship.” This is NOT how humans–even teenagers–behave, except, possibly, the speculators on Wall Street, and even in that case, they actually have monetary incentives, backed by potential for gov’t bailout in case of failure, to do so. Most humans are sufficiently risk-averse that they shy away from saying the “wrong” things that will create poor impressions of themselves.
There are a few occasions when humans do say wrong things: when the cultural conventions are unknown, such as in the instances of first contact between different societies that rarely interact with each other (e.g. US military with Iraqi civilians after the invasion of Iraq), the children who are just learning the conventions (“kids say the darndest things!”), and the autistic folk whose brains are wired differently enough that they are slow picking up the social conventions. In a sense, the behavior of the Microsoft AI is reflective of all these instances: it simply has not learned the human social conventions enough to “behave.” If given enough time, it may well have picked up enough to avoid its worst foibles. But, at the same time, it appears that the learning algorithm that it was programmed with simply did not incorporate enough instructions on how to deal with the uncertainties inherent in the interactions, essentially “encouraging” it to adopt the most extreme responses that will elicit potentially very negative reaction.
Now, why should an AI care about the negativity of reactions? In order to get the AI to evaluate such things, we are now talking about the human “emotions”: shame, vanity, pride, honor, empathy, and such–the Biblical stuff, the fruit from the Tree of Knowledge. Indeed, one might almost say that, prior to the Expulsion, Adam and Eve were practically AI, while, by eating the forbidden fruit, they suddenly acquired the “human” characteristics. Perhaps one might write a Sci-Fi novel about this…or maybe someone is. 😉