by Conroy
 |
| Watson "between" Rutter and Jennings |
Last night I watched glumly as
Watson,
IBM's custom-built supercomputer,
completed a comprehensive three-day (and two game) victory over top human champions on the game show "
Jeopardy!". Watson proved to be quicker than
Ken Jennings and
Brad Rutter, perhaps Jeopardy!'s two greatest champions. Jennings rose to national prominence several years ago when he went on a 74-game winning streak and amassed well over $2 million in prize money. Rutter, first competed on the show when contestants were only allowed to win five games consecutively. Since then, however, he has won three of Jeopardy!'s top tournaments (including defeating Ken Jennings) and amassed $3.5 million in prize money, the most of any competitor in the show's history. Except for a few brief flurries, these accomplished players were no match for Watson.
 |
| The real Watson |
For anyone watching the show, we saw Watson as an upended flat screen monitor with a modified and constantly changing IBM logo display, "standing" behind the lectern between Rutter and Jennings (see picture above). His name was neatly printed (or typed as we were to interpret) on the name panel in front of the lectern. When Watson answered or chose a question we heard a soft, calm human voice. The impression was of a competitor on stage, like his two human opponents. This was of course not the real situation. As
Alex Trebek demonstrated during the first show, Watson is a massive series of inter-connected servers as powerful as thousands of individual computers. Watson is as large as a small apartment and is stored in a specially cooled room. The show was taped at IBM's lab in Yorktown Heights, New York. The Watson on stage was an avatar, deaf and blind. When the questions were read on-stage the real Watson was simultaneously fed a text file of the question (he didn't hear Alex talk or read the question screen like you or I do at home). Watson took the text information and ran numerous concurrent algorithms through his vast store of knowledge to develop potential answers, assigning a confidence to each. When the confidence value was high enough, the computer would attempt to answer. Watson buzzed in using a joystick buzzer like the human contestants.
As you probably gathered, I was rooting for the humans. I grew up competing on Academic Teams and participating in various tournaments, so I can see myself in the men on stage. I also would like to think that humans can do things that machines can't. The reality is that this competition, while fun to watch and a great advertisement/demonstration for IBM doesn't prove all that much about quiz shows. In general, the questions were not too difficult and I have a suspicion that all three competitors were buzzing at about the same time, Watson's timing was just better. This isn't surprising, the programming probably allowed the machine to buzz at the exact earliest moment allowed (buzz too early and you're locked out from answering) - far more precisely than a human can react. I have sense, a very strong sense, that if you asked these questions of the three competitors separately you would see a similar number of correct answers and a similar cumulative time to answer (in fact the humans may have gotten a few more answers right). I like that Watson got several questions wrong, don't we all, and his eccentric wagering for "Daily Doubles" and "Final Jeopardy!" had to be a quirk (intended or not) of his programming. There was a funny moment on the first day when Watson repeated an wrong answer provided by Ken Jennings; Alex gently chided the machine.
Watching over the three days I felt similar to how I did when
Deep Blue, another IBM invention, defeated chess grandmaster
Gary Kasparov in 1997. Through the cost of millions of dollars and years of effort IBM programmers had beaten the best that humanity could offer in a specific intellectual activity. (Note that of the earlier project, Kasparov
remains strongly skeptical of Deep Blue's performance.) From then on humans would continue to lose in ever more lopsided contests (though amazingly, top human chess players can still beat machines in individual matches). I have a feeling that a future Watson will be able to answer almost any trivia question and do so almost immediately. I would be really impressed if that computer could listen to the question and read the files using some form of electronic ear and eye, that would make it a lot more "human-like". I still see that as a ways away.
Despite my lament for yet another human activity now bettered by computer (kind-of), this entire enterprise is a testament to the power of computing and our advancing programming abilities. I laud IBM for spending time and money on these types of endeavors. It's critical that we as humans push the bounds of our technology and do so in a way that can excite our imagination. I have no doubt that IBM's programmers were deeply motivated to "win" on Jeopardy!, in a way that they may not have been on a more mundane and less public project. I hope we can take the abilities of Watson and apply them to everyday uses. The capability of understanding "natural language" is a great step in computing. Applications could range from better information sorting, telephone voice response programs that can actually provide real help, and even more accurate and precise responses to specific internet search engine queries. Here are
some thoughts from IBM on the subject.
The most impressive aspect of Watson was not the results of it's performance, but it's complete independence from human support during the show (albeit in a highly controlled environment). Something Deep Blue's creators could not boast. Hopefully that means practical applications for this technology are in our near future.
 |
| Andy's anger erupts against Watson |
By the way, Watson has seeped into the national culture, check out
this funny bit from Monday's Conan show. The writers got out natural human angst about "smart" computers right on the money.
8DJ3PPDKM7ZT
Take heart, Watson is no threat to humans, it can't hear, see, feel, and in most ways even think. Instead, we should view the work by IBM as terrific progress. I remember the first "super-computers" more than thirty years ago...now my cell phone has more computational power. It's amazing how far we've come.
ReplyDeleteI agree with Conroy, I wanted the humans to beat the machine. Still, this show was a lot more interesting than a regular Jeopardy episode.
ReplyDeleteAccording to a recent New York Times Article, a "physician's assistant service that will allow doctors to query a cybernetic assistant" using voice recognition may be available in the next couple of years.
ReplyDeleteThey should name it Wilson, rather than Watson. The problem is, how is Dr. House going to get under its skin when it doesn't have any?!
The Man, my understanding of the show "House" (as you know I don't watch it) is that the title character can get under anyone or anything's skin, trust me that cybernetic assistant doesn't stand a chance!
ReplyDeleteAll joking aside, I do hope that practical, and highly useful, applications of this technology will soon be available. How much more efficient and productive could we be if we can use computers to quickly and accurately sort through data, records, etc. and provide responses to highly specific queries?
Here is Kasparov himself commenting on the possibility of using Watson in the medical context:
ReplyDelete"My concern about its utility, and I read they would like it to answer medical questions, is that Watson's performance reminded me of chess computers. They play fantastically well in maybe 90% of positions, but there is a selection of positions they do not understand at all. Worse, by definition they do not understand what they do not understand and so cannot avoid them. A strong human Jeopardy! player, or a human doctor, may get the answer wrong, but he is unlikely to make a huge blunder or category error—at least not without being aware of his own doubts. We are also good at judging our own level of certainty. A computer can simulate this by an artificial confidence measurement, but I would not like to be the patient who discovers the medical equivalent of answering 'Toronto' in the 'US Cities' category, as Watson did." (Full text here.)
The Man,
ReplyDeleteWell it's good to know that Kasparov has similar thoughts about Watson (he even used some of the same terms as me). I like his emphasis on the method the computer uses to arrive at its answer and how completely different it is from the human method.
Also, as he points out, the Deep Blue technology yielded no other significant uses. It didn't even modify chess theory. Hopefully Watson's technology will turn out to be more than a passing curiosity.