Did the Watson Jeopardy experience have an "Open the pod bay doors, Hal" feel to it? It creeped me out a little.
Right, the circular Watson avatar looks a *little* too much like HAL's red light, right? I think it's reading Brad's and my lips.
When anyone mentions 2001 to them (or Terminator, or Matrix, or Tron, or...) IBM prefers to bring up the helpful question-answering computer on Star Trek. C'mon IBM! You just invented SkyNet! Own it!
What is your strategy for beating a computer with access to the world wide web? Buzz early and often? Or wait for it to make the wrong "educated guess"?
Actually, Watson isn't live on the Web. But it might as well be...15 trillion bytes of data from books and Internet resources--roughly the size of the printed material in the Library of Congress--have been pumped into its database.
That said, yes, given Watson's many advantages, some aggressive play by the human contestants is going to be called for.
Hi Ken- Seemed to me, for many of the questions, that the computer was just better at buzzing in. Does Watson have an unfair advantage for timing the buzz-in? Thanks.
As Jeopardy devotees know, if you're trying to win on the show, the buzzer is all. On any given night, nearly all the contestants know nearly all the answers, so it's just a matter of who masters buzzer rhythm the best.
Watson does have a big advantage in this regard, since it can knock out a microsecond-precise buzz every single time with little or no variation. Human reflexes can't compete with computer circuits in this regard. But I wouldn't call this unfair...precise timing just happens to be one thing computers are better at than we humans. It's not like I think Watson should try buzzing in more erratically just to give homo sapiens a chance.
Because you champion humans obviously do well on high value answers, wouldn't your best strategy have been to select them?
That actually is my strategy. Watson sometimes takes some time to get acclimated to a category, so starting at the bottom gives me a chance to rack up some money before it gets confident. In theory!
Hi Ken, thanks for taking questions today! Last night, when you answered a question incorrectly, Watson buzzed in and repeated your wrong answer. Do you know if this obvious defect in Watson's programming was fixed for later rounds? It seemed like a really glaring oversight.
Not an oversight: a known weakness. Watson is both blind and deaf to most of the events of the game (except the text of the clues and the correct answers once they're revealed, which it receives electronically) so it has no way of knowing what the last wrong answer was. It's a rare hiccup, so I'm sure IBM isn't pumped that it happened in prime time last night!
Do you think some questions or categories have been tailored with key words so Watson can more easily score?
To keep the playing field level, Jeopardy used a random selection of game boards that had been pre-written for regular all-human games. The only concessions to Watson: since it's blind and deaf, all Audio and Video Daily Doubles are out, as are categories that only make sense if Alex verbally explains them.
Industrial Strength Magnets.
They give Brad and me bottled water during every commercial break. I think you can see where this is going if the machine starts to build a big lead.
So, Ken, do you have a backup plan that just might involve slyly tripping on Watson's cord?
Alternately, you can just tell it a paradox like "This sentence is a lie!" and Watson will start to fume and freak out in confusion. This strategy brought to you by Captain Kirk, ca. 1967.
I was suprised that you only got through the first round. I found too much of the first night devoted to IBM, like a long commercial. What is the format of the next two nights?
The contest is two games stretched out over three nights: another half game tomorrow, with the final game Wednesday. In addition to giving IBM a nice infomercial spot (if you buy a Watson 1.0 license for home use, you also get a Watson mouse pad and some steak knives!) the documentary spots help explain how the contest came together, why this truly represents an AI breakthrough, and so forth.
It reminds me of the "human interest" stuff that always clogs up Olympic footage. Revealed tonight: Watson hopes to win these games for its adorable little sister, who has leukemia!
It was interesting to me that the "Which decade" category seemed especially hard for Watson. Why was that?
I think it took it a little while to figure out that the answers would all be decades. This seems incredible to a human playing along at home, but basic contextual issues like this are incredibly hard for machine intelligence to master. The cool thing about Watson is it learns from those mistakes. By the end of the category, it had learned that the answers were going to be decades, and adjusted accordingly.
It is clearly the modern descendent of "Joshua," the learning computer in the 1980s movie WarGames. "How about a nice game of Jeopardy, Ken?" "No, let's play Global Thermonuclear War."
My understanding is that the shows are taped and you obviously know the outcome. You made the recent comment on MSNBC that in order to win you had to play and bet recklessly. Do humans have an advantage in terms of betting (game theory) Your comment seems to be a tell that a human won, any guidance appreciated.
I would say that Watson has the wagering edge--like you say, it's all game theory and math, and even a cheap PC is pretty good at doing math at high speeds. That said, a human player might be more willing to take risks that Watson is too smart to try. In the practice games I saw, betting big on Daily Doubles and Final Jeopardy seemed like the only way to cancel out Watson's big buzzer advantage.
Good morning Mr. Jennings, thank you very much for doing this chat. You are my favorite Jeopardy Champion (with the possible exception of Babu Srinivasan, who has one of the coolest names ever). My question is this: with all the Jeopardy appearances you've had, how did this one compare as far as pressure? I can imagine that such a high-profile game could potentially be nerve-wracking. Or do you instead approach every game with the same mindset, regardless of opponent?
Babu! I'm sure he appreciates the shoutout.
I tried to think of the computer the same as any other opponent, but in practice that turned out to be pretty hard, given the creepy insectoid clicking of its mechanical thumb buzzing relentlessly just to my left. I did feel like the underdog this time, which isn't a position I'm used to being in on Jeopardy.
If you are the winner, would you be willing to sit with the Watson designers to improve the machine even further? If so, what would you suggest?
The Watson team told me two things after the match: that the idea for Watson was born after watching my 2004 streak on Jeopardy, and that they watched LOTS of tape of me while honing its skills. "There's a lot of you in Watson," one guy said. So I already feel like the Dr. Frankenstein here. If it goes amuck and kills humanity and stuff so sorry lolz my bad!
Will a single winner be named on Wednesday after two games are played? How will the best player be determined?
Two game total-point final, just like most Jeopardy tournaments. Plus a pro wrestling-style "cage match" with folding chairs afterwards.
Ken, you're hilarious. Not only are you the top Jeopardy champion, you've also got the most personality. Any chance you'll write any more books? I read your first in practically a day, I loved it. Your daily trivia book also has an honored place in my bathroom. Next to the litter box.
I'd like to thank my mom for taking part in the chat! Love to you and Dad!
My next book, Maphead, comes out in September plug plug plug. It's about geography geeks of all kinds: antique map collectors, geocachers, pint-sized National Geographic Bee prodigies, etc.
I want to know how Watson selected the Daily Double early in the round. It was an exception to him going top-down in the categories!
Daily Doubles aren't distributed randomly...basically, it had thousands of old Jeopardy games in its head and knew where to look. And got very lucky.
Most players go top-down, but some hunt for Daily Doubles like Watson. Brad Rutter calls this an arms race: if one player does it, you have to join in to keep up.
I played in the sparring matches last year at IBM Research. Watching last night shows me they have really improved Watson's response time and there are far less 'weird' responses. Does his quirky play bother you?
His--er, its--quirky play is half the fun. I watch video of some of the sparring matches and loved the little goofs. It got Jamie Foxx and Ludwig van Beethoven confused in one game. Common mistake!
It just shows how hard it is to do what Watson is doing: answering natural-language questions. Obviously our brains have all kinds of built-in "double-checks" that we do to make sure the information we're retrieving is correct. Watson, despite the four years and tens of millions dollars spent, is still much less sophisticated in that regard than our little lump of neurons.
Before and After is one of my favorite Jeopardy! categories. How do you think Watson might do in categories involving humor, puns, or other types of wordplay that require more thought (or at least, a different kind of thinking) than simple word association?
Watson is at a disadvantage when it comes to humor, wordplay, irony, and so forth. But then again, so are many humans that I meet, particularly the ones working rental car counters.
Much IBM effort went into teaching Watson how to play punny categories like Before and After, Crossword Clues, and so forth...and it's amazingly good. In a sparring match, I have seen it sweep (go 5-for-5 in) a Before and After category.
Now do you know how people felt when they were competing against you?
This is very perceptive. Knowing lots of answers but being a millisecond slow on the buzzer is indeed very frustrating. To the 149 Ken Jennings losers back in 2004: if you are cheering for Watson right now, I forgive you.
Karma is a bitch. Unless you can't say "bitch" in the Washington Post, in which case karma is, uh, a fickle strumpet or something.
Can the Washington Post software engineers reverse this so we readers provide the answers and Jennings provides the questions? It seems appropriate.
What is "Congratulations, you could be a drive time DJ!" Every radio personality in America has had this idea every time they try to interview me. In practice, it doesn't work so hot.
Ken - you are looking grim on the show last night. Enjoy the game! Thats the one thing Watson can't do.
Believe me, I was enjoying (almost) every second. Getting beat on the buzzer is frustrating, but are you kidding? I AM PLAYING A PRIME-TIME GAME SHOW AGAINST A SUPER-ADVANCED ROBOT! This is the coolest thing I will every do in my life by a factor of a million. The future is here.
Given that it is in IBM's (marketing) interest for Watson to win, what sort of oversight (if any) exists to make sure that Watson isn't getting the questions provided to him ahead of time or any funny business isn't taking place?
Not rigging a quiz show in America isn't just a good idea--IT'S THE LAW. Since the scandals of the 50s, it's actually been made a felony to screw around with the outcome of a game show. There was VERY careful oversight (an independent auditor) on every aspect of gameplay here. Plus, from IBM's point of view, this is a research experiment. (Well, and a PR bonanza.) They don't want their data tainted either.
I have already read plenty of doomsday reports for the blue-collar workforce that this technology could replace customer service representatives, in-patient counseling, bank tellers, cashiers, etc. Do you think Watson could replace Alex Trebek? Would you be excited to be part of a game-show where a computer asked questions to another computer?
I don't know if it's doomsday, but yeah, I think it's inevitable that lots of your question-answering day will eventually be re-outsourced from some guy in India to some piece of software.
That said, I don't think this means it's lights out for trivia or quiz shows. The analogy I use is track and field: humans keep running races even though cars and trains have been faster than us for more than a century. It's all about the human psychology of the contest, not just the outcome.
Believe me, computers have been able to play Wheel of Fortune and The Price Is Right at near-grandmaster levels for years, yet those shows soldier on.
When you were on your epic winning streak, it seemed you were able to intimidate other players - you would get on a run and they would have that 'give up' look. Is this really part of your 'game', and how does it affect you since you since watson is immune to this?
This is a big difference. Fair or not, lots of human opponents of mine would just take themselves out of the game once they found out who they were playing. Watson never gets stage fright. Watson never gets frustrated by wrong answers, or psyched out by the caliber of the opposition. It doesn't know or care who I am. It just keeps coming. It's Terminator, with a friendlier avatar.
There was an article in the Post the other day about trivia now being trivial because of the ability to look up everything in an instant. What is your take on this? Has Watson changed that? Also, just in general, how do you know so much trivia?
It's true that trivia geeks like me are much less useful a public resource in the days of Google (and not Watson). I worry that, just as we don't remember phone numbers now that we all carry mobile phones, we'll start to think we can stop knowing facts that we carry smartphones. I think this would be trouble for a lot of reasons. Facts aren't nuisances, no matter what certain presidential administrations would have you believe. They're the basis for informed decisions and analytical thinking.
On the other hand, in a not-too-distant future where nobody knows their state capitals anymore, maybe trivia geeks will be revered for their even more fantastical-seeming abilities! We will be like gods to you, carried on litters to your feasts.
Ken, What do you think of this "man vs. machine" version of Jeopardy? I found it interesting that in the middles of last night's Jeopardy, the producers of Jeopardy showed Watson getting questions horribly wrong when they tested it a few times, so they made some fixes and had it play 100 games. It is easy to feed a machine tons of facts, but I find it more interesting when the game tests 3 HUMAN contestants. This seems gimmicky to me, and I'm a fan of the show! Your thoughts ?
It's admittedly a gimmick, but outside the little world of Jeopardy, it has huge ramifications. Not just business-related ones ("Will IBM's stock price go up?"), but cultural and even philosophical ones. Will machines someday think? If so, do they possess consciousness? How does this change life for Earth's other lifeforms, namely us? The ideas surrounding artificial intelligence have always fascinated me, and Watson is an amazing breakthrough in one small corner of the field.
On last night's show, I noticed you buzzing in even when you didn't know the answer right away, taking a second after Alex called on you to finish reading the question and give an answer. In your opinion, is this the only way to beat Watson?
Good human players do this all the time: you buzz when you see something that trips some "This looks familiar!" switch in your brain and count on dredging it out in the five seconds after Alex calls on you.
Watson can't do this: it only buzzes once it has an answer in mind and a sufficiently high confidence interval. As weird as it sounds, yes, the human brain still has a speed advantage over a 2,880-processor-core computer.
Did Watson's voice throw you off your game at all?
Yes, because I wanted him to sound like Darrell Hammon doing Sean Connery on SNL's Celebrity Jeopardy instead. "That'sh not what your mother shaid lasht night, Trebek!"
Which idea bothers you more: a Watson victory or a Brad Rutter victory?
Both Brad and I are on the record as cheering for our own species, even if it means cheering for the other guy.
That said, Brad spanked me pretty badly last time we played, so the chance for a rematch with him was a nice sidelight for me.
How does one even begin to prepare for Jeopardy? Does the show suggest areas on which to concentrate?
Nope...as fans of the show know, the categories can be any question under the sun. Savvy would-be contestants know to bone up on lists like world capitals, US Presidents, and so on, but, honestly, cramming will only get you so far. The way to win on Jeopardy is to be a rabidly curious, information-omnivorous person your entire life. This is definitely a specific "type" of person, I learned while researching my book Brainiac, about these information sponges. There must be a gene for it. It's in our DNA.
Was there anybody in the crowd cheering for you or were they all Watson supporters?
It was an all-IBM crowd: programmers, executives. Stockholders all! They wanted human blood. It was gladiatorial out there. The stage had a big Watson logo on it too. This was definitely an away game for humanity.
Hi Ken! I also really like the charity VillageReach and am wondering how you decided to support it?
I'm glad someone asked this, so I could finally end with a plug that doesn't mention my books. Half the human's winnings go to charity in these matches--and all of Watson's winnings, since I presume it doesn't have a mortgage to pay like Brad and I do.
It took me agonizing weeks to pick a charity, but once I started talking to the folks at VillageReach, I knew I was on the right track. It's a Seattle-based nonprofit that works on getting medical supplies like vaccines to the most remote parts of the world--not just Mozambique, but the teeny middle-of-nowhere village in Mozambique, which is apparently the hard part of the equation.
It's undeniably worthy work, and it's slightly wonkish, which seemed Jeopardy-appropriate. The GiveWell philanthropy index rates them as the nation's top charity when it comes to effectiveness and transparency. They estimate that VillageReach saves another child's life every time you donate just several hundred dollars. How could I say no?
If anyone has a few bucks to spend, or is going to be playing a quiz show against a super-intelligent robot anytime soon and needs to select a charity, I highly recommend the fine folks at VillageReach.