Would love to hear some inside baseball on Project Debater, or even the planning of the event itself. It's been a while since one of these Big Blue demos panned out once removed from the venue, so I think they'd do well to be as transparent as possible about technology, API-accessible demos, etc.
Call me a cynic, but I don't trust anything IBM puts out on AI. There have been so many bullshit PR articles, with lots of nice working packaging bullshit. This one feels the same.
It's so discouraging hearing all about Watson and health for past X years and then reading AI insider articles refuting the work as total bullshit and a PR job to keep IBM relevant today.
I think you made the same.mistake as the media - thinking that today's AI systems are more general than they truly are.
IBM did build a computer that could play jeapordy. But being able to play jeapordy doesn't tell us how good any of the subcomponents are at, e.g. understanding doctors notes/journal articles.
The same thing goes here, they built a system that can debate with some ability. Can it be used to spot fake news as a researcher says? Maybe some subcomponents could be reused for that, but there's no guarantee that they will be good enough for that task either.
You really have to take modern AI at face value and not extrapolate from what it actually does to anything else, otherwise you will whipsaw between optimism and cynicism.
Actually, real debate is about responding to the other debaters arguments and countering them. Thus this IBM marketing show was not really a debate.
This was a demo that IBM could synthesize a plausible argument that supports or refutes a given assertion. That's interesting and a bit impressive (presuming its argument isn't simply a regurgitation of some position paper it found online). But without understanding cause and effect, its arguments will remain very superficial, probably driven by a small catalog of argument 'frames' (templates) that adaptively fill a handful of slots like [needs] [means] [goal] [conflicts] and [emotional hooks]. Using such simple recipes it can produce verbiage that plausibly sounds humanlike, but isn't actually reasoning. It's likely that the system couldn't even diagnose help desk problems using basic logic, e.g. backtracking to thise dependencies that might have caused the given outcome, thereby identifying only the possible and plausible causes.
What you write is true, but this mistake was deliberately spread through IBM's hype, such as seemingly empty promises to apply Watson to all sorts of business, medical, and political problems. This hype is the reason these demonstrations get as much attention as they do. This hype, which IBM uses to sell unrelated services, also seems to be the only thing IBM actually gets out of funding all of its AI development.
I worked at IBM research for a year and I got the impression that they have quite a few people who do absolutely top-notch work. As soon as it's time to commercialize the management machine kicks in and pretty much kills any kind of innovation though. So I believe they have the tech at a research/prototype level but have no ability to bring it to market.
I took it as the author being somewhat unsettled by the appearance of dissembling, but regardless, the explanation given by Jeff Welser suggests something more mundane: it generates a bunch of candidate responses, and sometimes its scoring algorithm picks one that does not address the points made by the other side.
An accusation of dissembling implies intent, and that probably requires some sort of theory of mind, to make assumptions about how the response will be received. The author of the article seems to be anthropomorphizing, attributing greater cognitive powers to the system than it is displaying.
Welser also says "there’s been no effort to actually have it play tricky or dissembling games", leaving open the question of whether it has nevertheless been exposed, in training, to arguments that avoid the issue, and whether that has influenced the way in which it constructs and scores candidate replies.
Something I read on the anthropomorphisation of AI really clicked with me, and explains much of the media and marketing wrongness.
If I tell a random person on the street that my AI can play checkers, they think, "A human can play checkers. A human can also drive a car. Therefore, if an AI can play checkers it must also be able to drive a car."
People use AI tasks as markers that AI is AGI / human, then extrapolate to logical abilities it must have.
Which, if you don't know what a matrix is, is fairly understandable.
For a few years I have been predicting the advent of a kind of cognitive weapon of mass destruction, a "propaganda doomsday weapon."
This would be a dishonest/manipulative debate AI paired with big data and microtargeting. It's like pairing a trained and primed con artist with every single human being in parallel and having it shadow and work on them 24/7. Big data and cross agent "fleet wide learning" would allow it to get smarter very rapidly.
I consider this scenario to be as much of an existential threat as atomic and biological weapons.
I have a 1920x1080 24" display and you can't fit more than eight words in width on the screen and feel the need to hide all the navigation in a hamburger menu that disappears when scrolling? This is some of the actual worst design I've seen, ever.
There's random call-to-action buttons, spaces where I couldn't tell if something was loading or if the blank was intentional, things fading and moving in too slowly, and animations that are just plain weird.
To me it looks like bold design that got a bit too...bold.
Edit: Don't get me wrong, the Plex typeface is great, but the website dedicated to it is wonky at best.
Thought experiment: Given the mass unemployment that would come from autonomous self driving vehicles and AI agents that can replace humans: if you had the tech in place would you release it, or wait for something like Universal Basic Income to be rolled out first?
I don't buy the idea that IBM is ahead of Google by a long shot, but the idea that Google is waiting for a Universal Basic Income to release its economy wrecking AIs is even sillier.
I know this is hugely unpopular, but...that sounds like a problem that communism was initially designed for, where productivity is so high that people can just take what they need.
This is unexpected, but now I'm curious. How are you using "AI" here? Are there other things you would point to that indicate IBM is ahead of everyone else?
The problem with their ML strategy is that it consistently over-promises and under-delivers. The marketing and messaging doesn't trail the research; the research is driven by the corporate messaging by half a decade or more. It's completely backwards, and that's just the start, but it's what you should expect from a corporation paralysed by risk aversion.
Some of the insider postmortems on Watson Health that are floating around on the internet are... interesting.
I suspect that they are not genuinely pursuing AI as a business, but as a means to advertise their brand in order to sell their more conventional services.
IBM is good at AI research in silod teams and applications, terrible at turning it into a product and integrating the Watson portfolio. Worked there too, devops'n several Watson techs.
IBMs biggest problems are still cultural, middle management and recruiting / retaining skilled developers. This is why they cannot deliver on the hype their amazing marketing teams create.
" How can they be good at ML/AI if the people that work there have a fraction of the intelligence of researchers at FB/Google? "
They have plenty of very, very smart people. The researchers are not IBM's problem but the management that is not capable of bringing that research to market.
I’m skeptical. There are lots of companies that top ML experts turn their nose at (Salesforce, Amazon) and I can’t imagine that IBM would attract better people than that.
Do you think that only top researchers are capable of moving the needle? And that these researchers only work at brand name AI companies.
It seems FB and Google have had more negative societal impacts (ie fake news, public manipulation) than IBM, Salesforce, and numerous other niche AI companies.
We all need a more nuanced perspective on all thinks than this absolutism we have embraced as a society, online.
> I still maintain that IBM is above everyone else in the AI game, everyone.
Yeah I don't know about that. At best they are starting to incorporate deep learning into Watson. I'm guessing Watson has always been a rules engine and more specifically one designed with a lot of statistical analysis. That's far below the level of sophistication we're at with NNs.
While it might be nice to use whatever that tool is - it's a wrapper on top of Tensorflow/Keras or Pytorch... Wouldn't it make sense for it to be a wrapper on top of "Watson's API" whatever that is.
> Yeah I don't know about that. At best they are starting to incorporate deep learning into Watson. I'm guessing Watson has always been a rules engine and more specifically one designed with a lot of statistical analysis.
There's so much confusion in the market as to what Watson is, and much of it is IBM Marketing's fault. Watson is a Natural Language Processing platform. At a low level it does all the standard NLP stuff - tokenization, part of speech tagging, lemmatization, dependency parsing, co-reference resolution, stopword removal, synonym expansion, etc. Some of this is rule based, some is ML based. Building on top of that you have stuff like an Intent Classifier, Passage Retrieval with a Ranker, Named Entity extraction, Relationship extraction, and Sentiment tagging. Building on top of that, you have the actual commercial applications - chatbot/assistant, semantic search engine, knowledge graph, etc. That's what it is. It's not vaporware or smoke and mirrors, but it's obviously not AGI either.
Like I say in a previous comment, and if memory serves, the original Watson system that beat the two human players in Jeopardy already used deep learning to train the transitions of a natural language chart parser. They didn't just suddendly discover deep learning. It's probably just their marketing team being hopelessly clueless about the tech.
I don't think you're right. All the exciting new papers/conference proceedings are coming from Google/Facebook/etc, what's an exciting work that IBM did recently? What IBM is good at is marketing, I don't know of actual advances in modern ML that are coming from there.
>> I still maintain that IBM is above everyone else in the AI game, everyone.
IBM are much more open to mixing up statistical machine learning and symbolic AI, and benefit from the advantages of both. For instance, Watson used a chart parser with transitions learned using deep learning [1] to fill in frames [2] from a natural-language sentence (the challenge sentence in the Jeopardy game) and also Prolog to identify relations between elements of the parse so as to figure out the sentence's context.
Other big AI players are pretty much exclusively focused on statistical machine learning and, in particular, deep learning, and then again for very specialised tasks like image recognition, speech recognition and machine translation. IBM on the other hand is building broader-scope systems, that perform more nuanced, less strict functions; like in this example, debating, or question answering (essentially) with Watson.
So yes, in some sense, IBM is way ahead of the others in creating powerful AI systems. Just let's not confuse IBM's research with the products it tries to sell based on that research.
___________
[1] You'll have to take my word for that- I've read the Watson research papers but my pdf copies of them are now all on a hard-drive that has since crashed and I don't remember how to find them online.
[2] A cognitive data structure first proposed by Minsky in the '70s:
IBM is just remarketing old systems as AI, there are plenty of companies, Google as one of the main one, that produce AI innovations, this it's just a PR stunt
Sounds pretty much like how human debates are already structured:
1. Grab key points from influential books and articles to create templated responses
2. Do no original research and thus be completely unable to rebut criticisms of the source data or analysis
3. Counter #2 above by falling back on crowd-pleasing templated responses, forcing the moderator to move the debate along
4. Never, ever concede anything, because "debates" are no longer about listening to an argument and carefully considering their merits, it is a time-boxed verbal combat sport where there must be a winner and a loser.
You think you're getting closer to the truth by being cynical, but what you're really doing is overshoot the target so far that you're basically just as far from the reality of debates as the uncynical view is.
its still the best way we have ... its the process of spectating the debate that the viewer can consume two opposing sides and then make a decision. To think that a debate itself could actually create truth is to give too much faith in the skill and equal skill of either debater. And in reality, no subject is completely one sided in its conclusions, it healthy to be able to make peace with paradoxes and contradictions. Requiring a "only one right way" standpoint is dogmatic, stubborn, and mentally weak.
A good debate gives the viewer a strong mind by being able to hold opposing views simultaneously.
How can we rule out the possibility that Watson, Sophia etc are some WiFi operated, human controlled PR stunts?
I'm not saying they are, but even when it did beat Kasparov, where was the proof, code we can run ourselves and see, witnesses, just basic accountability stuff that we can make us safely believe that these are not staged?
Seeing this with the Google Assitance "scandel" of claims that alot of the demos didn't add up. How do we know this is really working. Most startup that say they are doing "ML/AI" tech are just exporting the data to factories of workers in developing countries to shift the data. Besides in research papers, I have yet to see these claims be palpable in reality.
Deep blue beating Kasparov obviously wasn't staged because there was no human strong enough to beat Kasparov at chess. The jeopardy thing wasn't staged (but also wasn't necessarily that impressive). To this day you can download a chess engine greater than the strongest human for free online (stockfish).
This, however, is almost certainly a human-controlled PR-Stunt.
Asking for evidence is fine, but let's not conflate this with something real like chess.
1. There were only a handful of players around during those matches who could hold a candle to Kasparov. A team of the top GMs could maybe topple him, but that would require a significant conspiracy by individuals with no clear incentive to hype-up computer chess.
2) It wasn't long until you could run open source code on your home PC that could crush Kasparov.
I suspect that much of the power of Watson comes from it's data sets and preprocessing, and that technically it wouldn't look nearly as impressive as Deep Mind's work. Open sourcing its core algorithms might not give any useful insights, especially if one lacks the data sets and a powerful enough custom-built machine.
I agree with you however, small set of training data could be made available with a downloadable Watson jr software so that you can see it playing chess against you, also proving somehow it's not just a clever algorithm and it's ML. I'm simplifying to make my point.
I can't even assume we all agree winning against Kasparov is proof enough. Just for argument's sake, you can buy him out, convince him that this is an amazing PR for him as well - lose one game, be double famous for life, just as you can convince a boxer to lose a match. None of this tells me 100% that Watson is AI capable of winning a chess game in the 90s or hold an intellectual debate in 2018.
Again, I'm not saying it didn't happen, I'm saying burden of proof is bothering me.
A good friend of mine who is a VP Cloud at IBM said when I asked last year "no one can explain Watson at ibm - it's a black box" I also wrote a few apps with tensorflow.
Kasparov, winning Go game, playing fortnite, sure.What they are suggesting here and with Sophie, without proof of any kind, is beyond me.
Considering the Watson Jeopardy was more a PR stunt than an actual demo (no voice recognition, questions given as plain text, speed advantage of a computer) I'll hold my opinion until any technical details come out.
There is an article claiming that an AI held its own against a human in a debate. But the supplied video gives just a few seconds of canned material which could apply to any debate. Am I missing a video somewhere? Even on their website they don't seem to want to show an actual debate so that we can judge for ourselves whether it "held its own".
But now I am suspicious that it always debates the same people (they make reference to having worked with the machine for over a year). If that is the case, then they know precisely what to say to trigger certain canned responses (it makes jokes that were quite obviously well-planned by a human). The rest of the material seems to just be content culled from online documents and read out by the machine.
The achievement here, if there is one, is to parse online material and decide which snippets support a given argument and which don't.
I've long been interested in seeing debate formalized. That is, a back and forth structured discussion with the end goal of arriving at a narrowly scoped truth. One strategy for winning could include attacking core assumptions underlying an argument, thus weakening any claims that followed from that assumption (question the methodology/replication of a study that claim vaccines cause autism).
The problem I have with debates in popular culture is that it's often performative and participants devolve to whoever is best at dominating a discussion, or whoever is the more eloquent with not much light at the end of the tunnel.
This is a new low for IBM. If they had actually produced the system they are describing, they would bring a person to debate the machine in real time, and they would have an uncut clip of that happening, because it would be an absolute technical marvel.
Obviously I can not prove that they haven't developed a miracle device which can construct complete arguments and theories in real time, but if they had done, they'd've shown something more convincing.
"Watching the debate, I figured the answer was that it didn’t quite get it, but I wasn’t positive. I couldn’t tell the difference between an AI not being as smart as it could be and an AI being way smarter than I’ve seen an AI be before. It was a pretty cognitively dissonant moment. Like I said, unsettling."
... reminds me of when Kasparov lost to Deep Blue, when it made a strange move. He thought the move was too sophisticated for a computer. Fifteen years later, one of Big Blue's designers said the move was the result of a bug in Deep Blue's software.
Shame on you, Dieter Bohn, for either accepting bribes from IBM or happily regurgitating their marketing spume like some sick parody of a bird feeding it's child.
Pretty much everyone who works in tech knows that IBM Marketing has made much more progress in technology than IBM AI. Please don't add to the noise and confusion the average person is already under by writing an article so askew from reality. Even from the limited and carefully trimmed Debater quotes that you chose to include, it is apparent that what's going on is merely a gimmick. It is also apparent that you cherry picked the best sounding responses, then stripped any context away from them that might have exposed your charade.
So again, shame on you Dieter Bohn, for deceiving your audience. As a journalist you have a responsibility to educate and inform, and you have failed at that responsibility I'm this article.
Debates have really gone off the deep end with their scoring rules. They should be about convincing the audience not scoring based on how many points of the other side were responded to. If the other side makes a ridiculous claim the audience can see through it.
It does mean that topics need to be apolitical in order to ensure a fair score - no debate will turn a Republican into a Democrat. But for other topics this scoring works extremely well.
Does this mean AI is getting good or humans (at least ones AI has been compared to) are pretty low bar to clear and just chinese-rooming the arguments would pass as a debate? Reading some internet forums, it's hard not to think a bot could produce many of the comments. Watching some political coverage only intensifies the feeling. How many of the debates, on average, can be classified as more than chinese-rooming?
I wonder if anybody did a sort of reverse Turing test - just like regular one but no computers involved at all, unknown to the tester which is told one of them is an AI, and how many people would be declared failing it?
> Another IBM researcher suggested that this technology could help judge fake news.
That part is actually scary. Opaque algorithms from IBM would decide for us what are facts and what are fakes? And they'd know how? Because IBM marketing dept said they're so good? Thanks but no thanks.
79 comments
[ 3.2 ms ] story [ 139 ms ] threadIt's so discouraging hearing all about Watson and health for past X years and then reading AI insider articles refuting the work as total bullshit and a PR job to keep IBM relevant today.
Don't know why this would be any different.
IBM did build a computer that could play jeapordy. But being able to play jeapordy doesn't tell us how good any of the subcomponents are at, e.g. understanding doctors notes/journal articles.
The same thing goes here, they built a system that can debate with some ability. Can it be used to spot fake news as a researcher says? Maybe some subcomponents could be reused for that, but there's no guarantee that they will be good enough for that task either.
You really have to take modern AI at face value and not extrapolate from what it actually does to anything else, otherwise you will whipsaw between optimism and cynicism.
This was a demo that IBM could synthesize a plausible argument that supports or refutes a given assertion. That's interesting and a bit impressive (presuming its argument isn't simply a regurgitation of some position paper it found online). But without understanding cause and effect, its arguments will remain very superficial, probably driven by a small catalog of argument 'frames' (templates) that adaptively fill a handful of slots like [needs] [means] [goal] [conflicts] and [emotional hooks]. Using such simple recipes it can produce verbiage that plausibly sounds humanlike, but isn't actually reasoning. It's likely that the system couldn't even diagnose help desk problems using basic logic, e.g. backtracking to thise dependencies that might have caused the given outcome, thereby identifying only the possible and plausible causes.
I, for one, am not excited about the prospect of AI-generated logical fallacies.
Dismantling such is where the real work is at. Give me a heads up when they automated that.
An accusation of dissembling implies intent, and that probably requires some sort of theory of mind, to make assumptions about how the response will be received. The author of the article seems to be anthropomorphizing, attributing greater cognitive powers to the system than it is displaying.
Welser also says "there’s been no effort to actually have it play tricky or dissembling games", leaving open the question of whether it has nevertheless been exposed, in training, to arguments that avoid the issue, and whether that has influenced the way in which it constructs and scores candidate replies.
If I tell a random person on the street that my AI can play checkers, they think, "A human can play checkers. A human can also drive a car. Therefore, if an AI can play checkers it must also be able to drive a car."
People use AI tasks as markers that AI is AGI / human, then extrapolate to logical abilities it must have.
Which, if you don't know what a matrix is, is fairly understandable.
This would be a dishonest/manipulative debate AI paired with big data and microtargeting. It's like pairing a trained and primed con artist with every single human being in parallel and having it shadow and work on them 24/7. Big data and cross agent "fleet wide learning" would allow it to get smarter very rapidly.
I consider this scenario to be as much of an existential threat as atomic and biological weapons.
Dismantling such is where the real work is at. Give me a heads up when they automated that.
Now all the AIs know how to earn chmod775's trust.
Seriously, LOOK at this:
https://i.imgur.com/an7857N.png
I have a 1920x1080 24" display and you can't fit more than eight words in width on the screen and feel the need to hide all the navigation in a hamburger menu that disappears when scrolling? This is some of the actual worst design I've seen, ever.
https://www.ibm.com/plex/
There's random call-to-action buttons, spaces where I couldn't tell if something was loading or if the blank was intentional, things fading and moving in too slowly, and animations that are just plain weird.
To me it looks like bold design that got a bit too...bold.
Edit: Don't get me wrong, the Plex typeface is great, but the website dedicated to it is wonky at best.
Thing is that the media doesn't pay proportional attention and respect they do to other companies that haven't proven anything yet.
Where's Google thing that they showed last I/O where an AI would call a business to schedule an appointment? Vaporware.
Some of the insider postmortems on Watson Health that are floating around on the internet are... interesting.
It's being rolled out at right now: https://www.theverge.com/2018/12/5/18123785/google-duplex-ho...
Sounds like they are gearing up for a launch.
IBMs biggest problems are still cultural, middle management and recruiting / retaining skilled developers. This is why they cannot deliver on the hype their amazing marketing teams create.
They have plenty of very, very smart people. The researchers are not IBM's problem but the management that is not capable of bringing that research to market.
It seems FB and Google have had more negative societal impacts (ie fake news, public manipulation) than IBM, Salesforce, and numerous other niche AI companies.
We all need a more nuanced perspective on all thinks than this absolutism we have embraced as a society, online.
Yeah I don't know about that. At best they are starting to incorporate deep learning into Watson. I'm guessing Watson has always been a rules engine and more specifically one designed with a lot of statistical analysis. That's far below the level of sophistication we're at with NNs.
This is IBMs page on deep learning:
https://www.ibm.com/cloud/deep-learning
While it might be nice to use whatever that tool is - it's a wrapper on top of Tensorflow/Keras or Pytorch... Wouldn't it make sense for it to be a wrapper on top of "Watson's API" whatever that is.
There's so much confusion in the market as to what Watson is, and much of it is IBM Marketing's fault. Watson is a Natural Language Processing platform. At a low level it does all the standard NLP stuff - tokenization, part of speech tagging, lemmatization, dependency parsing, co-reference resolution, stopword removal, synonym expansion, etc. Some of this is rule based, some is ML based. Building on top of that you have stuff like an Intent Classifier, Passage Retrieval with a Ranker, Named Entity extraction, Relationship extraction, and Sentiment tagging. Building on top of that, you have the actual commercial applications - chatbot/assistant, semantic search engine, knowledge graph, etc. That's what it is. It's not vaporware or smoke and mirrors, but it's obviously not AGI either.
IBM are much more open to mixing up statistical machine learning and symbolic AI, and benefit from the advantages of both. For instance, Watson used a chart parser with transitions learned using deep learning [1] to fill in frames [2] from a natural-language sentence (the challenge sentence in the Jeopardy game) and also Prolog to identify relations between elements of the parse so as to figure out the sentence's context.
Other big AI players are pretty much exclusively focused on statistical machine learning and, in particular, deep learning, and then again for very specialised tasks like image recognition, speech recognition and machine translation. IBM on the other hand is building broader-scope systems, that perform more nuanced, less strict functions; like in this example, debating, or question answering (essentially) with Watson.
So yes, in some sense, IBM is way ahead of the others in creating powerful AI systems. Just let's not confuse IBM's research with the products it tries to sell based on that research.
___________
[1] You'll have to take my word for that- I've read the Watson research papers but my pdf copies of them are now all on a hard-drive that has since crashed and I don't remember how to find them online.
[2] A cognitive data structure first proposed by Minsky in the '70s:
https://en.wikipedia.org/wiki/Frame_(artificial_intelligence...
1. Grab key points from influential books and articles to create templated responses
2. Do no original research and thus be completely unable to rebut criticisms of the source data or analysis
3. Counter #2 above by falling back on crowd-pleasing templated responses, forcing the moderator to move the debate along
4. Never, ever concede anything, because "debates" are no longer about listening to an argument and carefully considering their merits, it is a time-boxed verbal combat sport where there must be a winner and a loser.
A good debate gives the viewer a strong mind by being able to hold opposing views simultaneously.
I'm not saying they are, but even when it did beat Kasparov, where was the proof, code we can run ourselves and see, witnesses, just basic accountability stuff that we can make us safely believe that these are not staged?
Does anyone know anything?
Is this possible?
If so, then call it Data Science or Push-button analysis.
Is this a reference that I’m not picking up on?
Deep blue beating Kasparov obviously wasn't staged because there was no human strong enough to beat Kasparov at chess. The jeopardy thing wasn't staged (but also wasn't necessarily that impressive). To this day you can download a chess engine greater than the strongest human for free online (stockfish).
This, however, is almost certainly a human-controlled PR-Stunt.
Asking for evidence is fine, but let's not conflate this with something real like chess.
Thank you. Unless they prove to us that it is not, it is.
PS. Chess game could also be staged without having to find a better player. My other answer is above.
1. There were only a handful of players around during those matches who could hold a candle to Kasparov. A team of the top GMs could maybe topple him, but that would require a significant conspiracy by individuals with no clear incentive to hype-up computer chess.
2) It wasn't long until you could run open source code on your home PC that could crush Kasparov.
I suspect that much of the power of Watson comes from it's data sets and preprocessing, and that technically it wouldn't look nearly as impressive as Deep Mind's work. Open sourcing its core algorithms might not give any useful insights, especially if one lacks the data sets and a powerful enough custom-built machine.
I can't even assume we all agree winning against Kasparov is proof enough. Just for argument's sake, you can buy him out, convince him that this is an amazing PR for him as well - lose one game, be double famous for life, just as you can convince a boxer to lose a match. None of this tells me 100% that Watson is AI capable of winning a chess game in the 90s or hold an intellectual debate in 2018. Again, I'm not saying it didn't happen, I'm saying burden of proof is bothering me. A good friend of mine who is a VP Cloud at IBM said when I asked last year "no one can explain Watson at ibm - it's a black box" I also wrote a few apps with tensorflow.
Kasparov, winning Go game, playing fortnite, sure.What they are suggesting here and with Sophie, without proof of any kind, is beyond me.
https://www.youtube.com/watch?v=PkSzmnA1CQQ https://www.youtube.com/watch?v=ZIY1uSxL-qQ
But now I am suspicious that it always debates the same people (they make reference to having worked with the machine for over a year). If that is the case, then they know precisely what to say to trigger certain canned responses (it makes jokes that were quite obviously well-planned by a human). The rest of the material seems to just be content culled from online documents and read out by the machine.
The achievement here, if there is one, is to parse online material and decide which snippets support a given argument and which don't.
The problem I have with debates in popular culture is that it's often performative and participants devolve to whoever is best at dominating a discussion, or whoever is the more eloquent with not much light at the end of the tunnel.
Obviously I can not prove that they haven't developed a miracle device which can construct complete arguments and theories in real time, but if they had done, they'd've shown something more convincing.
"Watching the debate, I figured the answer was that it didn’t quite get it, but I wasn’t positive. I couldn’t tell the difference between an AI not being as smart as it could be and an AI being way smarter than I’ve seen an AI be before. It was a pretty cognitively dissonant moment. Like I said, unsettling."
... reminds me of when Kasparov lost to Deep Blue, when it made a strange move. He thought the move was too sophisticated for a computer. Fifteen years later, one of Big Blue's designers said the move was the result of a bug in Deep Blue's software.
IBM is a marketing machine.
Pretty much everyone who works in tech knows that IBM Marketing has made much more progress in technology than IBM AI. Please don't add to the noise and confusion the average person is already under by writing an article so askew from reality. Even from the limited and carefully trimmed Debater quotes that you chose to include, it is apparent that what's going on is merely a gimmick. It is also apparent that you cherry picked the best sounding responses, then stripped any context away from them that might have exposed your charade.
So again, shame on you Dieter Bohn, for deceiving your audience. As a journalist you have a responsibility to educate and inform, and you have failed at that responsibility I'm this article.
It does mean that topics need to be apolitical in order to ensure a fair score - no debate will turn a Republican into a Democrat. But for other topics this scoring works extremely well.
I wonder if anybody did a sort of reverse Turing test - just like regular one but no computers involved at all, unknown to the tester which is told one of them is an AI, and how many people would be declared failing it?
> Another IBM researcher suggested that this technology could help judge fake news.
That part is actually scary. Opaque algorithms from IBM would decide for us what are facts and what are fakes? And they'd know how? Because IBM marketing dept said they're so good? Thanks but no thanks.