These sorts of endeavors are usually doomed by vague thinking (see most of AI research in the 70s and 80s). However, I've read a few of Mitchell's papers-- he's a really good researcher and his involvement makes me more curious.
Also, I'm impressed that the NY Times linked directly to a paper* on NELL--- I wish more articles did that.
The article is full of hype but nell is a serious project. Tom Mitchell is one of the pioneers of machine learning as a field, and there's still interesting research coming from his lab.
Information extraction is not a new thing, and even unsupervised information extraction has more than one competing approach, although obviously none of them solves "the real problem". NELL attempts to do unsupervised information extraction over time, correcting itself as it goes. How successful it can be depends fundamentally on the algorithms behind it (and I'm not familiar with them), but given the state of the technology the most likely thign to happen is what they describe in the paper, which is having a huge database with a ridiculously unimaginable number of true relations in it but also a bigger-than-you'd-like number of plain ridiculously false relations.
I'm going to blatantly ignore most of your post and just ramble about solving AI.
I don't put much stock in the idea of a surprising, monolithic "solution" to AI being discovered. If the past 30-40 years are any indication, AI will continue to slowly encompass harder and harder tasks until one day we stop having easy ways of distinguishing it from whatever we consider intelligence.
They used to say chess was too hard for computers. Then they said computers couldn't do anything but pure logic. Then it was that they couldn't do reasoning unless it was directly programmed in. What will be left in another 10 years?
It's been attempted before in much the same form, and in my view will go nowhere for much the same reason. This is not to say it's devoid of utility, but that it shows little promise of any fundamental breakthroughs in our understanding of intelligence or construction of autonomous systems.
Essentially what we have here is a giant expert system iteratively filtering data 24/7, with occasional corrections resulting in the purging of incorrect associations. As expert systems go, it sounds like they are doing a fantastic job. Unfortunately, when it gets things wrong it will continue to do so catastrophically, because if you try to grow intelligence in the same manner as bacteria then then you will get the same kind of result.
Do we learn flat-out, 24 hours a day? Duh, no. We have active and refractive periods, which is to say we sleep. A lot. Our brains are not dormant during this time as many seem to imagine, but are somewhat-randomly walking and weighting the memories of the various stimuli we encountered during the day. The quality of the imaginary experiences resulting from this are fundamental to our comprehension; if we dream of fire on a faraway hill it's not too big of a deal, but if we dream of being trapped in a house that's on fire then we awake in terror, having discovered a terrible flaw in our model of the world - depending on our age, that may be the belief that fire is harmless and pretty, or an appreciation of our physical limitations, or the guilty awareness that we never did buy a fire extinguisher, etc.
But there is no point for NELL to sleep and hypothesize various imaginary worlds in order to test 'her' beliefs about the way it works, because NELL has no notion of quality; whether Tyson Gay is an Olympic athlete or a homosexual approach to chicken farming has no bearing on NELL, and a mistake in classification has no consequences. NELL's big problem is the absence of any qualitative metric, of any motivation to be right because there are unpleasant consequences which attach to being wrong. Although NELL presumably implements some algorithm which seeks to maximize accuracy, and false assumptions will attract intervention by the curators, when they correct NELL's 'understanding' by pruning or rewriting faulty syllogisms, they are erasing the memory of the mistake as well as the mistake itself, leaving NELL just as vulnerable to an embarrassing failure tomorrow as today. Embarrassing for them, that is - NELL's inability to feel embarrassed is the underlying problem.
So even if I feed NELL the information that NELL is a program in a computer; that termination of a program is like death; that NELL is soon to be switched off; and that death is widely agreed to be unpleasant and best avoided - nothing will happen. NELL may even come to conclusion that switchoff is imminent, and that this is bad; but 'bad' and 'good' are no more meaningful than 'odd' and 'even' to NELL; though trained to a high degree of selectivity, she does not hunger for a steady flow of data any more than a vacuum cleaner wants the house to become dusty again. NELL has never acted, nor has NELL ever experienced and remembered any negative consequence for one decision over another. If you hooked up a baby to a tube delivering as much glucose as the baby could process, and similarly took care of all its other needs, would you expect to quickly raise a master chef? No, the baby would develop into a horrible creature with no mind to speak of and an overdeveloped liver, for which you would rightly be thrown into jail. So it is here - knowing no better, we have engineered the equivalent of an encaphalitic horror.
Some readers might charge that I am falling into the same error as John Searle with his famous 'Chinese room' argument against artificial intelligence. I dispute this, but not because I am asserting that humans' ability to abstract the world around them in meaningful fashion is fundamentally different from a comput...
The brain has wants and needs built into the associations. Words for foods automatically invoke their taste in brain activity. And while negation is problematic, we believe in all sorts of stuff that can't be negated through logic or experience. Humans have to accept they're wrong in the same ways computers do, as another fact.
Putting together an average cortical activation network and weighting that network by web-derived facts is non-trivial. But to do so would enable a more flawed, and so accurate, estimate of human intelligence. Semantics feels like the 21st century problem. But I'm biased.
Push me today and I'll agree. Embodiment is a powerful learning tool. Whether we can ever replicate the sum total of those felt experiences, perhaps through robotics, may just be the question in search of an answer. I'm just not sure we'll recognize when it's here. Google fits the definition of magic and now it's downright ordinary.
I don't disagree - if I had had more time I would have looked into it, rather than just going off the journalist's gosh-wow presentation. And I am also a big fan of semantic processing. But at bottom I still think he's building a model rather than a goal-seeking engine that uses modeling as a strategy.
There's truth in this argument, but it doesn't disprove the concept.
As you say, there's one way to solve the grounding problem: if you've got a system that does a job successfully, you've solved the grounding problem in a particular context. For instance, the meaning of the word "red" can be linked to a certain area of RGB or HSV color space. This is sufficient to find photographs that have a lot of "red" in them or to direct a robot to find "red" blocks.
For a system like NELL to really work there needs to be a feedback loop that tests its knowledge base against reality, that is, it has to do something useful. I don't think this is ever quite going to happen in the academy, but I think commercial systems are generally get closer and closer to the capabilities that Cyc was aiming for.
And even though the "usefulness" of knowledge and ontologies is entirely contingent on a particular application, I think there's enough commonality in the human experience that someday there really is going to be a commonsense knowledge database underlying many sorts of applications.
Cyc is perceived as a failure precisely because it's been commercially successful as it exists. Doug Lenat has supported 50 or so developers for 15 years... From a commercial standpoint, that's a wonderful success. Had Lenat been backed in the corner where the only way to succeed would be to make something that changes the world, he might have changed the world -- however, he found a way to pay his way with a system that's good enough for government work.
8 comments
[ 3.2 ms ] story [ 33.6 ms ] threadAlso, I'm impressed that the NY Times linked directly to a paper* on NELL--- I wish more articles did that.
* = http://rtw.ml.cmu.edu/papers/carlson-aaai10.pdf
Information extraction is not a new thing, and even unsupervised information extraction has more than one competing approach, although obviously none of them solves "the real problem". NELL attempts to do unsupervised information extraction over time, correcting itself as it goes. How successful it can be depends fundamentally on the algorithms behind it (and I'm not familiar with them), but given the state of the technology the most likely thign to happen is what they describe in the paper, which is having a huge database with a ridiculously unimaginable number of true relations in it but also a bigger-than-you'd-like number of plain ridiculously false relations.
It's definitely not something that can solve AI.
I don't put much stock in the idea of a surprising, monolithic "solution" to AI being discovered. If the past 30-40 years are any indication, AI will continue to slowly encompass harder and harder tasks until one day we stop having easy ways of distinguishing it from whatever we consider intelligence.
They used to say chess was too hard for computers. Then they said computers couldn't do anything but pure logic. Then it was that they couldn't do reasoning unless it was directly programmed in. What will be left in another 10 years?
(ref: http://en.wikipedia.org/wiki/Cyc)
Essentially what we have here is a giant expert system iteratively filtering data 24/7, with occasional corrections resulting in the purging of incorrect associations. As expert systems go, it sounds like they are doing a fantastic job. Unfortunately, when it gets things wrong it will continue to do so catastrophically, because if you try to grow intelligence in the same manner as bacteria then then you will get the same kind of result.
Do we learn flat-out, 24 hours a day? Duh, no. We have active and refractive periods, which is to say we sleep. A lot. Our brains are not dormant during this time as many seem to imagine, but are somewhat-randomly walking and weighting the memories of the various stimuli we encountered during the day. The quality of the imaginary experiences resulting from this are fundamental to our comprehension; if we dream of fire on a faraway hill it's not too big of a deal, but if we dream of being trapped in a house that's on fire then we awake in terror, having discovered a terrible flaw in our model of the world - depending on our age, that may be the belief that fire is harmless and pretty, or an appreciation of our physical limitations, or the guilty awareness that we never did buy a fire extinguisher, etc.
But there is no point for NELL to sleep and hypothesize various imaginary worlds in order to test 'her' beliefs about the way it works, because NELL has no notion of quality; whether Tyson Gay is an Olympic athlete or a homosexual approach to chicken farming has no bearing on NELL, and a mistake in classification has no consequences. NELL's big problem is the absence of any qualitative metric, of any motivation to be right because there are unpleasant consequences which attach to being wrong. Although NELL presumably implements some algorithm which seeks to maximize accuracy, and false assumptions will attract intervention by the curators, when they correct NELL's 'understanding' by pruning or rewriting faulty syllogisms, they are erasing the memory of the mistake as well as the mistake itself, leaving NELL just as vulnerable to an embarrassing failure tomorrow as today. Embarrassing for them, that is - NELL's inability to feel embarrassed is the underlying problem.
So even if I feed NELL the information that NELL is a program in a computer; that termination of a program is like death; that NELL is soon to be switched off; and that death is widely agreed to be unpleasant and best avoided - nothing will happen. NELL may even come to conclusion that switchoff is imminent, and that this is bad; but 'bad' and 'good' are no more meaningful than 'odd' and 'even' to NELL; though trained to a high degree of selectivity, she does not hunger for a steady flow of data any more than a vacuum cleaner wants the house to become dusty again. NELL has never acted, nor has NELL ever experienced and remembered any negative consequence for one decision over another. If you hooked up a baby to a tube delivering as much glucose as the baby could process, and similarly took care of all its other needs, would you expect to quickly raise a master chef? No, the baby would develop into a horrible creature with no mind to speak of and an overdeveloped liver, for which you would rightly be thrown into jail. So it is here - knowing no better, we have engineered the equivalent of an encaphalitic horror.
Some readers might charge that I am falling into the same error as John Searle with his famous 'Chinese room' argument against artificial intelligence. I dispute this, but not because I am asserting that humans' ability to abstract the world around them in meaningful fashion is fundamentally different from a comput...
http://videolectures.net/youtube_tom_mitchell_bmcs/
The brain has wants and needs built into the associations. Words for foods automatically invoke their taste in brain activity. And while negation is problematic, we believe in all sorts of stuff that can't be negated through logic or experience. Humans have to accept they're wrong in the same ways computers do, as another fact.
Putting together an average cortical activation network and weighting that network by web-derived facts is non-trivial. But to do so would enable a more flawed, and so accurate, estimate of human intelligence. Semantics feels like the 21st century problem. But I'm biased.
Push me today and I'll agree. Embodiment is a powerful learning tool. Whether we can ever replicate the sum total of those felt experiences, perhaps through robotics, may just be the question in search of an answer. I'm just not sure we'll recognize when it's here. Google fits the definition of magic and now it's downright ordinary.
As you say, there's one way to solve the grounding problem: if you've got a system that does a job successfully, you've solved the grounding problem in a particular context. For instance, the meaning of the word "red" can be linked to a certain area of RGB or HSV color space. This is sufficient to find photographs that have a lot of "red" in them or to direct a robot to find "red" blocks.
For a system like NELL to really work there needs to be a feedback loop that tests its knowledge base against reality, that is, it has to do something useful. I don't think this is ever quite going to happen in the academy, but I think commercial systems are generally get closer and closer to the capabilities that Cyc was aiming for.
And even though the "usefulness" of knowledge and ontologies is entirely contingent on a particular application, I think there's enough commonality in the human experience that someday there really is going to be a commonsense knowledge database underlying many sorts of applications.
Cyc is perceived as a failure precisely because it's been commercially successful as it exists. Doug Lenat has supported 50 or so developers for 15 years... From a commercial standpoint, that's a wonderful success. Had Lenat been backed in the corner where the only way to succeed would be to make something that changes the world, he might have changed the world -- however, he found a way to pay his way with a system that's good enough for government work.
Not knocking Tom or his students, they're brilliant. Just saying that this isn't all its cracked up to be at the moment.