A nice thought-parody of arguments about creating General AI.
It would be nice to have an updated version because the option 'we just need more computing power' (and training data) is now working pretty well for many machine learning domains (i.e. deeplearning).
Sure, throwing more computing power at a problem that's already conceptually solved, more or less, is going to yield results. But that won't help for creating general ai because it's not conceptually solved.
Can you give an example of something we do that's not conceptually solved?
It seems to me, (from a notably uneducated in AI perspective), that when dealing something deterministic like computing, general AI either works or it doesn't. You can't "do" something that's not conceptually solved in this domain, because what you are really doing is something else, albeit something that is very much like the goal.
What I meant by my comment was, I understood stinkytaco's comment as a request for a specific task that humans do (so, nothing as nebulous as "intelligence") that hasn't been "conceptually solved" with something like a walk through a search tree.
Even Go was "conceptually solved" decades ago- it's "only" iterating over a search tree; we were just waiting for the hardware and tree-pruning algorithms to catch up. Humor and other human tasks can be more precisely defined but still don't even have theoretical algorithms to solve them.
I didn't mean that humans can't do things that haven't been conceptually solved. I agree that humans can do humor and that humor isn't conceptually solved, so I'll assume that you mean to say that we can make computers do humor, even though it isn't conceptually solved.
I disagree that we can make computers do humor. Kobeya linked to a paper that shows unsupervised joke generation, but it's a stretch to call this humor. I haven't explored beyond what's in that article, but the examples of jokes produced do little to convince me that this program is doing humor:
I like my relationships like I like my source, open
I like my coffee like I like my war, cold
I like my boys like I like my sectors, bad
Show me an AI that can produce original joke forms and I'll reevaluate.
Go was not conceptually solved decades ago if it required the development of new tree-pruning algorithms to work. Improving an algorithm is conceptual innovation. If it could have worked with only a few orders of magnitude difference in performance, or with only marginally inferior results without new algorithms then maybe it was conceptually solved decades ago, else no.
You study these things on fixed forms because it is easier to analyze and understand what is going on. But the extension to general joke-making is straight forward, provided you have working NLP (a big if!): data-mine the concept graph for thoughts/memes which have similar structural content but surprisingly (in the technical sense) different meanings, then construct a sentence that maximizes probability during reading of the first interpretation but ends with the second. This captures basically all humor from puns to "I like my X like I like Y: Z" to most forms of stand-up comedy.
Interesting. Of course, computers tell bad jokes, I'm told.
Visual processing also comes to mind. Computers can "see", but it's very difficult to understand the many rules humans use to parse their environment, thus computers are not yet at human level of visual processing.
Makes me wonder if those things aren't "solved" by our brains, but we just don't quite know how. Another thing worth noting is that I know people who have no sense of humor. They simply do not understand jokes or sarcasm.
Well we understand it (mostly) now, but at the time it was created: Specifics of aerodynamics required for flight.[1]
The Wright brothers basically mimicked nature and tested till they got it right. They didn't use Bernoulli's equation to create a NACA airfoil and L/D max calculations. [2].
To that end, in one of my undergraduate Aeronautics classes, when we were doing a bunch of Bernoulli equations someone calculated that based on that approach helicopters couldn't actually fly. To which our instructor said, "Oh yea, helicopters fly by magic."
This is a decent example, but it's not quite comparable to developing general AI. We can build helicopters without a full conceptual understanding of aerodynamics, but we can't produce a physical simulation which will accurately determine which helicopter designs will and will not fly for any given design without a full understanding of aerodynamics under the conditions in which helicopters operate.
Creating aerodynamics simulation is creating an aerodynamic system. Creating a general intelligence is creating an intelligent system. A better analog to creating a helicopter using incomplete understanding of aerodynamics is something like creating a system that can discriminate between what is and isn't a general intelligence. We don't have such a system, but it at least doesn't seem like the sort of thing that would require a full understanding of intelligence.
That seems at best totally unrelated? You've asserted that general AI is some sort of different category than other things which we can and have built without understanding the fundamentals of their operation. But you haven't actually provided any real evidence of that.
You have a basis for that assertion? The human mind is basically lots of computing power, lots of data, and simple rules -- since all growth and learning is local, it must be simple in that sense at least. Yes we don't understand the neuroscience fully yet, but we do understand the ways in which the underlying mechanisms must be constrained. And we routinely build software programs with structure and rules that are vastly more complex than the neurons that make up our gray matter.
Maybe automatic hyper-parameter optimization of adversarial networks is all that it takes for general intelligence. If you look up the conference proceedings of NIPS or AAAI you'll see major progress being made on basically every type of learning and memory you can find in a psychological textbook. Of course these are controlled experiments being done, in the technical sense, and not agglomerative. But what's stopping us from combining a bunch of these algorithms together, or better yet a base framework that generically supports them all, and making it self-reflective? Just computing power and available data, really.
So no, it's not at all clear that this is a problem that is "not conceptually solved." People keep saying that whenever the subject comes up, but the reality on the ground is otherwise.
The question still seems up for argument but there is a fairly popular view that "General AI" whatever that is, is an aergent property many local system working in concert.
If you think that we have all the conceptual understanding to assemble a general artificial intelligence then you should be able to give an outline of how it would work — beyond just lots of computing power and data, that doesn't differentiate the problem form the problem of object classification or natural language processing. Maybe what you mean to say is that once we have the computing power and data collection technology required, so that researches can experiment, the unsolved conceptual problems will become easy to solve. Do you have a basis for that assertion? Regardless, if that's what you meant then the fact remains that the conceptual problems are not all solved.
I won't say that it's inconceivable that a sufficiently large and powerful neural net, with a simple structure based on simple rules, given a very comprehensive set of training data, could acheive a level of general intelligence.
But we don't have evidence of intelligence arising that way in the past. What we have is the human brain, an incredibly complex structure resembling a neural net with simple rules for individual node (maybe?). It's the result of evolutionary processes, which means that we should expect it to have extremely hard-to-comprehend properties (example [0]). We've seen evolutionary processes produce intelligence, but not learning processes. Again, that's not to say that evolutionary processes are necessary, just that we can be sure that they can lead to intelligence, and that we can't be sure for other methods.
> If you think that we have all the conceptual understanding to assemble a general artificial intelligence then you should be able to give an outline of how it would work — beyond just lots of computing power and data, that doesn't differentiate the problem form the problem of object classification or natural language processing.
> Maybe what you mean to say is that once we have the computing power and data collection technology required, so that researches can experiment, the unsolved conceptual problems will become easy to solve.
No, I mean that there appears to be a basis of a solution already. Actually, multiple solutions being pursued by different groups. It's like asking a rocket engineer in 1955 how to build a rocketship to the Moon, or a physicist in 1936 how to build an atomic bomb, or the Wright brothers in 1900 how to build an airplane. Sure, in every single one of these cases you wouldn't get an exact, definitive answer. The Wright brothers didn't even understand the aerodynamics of their airplane, for example. But there were known avenues of inquiry for which there was very solid reason to believe that they would not be dead ends.
We're at a point now with AI/ML where solutions can be learned by machines for any solvable problem. It just needs some humans doing the selection of algorithms and guiding in hyper parameter space. But there is active research on automating that meta level which is yielding results. And both the reinforcement learning and the older AGI communities have working, tested designs for cognitive architectures that are truly general.
I'm not claiming we're done. I'm just saying that we're basically at the level of a working Wright flyer -- a bunch of research projects individually exhibit intelligence in separate domains, and a couple of cognitive architectures for generalizing them which have been shown to work on toy problems. There's no known unknowns that would cause these approaches to fail, so the reasonable expectation is that in the coming decades we will see the rise of useful AGI. Just like a reasonable observer with all the facts in 1905 should have predicted consumer passenger air travel.
> But we don't have evidence of intelligence arising that way in the past. What we have is the human brain, an incredibly complex structure...
Yes the human brain is difficult to understand. So is the flight of a bird. It's a good thing that we don't need to replicate the mechanics of bird flight to build flying machines -- otherwise you and I would still be stuck to trains and boats for getting around.
I suggest looking not at a neuroscience text book but a psychology textbook. Ask yourself not whether you can replicate exactly the conditions going on in the brain, but rather ask if you can implement a program to the same general functional description as the psychology textbook provides. That's a much easier task, and one well within the capabilities we have today.
A surprising number of the things that have been solved in the past decade with more computing powers were things that were not conceptually solved, and which we had no good ideas how to solve.
A good example is Google Translate. We stopped trying to "understand" language and just threw a large dataset at pattern matching. And it worked amazingly well.
For other things we throw ideas of "this might improve what we can already do" and it winds up effectively solved, whether or not we understand it. This happened, for example, with Alpha-Go.
So "conceptually solved" and "AI can do it" are pretty much disconnected.
This was actually my breakthrough for Machine Learning. Prior to really understanding ML and DL I thought it was just an update to expert systems and that we still needed to "understand intelligence" before we created it.
The genius of the RL feedback loop makes me think we can get to AGI without understanding human intelligence holistically.
I still don't understand why google translate is often cited as something that works. I've never seen it perform much better that just looking every word in a dictionary (which i always end up doing, since you're never sure it picked the correct translation for the specific context)
To me google translate seems like the perfect illustration of the shortcomings of this purely data driven approach...
Alpha go, on the other hand is a much much more elaborated algorithm ( from my very limited knowledge) and it does produce amazing results.
The reason why you don't understand is because it's shortcomings are your baseline, while more experienced people's baseline was people spending tens of millions to get something that only worked for toy examples.
Yes, I can list plenty of cases where translation fails. For example not understanding that in a conversation about specialist topic X (eg Ruby programming, or the game of G) that seemingly common terms have specific meanings. But I can also use it on a random newspaper article in a different language and expect to understand the result. This was something that I did not expect to happen in my lifetime.
I see now that "conceptually solved" is just too ambiguous. Translation is conceptually solved, in the sense that I meant to evoke, to the extent that it's solved. We have all the conceptual understanding necessary to translate between human languages. We don't have a full conceptual model of language, but we discovered that that's not necessary for translation.
Suppose a team of reseachers were given a computing platform a trillion times more powerful than the best systems available today, and a trillion dollars to build a set of training data. Would they be able to create a general AI in a predictable timeframe? If not then there's at least some area where we need to develop conceptual understanding. My intuition says that this is the case.
If you think that general AI can, in fact, be developed the same way as google translate, then what do you think the training data would look like? I'd imagine that the translation training set looks something like a massive Rosetta stone, though I haven't looked into it. Inputs associated with a valid output. What would a set of input/output pairs look like for a general AI? Any valid English query/statement/remark/essay with an "intelligent" response? Like a chatterbot? Doesn't seem like that would work. Or maybe I'm wrong. Maybe a chatterbot with ungodly amounts of computational resources and training data would develop the ability to reason. Maybe I'm just not appreciating how much that could change the game.
But it seems more likely that the first general AI will look more like Alphago, where we started with a reasonably solid understanding of how to approach the game, and then introduced deep learning where we discovered it was the best approach. We don't have a reasonably solid understanding of how to organize a general intelligence, and we don't know what it would take to get there.
> We have all the conceptual understanding necessary to translate between human languages.
Except that we don't. The original Google Translate used the conceptual modeling approach to translation, and it was absolute shit for any non-closely related language pairs. Google Translate (and Baidu, etc.) now uses deep networks whose operations are rather opaque. We don't understand in detail how they work. But that didn't stop us from building them.
In context, that claim was an explanation of what I was trying to get at with the phrase "conceptual understanding". We don't have a conceptual understanding of language. We do have a conceptual understanding of how to translate between human languages — Proof: we can translate between human languages. If you think that's not adequate proof then you're disagreeing with me on the semantics of "conceptual understanding". If you have an alternative name for that then I'd appreciate it because I think my usage is confusing.
It's not a useless notion of "conceptual understanding" though, because while being able to have a program complete a task implies having the conceptual understanding necessary for that task, it's possible to have the conceptual understanding but not be able to make a program, if, for instance, it would require greater (within reason) computing resources.
It does sound pretty useless because the same logic can be used to prove that we have a conceptual understanding of general intelligence, because after all we exist and we think.
Google Translate itself may have been done through statistical machine translation, but that wasn't Google's first attempt at the subject.
In particular Google spent around $100 million in 2003 for Applied Semantics, in part hoping that their natural language processing expertise could be used for translation. That technology was superseded by statistical techniques.
I disagree strongly with this view that hard problems need to be "conceptually solved" in a vacuum first. That experimentation and resources don't matter.
Computing increases have enabled a ton more experimentation with AI. Experiments that would have taken years to do, can be done in hours now. Experimentation lets scientists develop intuition about their algorithms. And get a sense of what will and won't work. And have a better mental model of the problem.
Not to mention the benefit of having 10x or more funding and researchers working on it now that results have been shown.
It's a common myth that there has been little or no innovation in NNs and that it's just a matter of computing power. Taking the best algorithms of the 90s and running them on modern hardware would still give you poor results. For instance, most of the 90s research was on very shallow nets, they didn't know how to properly train deep ones.
What innovations do you have in mind? LSTMs and convnets were both invented in the 90s. Stochastic gradient descent, momentum, etc. have been around for decades. I feel like it's quite the opposite of what you are saying -- in the 2000s there was a lot of innovation, but it mostly turned out to be unnecessary.
What has improved is the variety of architectures, but I don't think there has been anything fundamentally new after LSTMs. The only exception I can think of is generative adversarial networks, and even ideas behind adversarial training have been around in other domains for many years now.
I believe the author himself said that computation improvements does make it easier to solve general AI. I think in value alignment theory there are still problems that they don't know how to do properly even with infinite computation though.
I'd rephrase the last line as the lesson is more the classic early optimization mistake. If you can't make something as smart as a fish then don't worry so much about super-human levels of optimization.
The mathematically innumerate comments are almost a troll that if the thought experiment worked and an artificial mathematician were invented, a significant fraction of the real world would not recognize or agree with its results.
There's a very cruel saying about people should do what they're born to do, with the dark insinuation that grandma the knitter should be locked in a sweatshop to sew against her will or (insert trendy CS tech here) children should be euthanized as a general policy until the tech gains significant commercial traction. Anyway unless an AI is quite heavily socialized into our culture what we're likely to grow might be ideally suited to research Klingon Warp Engine Fields in the 24th century but to us the output is going to look like hard to compress digital noise, so there's that problem.
This is indeed a brilliant article, at least as far framing the problem with some objections goes.
Of course, what's missing from hypothetical artificial arithmetic is simple symbolic manipulation. And I am pretty sure that "More gofi", a more explicit treatment of AI, is something that Noam Chomsky and other have as their favorite bullet point on how to go further.
However, that too has been tried to whatever degree. If you had an overt, tractable logical theory that explained all intelligent behavior, then yes you'd have the missing ingredient of human intelligence. But unlike arithmetic calculation, it seems unlikely that human intelligence has this quality.
Indeed, all human "dealing with the world" behavior together involves a black box whose entirety is not subject to rational or logical reflection or description but whose broad outline. A person can't give a complete-enough-to-write-an-algorthim account of walking down the street recognizing things but the person can likely give a good why the street light they see is a street light - ie, heuristic black box behavior and logical/deductive behavior is intimately tied within human behavior.
And this might give some clue what's missing modern AI.
>Of course, what's missing from hypothetical artificial arithmetic is simple symbolic manipulation. And I am pretty sure that "More gofi", a more explicit treatment of AI, is something that Noam Chomsky and other have as their favorite bullet point on how to go further.
Unlikely as Minsky (the prince of gofai) and Chomsky were, umm, hardly fans of each other's work.
This reminds of Jeff Hawkins' book On Intelligence. He takes a similar stance on the overarching/conceptual solution vs. more specialized solutions that dance around the knowledge gap. He criticizes neural networks as not sufficiently detailed to mirror the brain's behavior, and proposes the Hierarchical Temporal Memory model, or HTM.
The hierarchy is crucial: as in the brain, there are different regions at various levels of cognition, with higher regions responsible for greater levels of abstraction. A moment of real learning/understanding is characterized by a sudden, system-wide switch of signals between layers mostly traveling up the hierarchy (this input is confusing! I don't know what to do with it!) to signals mostly cascading down (AHA! I get what that means! Let's do this in response).
It was written in 2004, so like this article it lacks some of the insight we've gained over the past decade or so. But I think some of its general insights are still relevant. Whether it's NNs or HTMs or some yet-to-be-discovered algorithm that ends up being the driver of general AI, this article definitely resonates. At the most general levels of AI, we are still in the "confused" phase: viewing the state of AI as a whole system, our signals are still largely questions going up the conceptual hierarchy, rather than understanding cascading down.
> With the graphical-model insight in hand, you can give a mathematical explanation of exactly why first-order logic has the wrong properties for the job, and express the correct solution in a compact way that captures all the common-sense details in one elegant swoop.
What was the compact, correct way referring to here?
>the label "thirty-seven" is meaningful, not because of any inherent property of the words themselves
either this "label" has no place in the story or it is not meant to be part of the story, but then our number-words do have an inherent structure, so they are easy to understand.
44 comments
[ 3.3 ms ] story [ 103 ms ] threadIt would be nice to have an updated version because the option 'we just need more computing power' (and training data) is now working pretty well for many machine learning domains (i.e. deeplearning).
It seems to me, (from a notably uneducated in AI perspective), that when dealing something deterministic like computing, general AI either works or it doesn't. You can't "do" something that's not conceptually solved in this domain, because what you are really doing is something else, albeit something that is very much like the goal.
Even Go was "conceptually solved" decades ago- it's "only" iterating over a search tree; we were just waiting for the hardware and tree-pruning algorithms to catch up. Humor and other human tasks can be more precisely defined but still don't even have theoretical algorithms to solve them.
I disagree that we can make computers do humor. Kobeya linked to a paper that shows unsupervised joke generation, but it's a stretch to call this humor. I haven't explored beyond what's in that article, but the examples of jokes produced do little to convince me that this program is doing humor:
I like my relationships like I like my source, open
I like my coffee like I like my war, cold
I like my boys like I like my sectors, bad
Show me an AI that can produce original joke forms and I'll reevaluate.
Go was not conceptually solved decades ago if it required the development of new tree-pruning algorithms to work. Improving an algorithm is conceptual innovation. If it could have worked with only a few orders of magnitude difference in performance, or with only marginally inferior results without new algorithms then maybe it was conceptually solved decades ago, else no.
Visual processing also comes to mind. Computers can "see", but it's very difficult to understand the many rules humans use to parse their environment, thus computers are not yet at human level of visual processing.
Makes me wonder if those things aren't "solved" by our brains, but we just don't quite know how. Another thing worth noting is that I know people who have no sense of humor. They simply do not understand jokes or sarcasm.
http://homepages.inf.ed.ac.uk/s0894589/petrovic13unsupervise...
The Wright brothers basically mimicked nature and tested till they got it right. They didn't use Bernoulli's equation to create a NACA airfoil and L/D max calculations. [2].
To that end, in one of my undergraduate Aeronautics classes, when we were doing a bunch of Bernoulli equations someone calculated that based on that approach helicopters couldn't actually fly. To which our instructor said, "Oh yea, helicopters fly by magic."
[1]https://secretofflight.wordpress.com/incorrect-theories/ [2]https://wright.nasa.gov/overview.htm
Creating aerodynamics simulation is creating an aerodynamic system. Creating a general intelligence is creating an intelligent system. A better analog to creating a helicopter using incomplete understanding of aerodynamics is something like creating a system that can discriminate between what is and isn't a general intelligence. We don't have such a system, but it at least doesn't seem like the sort of thing that would require a full understanding of intelligence.
Maybe automatic hyper-parameter optimization of adversarial networks is all that it takes for general intelligence. If you look up the conference proceedings of NIPS or AAAI you'll see major progress being made on basically every type of learning and memory you can find in a psychological textbook. Of course these are controlled experiments being done, in the technical sense, and not agglomerative. But what's stopping us from combining a bunch of these algorithms together, or better yet a base framework that generically supports them all, and making it self-reflective? Just computing power and available data, really.
So no, it's not at all clear that this is a problem that is "not conceptually solved." People keep saying that whenever the subject comes up, but the reality on the ground is otherwise.
These people wouldn't happen to be domain experts...?
I won't say that it's inconceivable that a sufficiently large and powerful neural net, with a simple structure based on simple rules, given a very comprehensive set of training data, could acheive a level of general intelligence.
But we don't have evidence of intelligence arising that way in the past. What we have is the human brain, an incredibly complex structure resembling a neural net with simple rules for individual node (maybe?). It's the result of evolutionary processes, which means that we should expect it to have extremely hard-to-comprehend properties (example [0]). We've seen evolutionary processes produce intelligence, but not learning processes. Again, that's not to say that evolutionary processes are necessary, just that we can be sure that they can lead to intelligence, and that we can't be sure for other methods.
[0] https://www.damninteresting.com/on-the-origin-of-circuits/
Sure:
https://www.amazon.com/Engineering-General-Intelligence-Part...
> Maybe what you mean to say is that once we have the computing power and data collection technology required, so that researches can experiment, the unsolved conceptual problems will become easy to solve.
No, I mean that there appears to be a basis of a solution already. Actually, multiple solutions being pursued by different groups. It's like asking a rocket engineer in 1955 how to build a rocketship to the Moon, or a physicist in 1936 how to build an atomic bomb, or the Wright brothers in 1900 how to build an airplane. Sure, in every single one of these cases you wouldn't get an exact, definitive answer. The Wright brothers didn't even understand the aerodynamics of their airplane, for example. But there were known avenues of inquiry for which there was very solid reason to believe that they would not be dead ends.
We're at a point now with AI/ML where solutions can be learned by machines for any solvable problem. It just needs some humans doing the selection of algorithms and guiding in hyper parameter space. But there is active research on automating that meta level which is yielding results. And both the reinforcement learning and the older AGI communities have working, tested designs for cognitive architectures that are truly general.
I'm not claiming we're done. I'm just saying that we're basically at the level of a working Wright flyer -- a bunch of research projects individually exhibit intelligence in separate domains, and a couple of cognitive architectures for generalizing them which have been shown to work on toy problems. There's no known unknowns that would cause these approaches to fail, so the reasonable expectation is that in the coming decades we will see the rise of useful AGI. Just like a reasonable observer with all the facts in 1905 should have predicted consumer passenger air travel.
> But we don't have evidence of intelligence arising that way in the past. What we have is the human brain, an incredibly complex structure...
Yes the human brain is difficult to understand. So is the flight of a bird. It's a good thing that we don't need to replicate the mechanics of bird flight to build flying machines -- otherwise you and I would still be stuck to trains and boats for getting around.
I suggest looking not at a neuroscience text book but a psychology textbook. Ask yourself not whether you can replicate exactly the conditions going on in the brain, but rather ask if you can implement a program to the same general functional description as the psychology textbook provides. That's a much easier task, and one well within the capabilities we have today.
A good example is Google Translate. We stopped trying to "understand" language and just threw a large dataset at pattern matching. And it worked amazingly well.
For other things we throw ideas of "this might improve what we can already do" and it winds up effectively solved, whether or not we understand it. This happened, for example, with Alpha-Go.
So "conceptually solved" and "AI can do it" are pretty much disconnected.
The genius of the RL feedback loop makes me think we can get to AGI without understanding human intelligence holistically.
To me google translate seems like the perfect illustration of the shortcomings of this purely data driven approach...
Alpha go, on the other hand is a much much more elaborated algorithm ( from my very limited knowledge) and it does produce amazing results.
Yes, I can list plenty of cases where translation fails. For example not understanding that in a conversation about specialist topic X (eg Ruby programming, or the game of G) that seemingly common terms have specific meanings. But I can also use it on a random newspaper article in a different language and expect to understand the result. This was something that I did not expect to happen in my lifetime.
Suppose a team of reseachers were given a computing platform a trillion times more powerful than the best systems available today, and a trillion dollars to build a set of training data. Would they be able to create a general AI in a predictable timeframe? If not then there's at least some area where we need to develop conceptual understanding. My intuition says that this is the case.
If you think that general AI can, in fact, be developed the same way as google translate, then what do you think the training data would look like? I'd imagine that the translation training set looks something like a massive Rosetta stone, though I haven't looked into it. Inputs associated with a valid output. What would a set of input/output pairs look like for a general AI? Any valid English query/statement/remark/essay with an "intelligent" response? Like a chatterbot? Doesn't seem like that would work. Or maybe I'm wrong. Maybe a chatterbot with ungodly amounts of computational resources and training data would develop the ability to reason. Maybe I'm just not appreciating how much that could change the game.
But it seems more likely that the first general AI will look more like Alphago, where we started with a reasonably solid understanding of how to approach the game, and then introduced deep learning where we discovered it was the best approach. We don't have a reasonably solid understanding of how to organize a general intelligence, and we don't know what it would take to get there.
Except that we don't. The original Google Translate used the conceptual modeling approach to translation, and it was absolute shit for any non-closely related language pairs. Google Translate (and Baidu, etc.) now uses deep networks whose operations are rather opaque. We don't understand in detail how they work. But that didn't stop us from building them.
It's not a useless notion of "conceptual understanding" though, because while being able to have a program complete a task implies having the conceptual understanding necessary for that task, it's possible to have the conceptual understanding but not be able to make a program, if, for instance, it would require greater (within reason) computing resources.
In particular Google spent around $100 million in 2003 for Applied Semantics, in part hoping that their natural language processing expertise could be used for translation. That technology was superseded by statistical techniques.
Computing increases have enabled a ton more experimentation with AI. Experiments that would have taken years to do, can be done in hours now. Experimentation lets scientists develop intuition about their algorithms. And get a sense of what will and won't work. And have a better mental model of the problem.
Not to mention the benefit of having 10x or more funding and researchers working on it now that results have been shown.
It's a common myth that there has been little or no innovation in NNs and that it's just a matter of computing power. Taking the best algorithms of the 90s and running them on modern hardware would still give you poor results. For instance, most of the 90s research was on very shallow nets, they didn't know how to properly train deep ones.
What has improved is the variety of architectures, but I don't think there has been anything fundamentally new after LSTMs. The only exception I can think of is generative adversarial networks, and even ideas behind adversarial training have been around in other domains for many years now.
The mathematically innumerate comments are almost a troll that if the thought experiment worked and an artificial mathematician were invented, a significant fraction of the real world would not recognize or agree with its results.
There's a very cruel saying about people should do what they're born to do, with the dark insinuation that grandma the knitter should be locked in a sweatshop to sew against her will or (insert trendy CS tech here) children should be euthanized as a general policy until the tech gains significant commercial traction. Anyway unless an AI is quite heavily socialized into our culture what we're likely to grow might be ideally suited to research Klingon Warp Engine Fields in the 24th century but to us the output is going to look like hard to compress digital noise, so there's that problem.
Of course, what's missing from hypothetical artificial arithmetic is simple symbolic manipulation. And I am pretty sure that "More gofi", a more explicit treatment of AI, is something that Noam Chomsky and other have as their favorite bullet point on how to go further.
However, that too has been tried to whatever degree. If you had an overt, tractable logical theory that explained all intelligent behavior, then yes you'd have the missing ingredient of human intelligence. But unlike arithmetic calculation, it seems unlikely that human intelligence has this quality.
Indeed, all human "dealing with the world" behavior together involves a black box whose entirety is not subject to rational or logical reflection or description but whose broad outline. A person can't give a complete-enough-to-write-an-algorthim account of walking down the street recognizing things but the person can likely give a good why the street light they see is a street light - ie, heuristic black box behavior and logical/deductive behavior is intimately tied within human behavior.
And this might give some clue what's missing modern AI.
Unlikely as Minsky (the prince of gofai) and Chomsky were, umm, hardly fans of each other's work.
The hierarchy is crucial: as in the brain, there are different regions at various levels of cognition, with higher regions responsible for greater levels of abstraction. A moment of real learning/understanding is characterized by a sudden, system-wide switch of signals between layers mostly traveling up the hierarchy (this input is confusing! I don't know what to do with it!) to signals mostly cascading down (AHA! I get what that means! Let's do this in response).
It was written in 2004, so like this article it lacks some of the insight we've gained over the past decade or so. But I think some of its general insights are still relevant. Whether it's NNs or HTMs or some yet-to-be-discovered algorithm that ends up being the driver of general AI, this article definitely resonates. At the most general levels of AI, we are still in the "confused" phase: viewing the state of AI as a whole system, our signals are still largely questions going up the conceptual hierarchy, rather than understanding cascading down.
https://arxiv.org/abs/1410.0736
https://www.cs.toronto.edu/~rsalakhu/papers/HD_PAMI.pdf
What was the compact, correct way referring to here?
either this "label" has no place in the story or it is not meant to be part of the story, but then our number-words do have an inherent structure, so they are easy to understand.