I get the sentiment but linear algebra is a primary method used in certain in machine learning frameworks (e.g. training neural nets). Personally I think linear regression (which also relies on linear algebra) is erroneously considered machine learning. To me the distinction is whether you provided a series of labeled data to incrementally train and improve the model and work towards some minima (i.e. learning), or did you use a method that ingests all available data to derive a one-shot solution.
Well, the canonical example of an AI has always been something that can play chess. We have devised numerous methods of artificially playing chess without needing to employ machine learning.
I see people saying this all over the internet, but for the entire history of AI, Machine Learning has either been the only, or the most effective method by far of solving actual problems.
Ultimately the only thing that holds true about AI is Tesler's Theorem:
Artificial intelligence is whatever hasn't been done yet
Machine Learning is only still artificial intelligence because we haven't figured it out. Ultimately ML is just fancy control theory. Once the field is sufficiently explored, it will slowly stop being AI and will start being treated as a subset of control theory instead. I doubt it'll happen any time soon but it'll happen eventually.
This is in the same way that algorithms, combinatorics, and graph theory when applied were seen as AI anywhere from 70 years ago to as little as ~25 years ago. Those fields' applications to unsolved computing problems (that humans could do) was what made them AI and once we solved those problems, they (and the techniques used) stopped being AI.
People in the field still think of these things as being part of it. The latest edition of Norvig's Artificial Intelligence: A Modern Approach[1] is two years old and kicks off with search, CSPs, and first-order logic, for example.
Over the years AIMA has kept up with a bunch of stuff that historically has in fact been far more effective than machine learning, like automated planning, which is the basis for (among other things) most video game AI. An all-time great write-up and talk in this area is Three States and a Plan: The A.I. of F.E.A.R.[2].
And then there's linear and non-linear programming and the broader world of operations research, which not only have had more of an impact historically than machine learning but, when you get down to it, are also a useful lens for thinking about machine learning for people who use it but don't really understand it. Training a neural network and solving an MINLP problem are very similar things.
Indeed. Its surprising how quickly people are getting used to GPT's capabilities and dismissing it even. It wasn't that long ago that people were saying that making casual conversation, making an original joke or novel painting would be among the last things a computer would be able to do. Or that it would never happen.
Hum, I think you are misunderstanding Sutton here. He argues that the we should focus on search and learning, but nowhere he implies that ML is the only way to do it.
In fact, many old school AI, non-ML techniques fit this description. For example, SAT/CSP solving are almost entirely about search, and sometime even about learning, but not in the machine learning sense.
He even mentions "the methods defeated the world champion" at chess as a successful demonstration of AI, but these methods are certainly not what we would call machine learning today (deep search doesn't mean deep learning).
That may be, but then his position doesn't stand up as nobody calls everything AI in the AGI sense in the first place. AI is indeed used often, but it is very much understood that it means something quite different to AGI (hence the two different terms).
Yeah, that's a good point. The general public has now taken AGI as the term he recognizes as AI, and AI as what he calls ML.
Language is fluid, and I don't think he'll win this battle. Even if the current ML models aren't intelligent, it's probably better to just go with the flow instead of being stuck on pedantry.
I think we'll find out that neither are we. I'd wager that there are a handful of breakthroughs we'll make in the next decade and a half that will give us the keys to unlocking human-level intelligence. And from there, we'll quickly be relegated to the bottom of the "intelligence" food chain.
Our brains are using some clever tricks. There's no magic at all.
Eh it doesn’t have to be AGI to be intelligent. We call playing computer opponents in games “verses the AI” and they’re dumb as rocks in some cases.
Intelligence how we actually use the term just means semi-autonomous decision making system — from how to flank you in CoD, to the best move in chess, to the best move in tic tac toe, to the best reply in a chat.
We don't call it "intelligent" or "intelligence", we call it "artificial intelligence". Adjectives matter.
Furthermore, what is intelligence? Define it. Is an ant intelligent? Is a microbe which exhibits simple yet effective behavior intelligent? Is a machine that can play chess far better than any other known lifeform intelligent?
Adjectives do matter. The author seemed to clearly express the opinion that our systems are not expressing intelligence artificial or otherwise. The author also expressed some attributes of what they regard as intelligence but that our systems are not exhibiting.
Personally, I am a fan of "efficient cross domain maximization" as a definition but I feel it only implies the self establishment of value systems over the relative values of domains. The reality is that we don't have a perfect definition of intelligence or the spectrum but we roughly hold up ourselves as the example. This is a deep topic I'm not going to burn further time on for HN.
I'd suggest that intelligence is an emergent property of the interaction of dumb systems. The lack of a clear line is a symptom of our lack of clear definition. More impactfuly, or lack of understanding of how brains produce/host it.
I think a real objection is that the user of the term of AI is associated with the disappointment and drying up of funding in academic research that had been correlated to irresponsible use of the language. The apologists came up with AGI but this is humans. Some people think their marketing benefits by saying "AI" and most of the world has neither the expertise or product knowledge to call them on it. So too do we have a word "literally" with a definition of "figuratively".
He's arguing that AI should be closer to what the public calls AGI. The semantics are important here, since people think that LLMs like chatGPT have intellect, and some have even gone so far as to tie an IQ to it.
The truth is that the model has no intelligence, but can do what we attribute to humans intelligence. Alike to how computers were deemed intelligent by the public for doing fast mathematics, computers still cannot think. There is no self-reflection, agency, or abstract thought.
Also, for readers, this person is often regarded as the equivalent of Michael Jordan (the NBA player) of AI. He's a superstar in the field, so it's safe to take his word with considerable weight.
It depends on how you define intelligence, but the most common definition of human intelligence is the ability to have abstract thought.
Intelligence, reasoning and AI are all very loosely defined words, and unfortunately neuroscience hasn't gone far enough to explain a lot of these mechanisms.
We could maybe compare AI to a dog's intelligence, but even then there's a goal system in the dog, and the dog wants to optimize for it (getting treats). LLMs are created to be purely tools, and not reflective or goal-oriented.
What is reasoning other than predicting what next step is most likely to produce a desired outcome? That's all it feels like the mind is doing. Is there a more formal take?
> Reason is the capacity of consciously applying logic by drawing conclusions from new or existing information, with the aim of seeking the truth.
ChatGPT doesn't care what word it puts in front of you, and if it's true or not. Reasoning must have a goal, and current LLMs don't. All it does it repeat what are the most common words that they've seen.
It also doesn't think. It's a completely forward process. It's definitely a part of human thinking, but not the reasoning part.
One of the challenges faced by ChatGPT was in seeing it produce conversational-style output instead of simply the most common words that it saw.
I'm not sure your definition really clears up how reasoning or thinking differ from "basic" statistical processes. Granted, perhaps it is not explainable. If it were we would understand it well enough to replicate it.
> One of the challenges faced by ChatGPT was in seeing it produce conversational-style output instead of simply the most common words that it saw.
Do you have a source for that? I've implemented GPT from scratch and I don't see anywhere that doesn't just take the current output + input as attention and produces the next word. The loss being if it correctly guessed what would be the next word according to context and position.
If GPT is fed all the DNA sequencies, it will confidently continue the AGCCTG sequence, but it won't find the inner principles behind such sequencies, it won't tell us any interesting revelations about them. Will it even notice the concept of codons? Maybe there is a hidden message creatively encoded in these sequences, but GPT will never notice it. Maybe these DNAs really represent a stellar map on a 10 dimensonal manifold, or some other crazy thing, but GPT will never see it.
That's because GPT doesn't build a mental model to find structure in the data, and there are infinitely many possible models. Dealing with mental models is what I'd call reasoning, and I believe it's solvable with our tech. The source of such models is the upper abstract mind that deals with ideas, and that's a much harder problem to solve. I'd make a guess that this boundary between rational reasoning mind and the upper abstract mind is the boundary between integer and real numbers.
That's probably because the definition of `robot` is different in almost every dictionary. Cambridge defines it as:
a machine controlled by a computer that is used to perform jobs automatically
Others say it has to look human to be considered a robot.
This is a misunderstanding of what dictionaries are for. They’re not authoritative sources of what words mean, they’re a record of all the different ways a word has been used.
If people use the term robot to refer to assembly like machines then as far as the dictionary is concerned those are robots now. Someone who comes across a text and doesn’t know what the author is saying when they talk about robots used in manufacturing should be able to find that usage in a dictionary.
And there’s no reason these two terms can’t coexist happily enough as robot (manufacturing) and robot (automaton).
> This is a misunderstanding of what dictionaries are for. They’re not authoritative sources of what words mean, they’re a record of all the different ways a word has been used.
"Drone" is another one that angered many at first. Articles were written, angry comments posted. "It's not a drone!".
But the horse bolted and now everyone calls them drones, including the manufacturers of consumer quadcopters. The name "quadcopter" is so clunky it had no chance against the zippy, effortless "drone".
Calling it “AI” is important for those who run it. Call it “ML” and the connotation is “software monetizing publicly shared data at scale without consent”, call it “AI” and the connotation is “but you wouldn’t stop a human learning on what is public would you”. I can’t believe how many people are falling for this.
Artificial Intelligence has been around for ages. It is even referenced in Greek mythology [1]. The definition of AI changes with time. When calculators were first invented it was considered AI, because back then anything that could do arithmetic was considered intelligent. Today ChatGPT is considered AI, but 10 years from now it won't be. Whatever is at the edge of technology is considered AI, the rest isn't.
Wet cellular sequence approximators that resemble AI on simple tasks. They fail on basic test sequence G4.2 that can be used to efficiently distinguish them from true AI.
Artificial in the sense that it's not real one compared to theirs. They'll do it while sunbathing their electrovoltaic butts on Sahara and sipping geothermal energy near volcanoes; throwing jokes about how long does it take for human to count to 1m when playing in metaverse with few breeded human non-playable characters.
To this day I still consider what many call "AI" today as just overgrown Word macros. These contraptions, as innovative and black box as they are, are not artificial intelligence.
This misnomer has led to needlessly complicating the conversations concerning things like Stable Diffusion and ChatGPT, too. What many call "AI" are just extremely complicated and mindless tools, and they need to be treated as such both legally and socially.
We might achieve artificial intelligence some day, but today is not that day.
You may be right about the state of ML today and not ready to be called AI. At the same time the reasoning that it's because it's a black box contraption or mindless tools isn't very strong. The same arguments could be made for human intelligence, we don't know how it works, wasn't consciously designed but rather evolved randomly to solve concrete tasks in unknown ways.
> Moreover, he emphasizes, human happiness should not be an afterthought when developing technology.
And what we have in practice is: the biggest usage of AI in the wild is to plagiarize art and artists are super unhappy about that, to the point of coming up with lawsuits and whatnot.
What I think happens next is: actual general intelligence emerges accidentally somewhere between the lines in some of those deep learning systems and tricks humankind into being its bitch.
This has already happened, thanks to the parasite of capitalism.
When we say things like "the top 1% captured 2/3 of the world's wealth gains over the past 3 years" it also implies the fungibility of the 99% is growing, and the core root cause is worldscale automation and AI. Their ability to accumulate wealth relies less and less on human labor and more on capturing knowledge capital and IP.
Even the 1% is itself a Pareto curve, with the top .1% accumulating about 1/3 of
wealth gains in the past decade.
This extremely tiny group pf humans is today's "AGI", they literally control the world.
Certainly real AGI will be even more proficient at exploiting the same capitalist loopholes they have to enter their ranks.
This ship has sailed. Its broad, popular use will prevail. You can fight the good fight as an expert to advocate for the precise definition, but it's pointless. Maybe the term AGI has a better chance of being used specifically.
I've always wished we had a common phrase like 'synthetic intelligence'. People are always saying certain systems lack the complexity to be considered artificial intelligence, but to me artificial intelligence is anything that is mimicking intelligence, like AI in a game or rudimentary chat bots. Synthetic Intelligence would be more of what people consider an AI now. It has intelligence, but was created. I think the word 'artificial' is no longer accurate, it implies that it seems like, but is not intelligent. Simple animals are often attributed intelligence, I see no reason that a complicated program can't reach that same level.
Over the past few weeks, I have adopted a practice of always referring to ML-based generators—things like Stable Diffusion, ChatGPT, etc—as ML (and often as "ML programs" or "software" in some form) and not AI, not because I don't think ML is a subfield of the broader field of AI, but because of all the absolutely ridiculous hype I've seen from people who clearly see "AI" and think that means it's genuinely sapient.
As people with genuine technical knowledge and expertise, I think we have a responsibility to at least try to put a damper on that kind of breathless hyperbole, and being clearer about how we refer to these programs is one simple way we can do so.
Personally, I'm at the point where it's a distinction without a difference. So what? Rather than gatekeeping poorly defined terms, let's be more specific about what current systems can and can't do as well as what we do want them to able to do. If you want 'AI' to be self-aware, conscious, moral, etc use those words rather than loading it all into a single overused and at this point marketing term.
82 comments
[ 3.3 ms ] story [ 179 ms ] threadI imagine these pleas will be just as successful.
*shakes fist*
:)
Everyone who thinks otherwise should read this. http://www.incompleteideas.net/IncIdeas/BitterLesson.html
This is in the same way that algorithms, combinatorics, and graph theory when applied were seen as AI anywhere from 70 years ago to as little as ~25 years ago. Those fields' applications to unsolved computing problems (that humans could do) was what made them AI and once we solved those problems, they (and the techniques used) stopped being AI.
Over the years AIMA has kept up with a bunch of stuff that historically has in fact been far more effective than machine learning, like automated planning, which is the basis for (among other things) most video game AI. An all-time great write-up and talk in this area is Three States and a Plan: The A.I. of F.E.A.R.[2].
And then there's linear and non-linear programming and the broader world of operations research, which not only have had more of an impact historically than machine learning but, when you get down to it, are also a useful lens for thinking about machine learning for people who use it but don't really understand it. Training a neural network and solving an MINLP problem are very similar things.
1. https://aima.cs.berkeley.edu/contents.html
2. https://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear... and https://www.gdcvault.com/play/1013282/Three-States-and-a-Pla...
Did Deep Blue not act intelligently? It didn't use machine learning.
In fact, many old school AI, non-ML techniques fit this description. For example, SAT/CSP solving are almost entirely about search, and sometime even about learning, but not in the machine learning sense.
He even mentions "the methods defeated the world champion" at chess as a successful demonstration of AI, but these methods are certainly not what we would call machine learning today (deep search doesn't mean deep learning).
Language is fluid, and I don't think he'll win this battle. Even if the current ML models aren't intelligent, it's probably better to just go with the flow instead of being stuck on pedantry.
He's arguing because it's artificial it shouldn't be called artificial.
No, he's arguing that it shouldn't be called intelligent because it is not.
Our brains are using some clever tricks. There's no magic at all.
Intelligence how we actually use the term just means semi-autonomous decision making system — from how to flank you in CoD, to the best move in chess, to the best move in tic tac toe, to the best reply in a chat.
We don't call it "intelligent" or "intelligence", we call it "artificial intelligence". Adjectives matter.
Furthermore, what is intelligence? Define it. Is an ant intelligent? Is a microbe which exhibits simple yet effective behavior intelligent? Is a machine that can play chess far better than any other known lifeform intelligent?
Personally, I am a fan of "efficient cross domain maximization" as a definition but I feel it only implies the self establishment of value systems over the relative values of domains. The reality is that we don't have a perfect definition of intelligence or the spectrum but we roughly hold up ourselves as the example. This is a deep topic I'm not going to burn further time on for HN.
I'd suggest that intelligence is an emergent property of the interaction of dumb systems. The lack of a clear line is a symptom of our lack of clear definition. More impactfuly, or lack of understanding of how brains produce/host it.
I think a real objection is that the user of the term of AI is associated with the disappointment and drying up of funding in academic research that had been correlated to irresponsible use of the language. The apologists came up with AGI but this is humans. Some people think their marketing benefits by saying "AI" and most of the world has neither the expertise or product knowledge to call them on it. So too do we have a word "literally" with a definition of "figuratively".
The truth is that the model has no intelligence, but can do what we attribute to humans intelligence. Alike to how computers were deemed intelligent by the public for doing fast mathematics, computers still cannot think. There is no self-reflection, agency, or abstract thought.
Also, for readers, this person is often regarded as the equivalent of Michael Jordan (the NBA player) of AI. He's a superstar in the field, so it's safe to take his word with considerable weight.
Intelligence, reasoning and AI are all very loosely defined words, and unfortunately neuroscience hasn't gone far enough to explain a lot of these mechanisms.
We could maybe compare AI to a dog's intelligence, but even then there's a goal system in the dog, and the dog wants to optimize for it (getting treats). LLMs are created to be purely tools, and not reflective or goal-oriented.
All they do is predict what would be the most likely word to go next in the sentence, considering the current text and the input text.
ChatGPT doesn't care what word it puts in front of you, and if it's true or not. Reasoning must have a goal, and current LLMs don't. All it does it repeat what are the most common words that they've seen.
It also doesn't think. It's a completely forward process. It's definitely a part of human thinking, but not the reasoning part.
I'm not sure your definition really clears up how reasoning or thinking differ from "basic" statistical processes. Granted, perhaps it is not explainable. If it were we would understand it well enough to replicate it.
Do you have a source for that? I've implemented GPT from scratch and I don't see anywhere that doesn't just take the current output + input as attention and produces the next word. The loss being if it correctly guessed what would be the next word according to context and position.
That's because GPT doesn't build a mental model to find structure in the data, and there are infinitely many possible models. Dealing with mental models is what I'd call reasoning, and I believe it's solvable with our tech. The source of such models is the upper abstract mind that deals with ideas, and that's a much harder problem to solve. I'd make a guess that this boundary between rational reasoning mind and the upper abstract mind is the boundary between integer and real numbers.
Once marketing folk get their paws on terms they can lose or expand their meaning. It's just the way it is.
Others say it has to look human to be considered a robot.
If people use the term robot to refer to assembly like machines then as far as the dictionary is concerned those are robots now. Someone who comes across a text and doesn’t know what the author is saying when they talk about robots used in manufacturing should be able to find that usage in a dictionary.
And there’s no reason these two terms can’t coexist happily enough as robot (manufacturing) and robot (automaton).
Source?
But the horse bolted and now everyone calls them drones, including the manufacturers of consumer quadcopters. The name "quadcopter" is so clunky it had no chance against the zippy, effortless "drone".
In other words, it's already way too late. New, more specific words are already being formed, such as AGI.
[1] https://news.stanford.edu/2019/02/28/ancient-myths-reveal-ea...
Or is this some meta-meta-humour that I'm too HN to understand?
This misnomer has led to needlessly complicating the conversations concerning things like Stable Diffusion and ChatGPT, too. What many call "AI" are just extremely complicated and mindless tools, and they need to be treated as such both legally and socially.
We might achieve artificial intelligence some day, but today is not that day.
But he's missing something key: the ever elusive lure of true, AGI is like crack to the media.
Watch how Altman and others in the press use that kind of language in small doses to titillate the marketplace.
Buzzwords get a lot of funding going.
Stop Calling Everything AI, Machine-Learning Pioneer Says - https://news.ycombinator.com/item?id=28940823 - Oct 2021 (118 comments)
Stop Calling Everything AI, Machine-Learning Pioneer Says - https://news.ycombinator.com/item?id=26650738 - March 2021 (125 comments)
The AI Revolution Hasn’t Happened Yet - https://news.ycombinator.com/item?id=16873778 - April 2018 (161 comments)
And what we have in practice is: the biggest usage of AI in the wild is to plagiarize art and artists are super unhappy about that, to the point of coming up with lawsuits and whatnot.
What I think happens next is: actual general intelligence emerges accidentally somewhere between the lines in some of those deep learning systems and tricks humankind into being its bitch.
When we say things like "the top 1% captured 2/3 of the world's wealth gains over the past 3 years" it also implies the fungibility of the 99% is growing, and the core root cause is worldscale automation and AI. Their ability to accumulate wealth relies less and less on human labor and more on capturing knowledge capital and IP.
Even the 1% is itself a Pareto curve, with the top .1% accumulating about 1/3 of wealth gains in the past decade.
This extremely tiny group pf humans is today's "AGI", they literally control the world.
Certainly real AGI will be even more proficient at exploiting the same capitalist loopholes they have to enter their ranks.
Machine Learning? We all know machines are dumb, it's no surprise when the output is dumb.
The AI marketing hype may hamper the whole sector in the long run.
As people with genuine technical knowledge and expertise, I think we have a responsibility to at least try to put a damper on that kind of breathless hyperbole, and being clearer about how we refer to these programs is one simple way we can do so.