It's nice to hear from someone who knows what they're talking about that GPT-3 is just a fancy and expensive autocomplete. The hype in some circles about it went as far as comparing it to AGI at some point which is just ridiculous.
What evidence do I have that I'm more than a fancy autocomplete, myself? The use of squishy protestations, in lieu of objective metrics, make LeCun's argument rather unconvincing.
High level reasoning and planning about what you are writing about. GPT-3 generates text one word at a time. The results are impressive for what it is. But it will not plan out what it wants to say ahead of time and construct the message to achieve that objective.
I was about to write a reply claiming that you're different from autocomplete because you take input from more sources than just the words you've said before (e.g. your vision), but actually I can't see how that's much different from a language model. The approach seems the same, and all that's really different is the shape of the input data.
But this uncovers difficult questions about free will. If we're all just autocompleting based on a combination of the world around us, our internal state, and the physical laws, then what even is intelligence anyway? This view reduces thought to nothing more than an interesting dust storm.
Still, I find the original argument compelling, if not logically convincing. There does seem to be something missing from GPT-3 that differs fundamentally from human intelligence or AGI. But maybe that's an illusion.
Edit: I don't think you should have been downvoted, since your question is valid and constructive in my view.
That's pretty much it. I do belive it's possible to actually develop "will", but almost nobody thinks that they need to work on such things. They confuse being a programmed robot with being a programmer.
Interesting. As a reading researcher, I imagine that this could potentially introduce subtle and difficult to spot bugs when you get a proposed completion that looks about right i.e. close enough to what you imagined. Has this been an issue in your experience?
Not the person to whom you asked this question, but I'm also a user of TabNine. In my experience, the "recommendations" / autocompletions provided by the tool are usually very short (probably less than 20 characters on average), and I don't use it for terribly complex chunks of code. Where I like it most is in the initialization of common code chunks like `if` and `for` loops, using variables instantiated in nearby preceding lines. It figures out things like `for(customer in customers)` as I'm writing `for`.
No, it's not auto-completing function blocks, just simple expressions that are easy to validate. E.g.,
let lo = 0;
let hi = vec.len();
let mid = lo + (hi
will autocomplete to `(hi - lo) / 2` as the second autocomplete option (so I'd hit tab twice). If you were to "score" it based on top-5 it'd probably be pretty bad at guessing my intent, but then again, I get to _opt-in_ to suggestions so it just needs to be right often enough, as it doesn't bother me much to keep typing.
You're correct. It's only autocomplete on steroids. But I think it's remarkable that something with the very simple goal of autocomplete can, for a few sentences, sound almost alive
You are just a fancy and efficient autocomplete too. When you speak or write, some words have a higher probability than others. You pick alternatives, but they are limited. Of course there are more layers in the human mind, but GPT-3 is a really impressive milestone towards AGI.
It's so easy to downplay every advanced tech, it's actually fun.
I'm not reducing GPT-3 to the extent that you're suggesting. I'm pointing out (and so does LeCun in his post) that it's a language model designed to continue a sequence of words. It has no understanding of the world and is no particularly suited for knowledge extraction or conversation.
> GPT-3 is a really impressive milestone towards AGI
We really don't know this. It's a big step for the field of language models, that's for sure. But we're so far from AGI that nobody knows which direction it's in and whether it exists at all.
> it's a language model designed to continue a sequence of words.
If a language model were able to do this task perfectly, it would be indistinguishable from intelligence, because continuing a sequence of words requires reasoning. You cannot conclude that has no understanding based solely on what it is trained to do when the task it is trained on would be sufficient to demonstrate understanding were it to fully succeed. There are lots of reasons to be skeptical of its potential, but this isn't one of them.
> you cannot conclude that [a model] has no understanding based solely on what it is trained to do
agreed.
but that's not everything we're basing our conclusions on – we also know that GPT-3 was trained purely on text, and i (and presumably GP) don't think that's a path towards "understanding".
in other words, i think being a language model [trained only using a text corpus] is a valid reason to be skeptical of its potential :)
There's this really weird idea that you need high-bandwidth highly-multimodal sensory data to ‘truly’ understand things, but this makes no sense to me.
1. It's all just data in the end. The signals from your eye are not any more real because they are in response to photons.
2. Deafblind people exist, and can even learn to speak vocally. They might have misconceptions about the visual world, but it's certainly not true that they lack general intelligence.
3. As an objection to GPT-3 as a pathway to AGI, the only thing stopping us training these models on high-bandwidth highly-multimodal data is scale, anyway, so the objection doesn't work.
4. GPT-3 is clearly capable of reasoning in ways that can only be explained with more sophisticated world models than at smaller scales. People asserted reasoning was at a limit with GPT-2, and it just clearly isn't.
What does this mean? The world with countries, borders, celebrities, hypes, newsletters and ideologies?
Because that world is as virtual and not grounded in reality as the world of data points fed into this model. Culture is made up. Language is made up. It may express itself in reality but so do the algorithms of social network sites.
If something can only live in a reality of data points on the internet this thing would live mostly in the same reality as we do.
> You are just a fancy and efficient autocomplete too.
We don't know enough about human cognition to say this.
Scott Aaronson has something interesting to say about this in a conversation with Lex Fridman, actually:
https://youtu.be/G_-BBniFFCM?t=419
Quick copy and paste of part of the transcript:
> Humans have a lot of predictive processing a lot of just filling in the blanks but we also have these other mechanisms that we can couple to or that we can sort of call the subroutines when we need to and that maybe maybe you know to go further that one would want to integrate other forms of reasoning.
>You are just a fancy and efficient autocomplete too. When you speak or write, some words have a higher probability than others
No you're not, and that is very easy to disprove. Look at the sentence "John took the water bottle out of the backpack so that it would be lighter". What does it refer to in the sentence, the bottle or the backpack?
Did it statistically come to you or did you need to consult Google? No, you know the right answer, it's the backpack. Why? Because you have a physical understanding of the world. The bottle doesn't get lighter, when you take it out of the backpack, the backpack does, because the bottle is not in there any more. This is not statistics, it's not manipulating strings, it's having a fundamental physical model of the world in your head, and an idea about how entities operate in it.
When you talk you don't do random statistical inference, you match language to the semantics you want to express, which is not statistical.
It may be, but there's a lot in that fancy. If it were 'just' an autocomplete we'd all be using markov chains for our dumb chatbots like we were in the 2000s
There’s a lot of baggage being thrown into the word fancy here. Any (and I mean any) distribution can be factored as a sequence of its random variables, with the next one being conditional on everything that’s come before, aka autocomplete.
That said, I agree more closely with LeCun than the hypers here.
I didn't need to log in to read the article at the original URL, though I had to close a cookie-wall and another modal prompting me to enjoy Facebook better by signing in.
And I couldn't read the article without logging in to my Facebook account.
Facebook seems to put different restrictions depending on where you live. I'm based in Western Europe and never been able to read anything from Facebook without logging in. Same for Instagram.
I think Facebook makes a guess about whether you have a Facebook account or not (or, whether you are likely to log into it) and throws up the wall accordingly. If you'll just bail, they'll show it to you anyways. If they think they can force you to log in, they will.
I'm sure his group has done some rigorous research that I can't even understand.
But in my experience, the few-shot learner attribute of GPT-3 makes it insanely useful. We have already found several use cases for it, one of which replaces 2 ML engineers.
Yes, it's not perfect, but it's pretty good at many things, and REALLY easy to use.
Can you go into more details where it's useful? As your comment here goes directly against what's argued in the linked Facebook post.
Also, if you've found a use case where GPT-3 replaces real humans, what did those humans actually spend their time on? Seems like either you're over-hyping GPT-3, or under-hyping humanity
The humans spent their time building a hideously difficult classification model. Out of the box GPT-3 worked better than the result of a year of their work.
Right, but GPT-3 can be used generally. That's the difference. It scales because you don't need to build an entirely new model for each different use case.
You just change the prelude and use it for something new.
It sounds like a big deal. What a tempting idea. And a colleague was mildly annoyed with me for how unimpressed I seemed.
But you have to understand, the use cases you mention are shallow and limited. The heart of GPT, the fine-tuning, is gone. And it looks like even OpenAI gave up on letting users fine-tune, because it means they essentially do build an entirely new, expensive model for each use case.
I wanted to make an HN Simulator, the way that https://www.reddit.com/r/SubSimulatorGPT2/ works. But that's far beyond the capabilities of metalearning (the idea that you describe).
I think the onus is on you to prove that the use cases are shallow and limited. I've seen GPT-3 already being used for diverse and interesting ideas that would not have occurred to me personally.
However, even if they are, the point stands: currently, there are teams of people at companies all over the world tuning models for these shallow and limited use-cases. GPT-3 can replace them all, without OpenAI needing to invest another cent in training for a particular customer's use-case. That is in fact game-changing for the ML/DL world and current applications thereof.
Is it AGI? Obviously not. But the vast majority of ML applications don't need to be.
For a more extensive rebuttal, I wrote one here. https://news.ycombinator.com/item?id=23346972 Though that was more a rebut of GPT in general as a path to AGI than metalearning in particular for generating memes.
>However, even if they are, the point stands: currently, there are teams of people at companies all over the world tuning models for these shallow and limited use-cases. GPT-3 can replace them all, without OpenAI needing to invest another cent in training for a particular customer's use-case. That is in fact game-changing for the ML/DL world and current applications thereof.
The counterpoint is that it would be significantly cheaper AND have better performance to fine-tune models to each customer's use case than it is to just run GPT-3 at inference.
Yes except they were saying the iPhone replaced Nokia's engineers.
GPT-3 is not doing what the ML engineers were doing (building models), GPT-3 is the end goal. The company just decided to outsource the work to OpenAI and pay a monthly fee to them instead of salaries to their ML engineers.
"We have already found several use cases for it, one of which replaces 2 ML engineers." -> Clearly makes it sounds like GPT-3 can do the job their ML engineers were doing.
From a business perspective, this is an irrelevant distinction. The requirement was satisfied in a different way, i.e. the engineers satisfying the requirement were replaced by GPT-3, the tool which satisfies the requirement.
That's interesting, GPT-3 can do classification too? Or did I misunderstood and you meant your engineers used classification to build a language model that didn't perform as well as GPT-3 (which is less surprising indeed) ?
GPT-3 can do classification. For example you can give it a prompt like "Hacker News is a website. Excel is a Windows program. Visual Studio is a Windows program. Safari is a Mac program. CPU-Z is", and even GPT2 will complete this with "a Windows program" (with GPT2 you need to try multiple times, discard useless results and average what's most common, but it works and is straight-forward to automate).
I would be interested in hearing more about this, within the bounds of what you can share publicly. Most of the touted GPT-3 use cases I've seen to date have dried up or are still in limbo, so hearing about a real production use would be exciting!
There are a couple of ways to do it. You can give it a prompt that shows examples of the classification and it mimics what it thinks is the correct behavior when you feed it new unclassified input. They also have a search endpoint that lets you do classification by giving it an input along with labels as the searchable documents and using the resulting semantic relevance scores.
In general GPT3 is not SotA on (any?) classification task, did you just not have enough data to fine tune a discriminative transformer model? Inference should be cheaper with a smaller transformer/also less lock-in.
I can't go into too much detail here about why we couldn't do that, but one aspect that we found VERY useful is that GPT-3 could draw on real world knowledge not present in the dataset to enhance the results.
Better IDEs have saved countless hours. Saving hours is equivalent to replacing jobs, unless demand is elastic enough to fill that time. Most of the time we are lucky enough that demand at a given price point is much larger than supply, but this won't last forever.
that's not how it works... as it is not a zero sum (i.e. the work is not bounded/fixed but it increases).
IDEs and higher level tools help engineers become more productive. They can do more, with less. This raises the bar on products, and the demand of customers for them (things are pretier, easier to use, etc..), which in turn creates more domains for software to be used, and more demand for engineers.
And when OpenAI says that your two entirely valid use cases are a safety concern, and denies you api access, what will you do? Better keep those ML engineers handy.
If you think this isn’t a concern, I’ve already seen it happen with my own eyes, rather than hearing about it second hand. They encouraged someone to make a writing tool. That someone then spent roughly six weeks prototyping, iterating, and giving constant feedback. All signals from OpenAI were “Yes, awesome!”
Then one day they simply declined to let them ship. Anything. Anything even resembling “a tool to generate huge quantities of outputs.” Which was, you know, the whole point.
You play, you pay. And I hope you’re ready to pay, because you won’t have your magical genie unless the magical genie’s caretakers believe you are sufficiently worthy.
They were forced to give Microsoft exclusive access, because it was one of the terms of Microsoft's billion-dollar cloud credit investment.
But you can't pay employees with cloud credits, so time will tell whether it was a correct decision. (It probably was. And I exaggerate slightly; the investment included a substantial sum of real dollars too. But most people see that billion dollar investment and think it's all dollars, when in fact it was largely credits.)
For anyone else who doesn’t want to deal with Facebook, here’s the post:
Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare.
GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time. GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
As a question-answering system, GPT-3 is not very good. Other approaches that are explicitly built to represent massive amount of knowledge in "neural" associative memories are better at it.
As a dialog system, it's not very good either. Again, other approaches that are explicitly trained to perform to interact with people are better at it.
It's entertaining, and perhaps mildly useful as a creative help. But trying to build intelligent machines by scaling up language models is like a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.
It's quite possible that some of the current approaches could be the basis of a good QA system for medical applicatioms. The system could be trained on the entire medical literature and answer questions from physicians.
But compiling massive amounts of operational knowledge from text is still very much a research topic.
High altitude planes going to the moon is a beautiful analogy. I think this is what I’ll use to explain to less technical friends why I think we’re still many years from self driving cars.
I have been using that analogy to explain why Tesla is many years away from a self driving car. Several others are building something that is fundamentally different.
Several things. Mostly that it has to be rolled out slowly because people lives are at stake so testing can run for quite a bit longer than you’d expect. Also that everyone wants to be the one who makes the breakthrough so companies will claim its right around the corner (like fusion) repetitively, i.e. Tesla saying it’d be here in 2018. We’re just at a point where these things can use parking lots so I wouldn’t expect a complete rollout several years as systems are built on top of other systems that have been widely tested and confirmed to work.
The question isn't whether high-altitude planes can go to the moon, it's whether human intelligence is closer to the clouds or to the moon. For all the talk about how language models "just" learn correlations, there's a remarkable dearth of evidence that humans do something qualitatively different.
Exactly - the analogy fails if a few assumptions we have about ourselves or what GPT-3 is actually “doing” are wrong. Until we hit some asymptotic limit on training these kinds of language models, I’m withholding judgement on what such a model will be capable of representing if/when that limit arrives.
Are you just dumbing down humans to match the model? LeCunn's post is very sparse on detail, but the point is that humans can easily reason about a vast number of things that any form of sequential language model cannot. That alone is evidence that humans are doing something qualitatively different.
It isn't conclusive evidence however, and larger models may produce significantly more human like results. But from what we know about how gpt-3 works, all the evidence is on the side of it not resembling human intelligence.
> there's a remarkable dearth of evidence that humans do something qualitatively different.
Perhaps now, but if history is any indication, when we (as humans) think we have a good grip on how something really works (like human intellect in this example), we've been wrong.
We model the world around us from observation and testing, find our errors, remodel, and improve over time.
Then at some point we find some piece of information that shows us our model was a decent approximation, but fundamentally wrong, and that we need to start from scratch.
If we find that we want to go beyond the moon (and we eventually will), or that the moon is further than we think, we'll again need a different approach.
I always feel like there's a certain beauty and cosmic humor to it.
> For all the talk about how language models "just" learn correlations, there's a remarkable dearth of evidence that humans do something qualitatively different.
GPT3 doesn't know the difference between a given set of characters and the idea/object the characters represent. It can associate "river" and "stream" and "water" but has no understanding beyond that they appear in patterns together. It couldn't possibly make the connection that river and streams are bodies of water, because there is no association with reality.
GPT3 wouldn't even know the difference between human language and characters derived from some random data source.
The only thing it does is identify deeply complex patterns, as long as there are humans around to notify it when it's doing a good job. It's going to be very useful for auto-complete, and jumping in to help users finish repetitive tasks, along with the other stuff ML is good at, but it's simply a GIGO pattern recognition system.
So I think you have it exactly backwards -- there is a dearth of evidence that AGI is even remotely possible. We have known the full anatomy of the C. Elegans ringworm since 1984 -- it's 1mm long and has 300 neurons. There is a foundation dedicated to replicating it's behavior[1], and all they have achieved is complex animation.
> It couldn't possibly make the connection that river and streams are bodies of water, because there is no association with reality.
Yet if you asked it if rivers and streams are bodies of water it would probably say that they are.
Likewise if I asked you if black holes and neutron stars are both celestial bodies you would say yes... but you've presumably never seen them, only read about them.
Now I think you could argue that you could ultimately tie your knowledge back to some reality, like star ties to the sun and how you've seen the sun, but I haven't been so convinced that we know enough about how the mind works to be sure that the distinction between form and meaning is real.
Assuming there is no "soul" which makes meat life special, I don't see any fundamental problems with building simulated intelligence.
I agree that GPT-3 should not be used for medical applications, but I disagree that it's "not very good" as a dialog system. I've found it to be insanely good though it may require effective prompt engineering to work well.
To remind people: Yann LeCun is an engineering superstar who was working on neural networks at Bell labs long before they were cool.
Those handwritten digits that are a scourge today (e.g. 'would any of these methods work on a different set of symbols?' is unasked) came from a competition to develop a commercial zip code reader for the U.S. Postal Service post back in the day of the Apple Newton. He won it!
I was a bagman for text classification data in the early 2000's and his reviews of the results you got using methods of the time (Naive Bayes, Rocchio, Perceptron, SVM) showed a depth of thought and attention to detail which helped me pick and choose tools to make classifiers with fairly predictable performance and development paths.
GPT-3 on the other hand does a good job of spouting nonsense like Peter Thiel and that has something to do with it's emotional appeal. People make fun of it and laugh at the mistakes it makes like those videos where somebody kicks down one of those Boston Robotics dogs: it's just good enough to be an object for those sort of feelings.
To remind people: Yann LeCun worked on artificial neural networks (ANN) during the period where they were actively shunned by most of the scientific community. You could barely publish a paper on ANN.
Just to demonstrate, one the most common books during period, "Artificial Intelligence: A Modern Approach, 2nd ed" by Norvig, 1080 pages, has less than one (1!) page dedicated to ANNs. I personally think Norvig is an idiot with regards to Artificial Intelligence, and his book (used in 1500 schools in 135 countries and regions) singlehandedly slowed down the progress of AI by a few years, until a new generation of students outgrew this archaic book.
For some context. The 2nd edition was published in 2002 (so maybe written in 2001-2002?). The fourth edition published in 2020 seems to have a bunch more things on NNs.
It is not fair to call Norvig an idiot. When I was studying AI in grad school, around 2002-2003, just about everyone thought that artificial neural networks were a dead end, compared to approaches like support vector machines. Sometimes the scientific consensus is wrong, and it takes a few heroic figures plugging away to prove it. That doesn't mean that everyone in the mainstream is an "idiot".
> Artificial Intelligence: A Modern Approach, 2nd ed
Published in 2002. At that point, ANN research had reached a pretty hard plateau with very few tangible results. Faulting Russel and Norvig for not going into depth about ANNs is kind of like faulting Richard Feynman for not going into depth about quantum computers in the Feynman Lectures.
Also, a lot of the subsequent work and breakthroughs on ANNs has been done at Google under Norvig's leadership as Director of Research.
As opposed to the AI techniques taught in the AIMA book (KR and logic reasoning), which had plateau'ed in the 70s...?
Norvig had to be pretty clueless to decide ANNs are such a dead-end, that they don't deserve even a chapter in his book, where all around him there are biological living proofs that neural networks are probably a pretty good bet for AI...
(Note: I held the same opinion in the mid 90s when I reviewed his 1st edition and I'm definitely no Feynmann-level. It's just common sense.)
> all around him there are biological living proofs that neural networks are probably a pretty good bet for AI...
100 years ago you would have been arguing that all around you are living proofs that ornithopters are probably a pretty good bet for artificial flight. You would have been wrong about that too.
We seem to be re-enacting the Symbolic vs Connectionist AI debate of the 80s, poorly.
All I'm saying is, Norvig should have been more humble and included a chapter or two about ANNs, with all the research accumulated thus far, instead of betting 100% for the symbolic approach. Let the next generation of students learn both approaches and decide for themselves. It's sad that a whole generation of students was taught AI from this archaic book.
> All I'm saying is, Norvig should have been more humble and included a chapter or two about ANNs
No, that is not all you're saying. You opened with this:
"I personally think Norvig is an idiot with regards to Artificial Intelligence,"
Not only did you lob an ad hominem at one of the most respected members of the community simply for making an editorial decision 18 years ago that you happen not to agree with today, you did it from a newly created anonymous HN account, and then you tried to deny it. Your conduct here has been thoroughly dishonorable. You should be ashamed of yourself.
I think Norvig is a smart person, I enjoy his books, papers and jupyter notebooks, but I always thought he was pretty clueless regarding AI, as history indeed demonstrated. That's not an ad hominem.
What is shameful was the extreme shunning of the mainstream scientific community to ANNs, in 90's-00's decades, to the point that it was considered career suicide to publish a ANN paper. I believe Yann LeCun has said similar things in the past[0], reminiscing the time when it took him several years(!) to get an ANN paper accepted for publication.
applied rigorous math (e.g. when computer science was new) to prove that a certain kind of single-layer neural network couldn't solve certain problems. (Can't learn XOR) It is like proving that it takes N log N comparisons to sort N items.
This dampened interest in neural networks for a long time but the "geometrical thinking in hyperdimensional space" is what the field is all about today.
Interest in neural networks was renewed with Werbos's (1975) backpropagation algorithm. There was continued progress in ANNs all this time.
I think the aversion to ANNs during the 90s was more philosophical and aesthetic - ANNs math is indeed "ugly" compared to symbolic logic, bayesian inference, SVM (in the 00's), and many other traditional AI methods.
> GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
Without excessive effort, humans don't have any knowledge of how the world actually works. They only appear to have some level of background knowledge, to the extent that this knowledge is present in their faint memories. But this knowledge is very shallow and disconnected from the underlying reality.
For example, all humans have the notion of object permanence, developed by about six months of life. Object permanence is the notion that things don't go away just because you can't see them.
ML systems need to be specifically trained to have object permanence, and GPT-3 almost certainly does not possess it.
Like, I get that it's hip to booster ML and GPT-3, and all of the stuff humans can do seem trivial, but it's really not the case and is something that is holding progress in AI back massively.
Object permanence isn't even defined in the stateless world of the GPT-3 API.
My comment was merely a jab at Yann's poor argument. I don't find humans to be trivial at all, but neither do I believe that they are infinitely complex.
The linked Nabla article is fair, albeit I would appreciate more technical details. It seems to be using the API in zero-shot fashion, which is not what one would do to get the most out of it.
I understand that you are being sarcastic, but I'll pretend that you're not ;)
There was a video showing the pattern recognition layer of a car 'forget' about objects it had previously correctly identified. There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered. To draw an analogy with human cognition, humans also aren't consciously aware of all the objects in their field of view all the time, but that doesn't mean they forget their existence.
I'm honestly not. I've seen that video, and given my experience of vision models, am actually concerned that object permanance is not being focused on.
> There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered.
I would be very surprised if this were true.
> To draw an analogy with human cognition, humans also aren't consciously aware of all the objects in their field of view all the time, but that doesn't mean they forget their existence.
This is exactly object permanence, in that babies start looking in the right direction for objects that disappear. I assume that Waymo at least have some kind of model for this, but am not convinced that it's particularly effective.
Personally, I think that self-driving will require a limited theory of mind model, and we are nowhere near that.
At the very least there must be some kind of a trajectory model which implies that multiple observations of the same object are analyzed as a sequence. How would it work without object permanence? Would it just erase objects undetected in the current frame and pretend they never existed? I suspect you're too focused on the vision aspect of the vehicle control system.
As both a GPT-3 sceptic and LeCun-sceptic (nothing personal, I'm just sceptical of a lot of the modern hype and seeming cliquey-ness of the AGI "progress" centres), this summary is very useful, but sounds a bit sour-grapesy from LeCun...
I think the difference between a large language model and a human intelligence is that the human may perform some extra computation to make additional connections on his own.
But other than that, aren't we all just large language models?
Not even remotely close. The difference is so big that it's almost harmful to the discussion to compare the way humans think (which we still don't have great understanding of) and the way language models work.
I completely agree but I do think humans have a language model, and considering how we use that to encode and decode the human experience might be useful in figuring out how we improve things like GPT-3.
Personally I feel that embodiment of some form, in which there is some vector space for a 'world model' that can be paired up to a language model, is a route forward. For example, if you have a Boston Dynamics (for example) robot that has a model for gravity, mass, acceleration, force, object manipulation, etc and you incorporate those into a language model, there is going to be a much richer latent space from which associations can be made between terms. If you ask GPT-3 the difference between various gaits, e.g. walk, trot, gallop, it's going to have associations with other contexts and adjectives used in the vicinity of those terms. However, if you enrich it with data from a Spot Mini that can actually execute those gaits, you're going to have information around velocity, inertia, power consumption and budget, object detection rates, route planning horizon, etc.
>.... A horse trainer once said to me, "Animals don't think, they just make associations." I responded to that by saying, "If making associations is not thinking, then I would have to conclude that I do not think." People with autism and animals both think by making visual associations. These associations are like snapshots of events and tend to be very specific. For example, a horse might fear bearded men when it sees one in the barn, but bearded men might be tolerated in the riding arena. In this situation the horse may only fear bearded men in the barn because he may have had a bad past experience in the barn with a bearded man.
I agree some unrealistic expectations have been created due to people posting cherry picked output.
That said, I've spent a lot of time with it this month and think it will be an extremely useful tool for creative works of all types. It's not to a point where you can just tell it to write a blog post (yet!) but it can generate novel snippets, ideas, and variations that are actually usable. Unskilled creatives should be worried. Skilled creatives should incorporate it into their workflow.
> GPT-3 doesn't have any knowledge of how the world actually works.
I think this is a philosophical question. There is a view that, basically, there is no such thing as knowledge, just language (or, at least, there is no distinction between knowledge and language). In this view, all there really is is language, which is mostly composed of metaphors and, ultimately, metaphors only refer to other metaphors, i.e. language is circular. In this view, not only is the ultimate, physical, concrete world beyond us but also we can't even talk about it. From this perspective, GPT-3 is not substantively different than what our minds are doing.
That view makes some strong claims (I don't find it convincing), but it's out there. A slightly different claim, though, is that "knowledge of how (we think) the world actually works" is encoded in language. To me, that seems trivially true. So, again, how you take this quote from LeCun depends on what you think knowledge is and your view of the relationship between knowledge and language.
Personally I do not find the whole "language = knowledge" argument convincing. But if you're interested in reading writers who make that argument (and perhaps I'm vulgarizing the argument a bit), Nietzsche makes it in On Truth and Falsity in their Extra-Moral Sense and George Lakoff makes it in Metaphors We Live By.
I'd also suggest Wittgenstein's
Tractatus Logico-Philosophicus, a seminal work of the logical positivist movement. Influenced by Frege's predicate calculus, the aim of the Tractatus was to determine an isomorphic relationship between language, thought, and external states of affairs. An axiomatic attempt to reveal a potentially ideal logical language, that is not interested in meaning per se, but merely an accurate reflection of the world. A closed system that essentially excludes non-falsifiable metaphysical question. Famously concluding with the instruction: "Whereof one cannot speak, thereof one must be silent." Part of Wittgenstein's project, even in its early aggressively logical form, was philosophy as a therapeutic. That is, the metaphysical questions concerning god, being, essence, and forms that had inspired thousands of years worth of fevered conversation, could be finally be quieted. That's not to say they couldn't be meditated on, but were not in the domain of his logical language, and so silence. Again, I think early Wittgenstein sometimes gets misinterpreted, "...therefore one cannot speak" does not, to me, mean that it can't be considered or one must forgo spirituality, just that it couldn't be spoken of within the project of the Tractatus.
Logical empiricism was ultimately a dead end as the criteria for even verifying empirical truth has long been contentious philosophically, and was further critiqued by contemporaries such as Quine who attacked the premise of the analytic/synthetic distinction (think Hume's fork, which Kant tried to solve) and Popper who cited the problem of induction to critique the fundamental premises of the positivists verificationism.
Wittgenstein is an interesting case, as the Tractatus is considered an early work of his, profoundly influential to analytic philosphy at the time, yet his later work, Philosophical Investigations is sometimes seen to retract the dogmatism found in the Tractatus. I tend to take the view that it's a continuation of his thought, rather than a retraction of his earlier work. Crudely, whereas his former thought represented a narrowly axiomatic definition of language and its truth value, PI investigates, among many other ideas, language as an activity, or game, that has meaning dependent on the context of its use, languages as families. Granted, Wittgenstein is a complex thinker and these are simply my interpretations.
It's also curious to note that as positivism was beginning to fall out of favor around the time of the second world war, a continental thinker such as Heidegger, whose thought luxuriated in the kind of metaphysical questions the positivists necessarily eschewed, rose to prominence and was infamously sanctioned by the NSDAP to philosophize about their presumed "destiny". Bit of a tangent, but I think the historical context is relevant, as often philosophical movements are birthed from pre- and post-war attitudes.
Are animals as intelligent as human? What language does to us is to be able to pass on knowledge to generations, knowledge can be accumulated. Animals may be able to pass on simple concept to next generation, they won't be able to accumulate knowledge without language and writing.
As a response to leftyted, erispoe's point neither depends on nor implies that animals are as intelligent as humans, it is simply bringing up a counter-example: what appears to be knowledge in animals lacking language.
Of course, you could always attempt to define knowledge such that it is purely verbal, or alternatively define whatever is going on in the brain of an animal to be language, but is either approach useful? In common usage, we recognise, as knowledge, various things that cannot be communicated by language, such as knowing how to ride a unicycle on a tightrope (I doubt you can learn it just from a book) and the infamous qualia which supposedly prove that the mind is dualistic. And what about the knowledge of how to use language? How does that get bootstrapped?
The knowledge of riding a unicycle on a tightrope does not give you the ability to ride a unicycle on a tightrope. Yes, you have to learn it through experiencing it, because you need to map it to your motors. Animal has instincts, they are able to trace water, for example, without teaching. It is also knowledge, but for our discussion, knowledge is what we obtained after birth, not something encoded in our DNA. This knowledge is stored in the format of language. Apparently some believes that that language is part of our DNA. That's why you can have Tazan, but not an animal can speak human language even if a human raise it since it was born because language cannot be learnt without support in code in DNA.
If we define knowledge that way, leftyted's original claim (knowledge is language) becomes true by definition.
This can lead to some confusion. In Frank Jackson's Knowledge Argument[1], dualists say that qualia are knowledge, knowledge is language, and so if Mary could not learn qualia by studying science, then materialism is false. This argument trades on being inconsistent over what it means to have knowledge of something.
In important ways yes, and in important ways no. Yes: can deal with dynamic spatial environments, can act towards a goal (intentionality), some almost certainly can plan how to reache these goals. No: Use of symbols and recursive language.
I think an important distinction to make is your use of the word "language", and how we think of language as it concerns human minds, and as it concerns GPT-3.
In our heads, language is a combination of words and concepts, and knowledge can be encoded by making connections between concepts, not simply words. If there is no concept or idea backing up the words, it can hardly be called knowledge. Consider the case of the man who did not speak French, yet memorised a French dictionary, and subsequently went on to win a Scrabble competition. Just because he knows the words, would you say he knows the language?
A language model such as GPT-3 operates only on words, not concepts. It can make connections between words on the basis of statistical correlations, but has no capacity for encoding concepts, and therefore cannot "know" anything.
> In our heads, language is a combination of words and concepts, and knowledge can be encoded by making connections between concepts, not simply words. If there is no concept or idea backing up the words, it can hardly be called knowledge.
Great point.
> A language model such as GPT-3 operates only on words, not concepts. It can make connections between words on the basis of statistical correlations, but has no capacity for encoding concepts, and therefore cannot "know" anything.
Are you sure? Aren't "concepts" encoded in how language is used, at least to some degree?
LeCun does say that models that explicitly attempt represent knowledge perform better than GPT-3 in terms of answering questions. I'm no expert but I believe him.
>Aren’t “concepts” encoded in how language is used, at least to some degree?
Good point and I think this shows up to the extent different languages might affect how we express particular concepts.
However I think it is more accurate to say that language solidifies and gives form to how we express concepts and the “concepts” themselves are independent of languages. Only our “expression” of these “concepts” depends on language.
For anyone interested in art and art history, this distinction was the central focus of the French surrealist painter Rene Magritte.
Language is how we store our knowledge, and language is a system of words. If a language model contains all the possible sentences you can say, it will complete any of your sentences, don't you think it knows what you know? The input is sequence of characters, so you can say it may or may not operate on words. It can operate on subwords, words or phrases where it see fit.
I like to think intelligence as clouds. If you dig deep down, they are just droplets, there are so many of them, they can appear to be so many different shapes. And they look complete different. Maybe intelligence is the same.
At the risk of reigniting the perpetual war about how to characterize machine intelligence, and by extension how to characterize the risk they pose, Yann has been (and still is AFAIK) more in the "existential AI risk is a long-term problem" group. In a 2016 interview LeCun said [1]:
> We’re very far from having machines that can learn the most basic things about the world in the way humans and animals can do. Like, yes, in particular areas machines have superhuman performance, but in terms of general intelligence we’re not even close to a rat. This makes a lot of questions people are asking themselves premature. . That’s not to say we shouldn’t think about them, but there’s no danger in the immediate or even medium term. There are real dangers in the department of AI, real risks, but they’re not Terminator scenarios.
That's pretty measured overall, but he doesn't know that there's no existential AI risk in the medium term. No one does, and that's the problem. Experts simply suspect that it's unlikely. Stuart Russell and him have debated similar topics [2].
To tie back to your point: I keep seeing LeCun brush over tricky questions like yours and the ones at [2] with an arrogant confidence. I wish that he would be more careful, and I hope that I have a skewed view of him.
He's not wrong, we're very far. And looking at past "progress" it seems that we'll get there very slowly. So it seems long-term.
Except people are bad at exponential processes. Yet when economics drives us we are suddenly good at making them happen. And this combo seems to be what makes these existential risks. (Like climate change, or other manifestations of the coordination problem.)
I don't know if i'd go as far as to agree that "there is no knowledge, only language" .. but I 100% agree one of the key insights from GPT-3 -- why training on language is so effective in the first place -- is that language is tightly coupled to reality
I'm not sure that you can assert that language is tightly coupled to reality, unless you're using the term "reality" to mean something akin to "as one perceives the world" (regardless of whether that perception is correct or not).
Most expressions of language that survived from a few thousand years ago are centered around myths, and while those myths may have contained certain moral or ethical lessons (that were and are subject to interpretation) they certainly weren't tightly coupled to reality in an objective sense.
Training on expressions of language (I separate the concept of language itself from its expression in the form of writing, speaking, etc) certainly has use cases but can GPT-3 recognize a previously unknown analogy and correlate it with the proper piece of applicable "knowledge" it has? If not then it really has no understanding.
2. yes, language is bottlenecked by human perception, as are all things
3. even the notion of myth and fiction is encoded in language. language is self-descriptive and self-aware and you can separate sense from non-sense.
4. i'm not talking about knowledge or understanding, but of addressing the question of why training on language let's GPT-3 make human-like predictions as if it knows about reality? either it's a fluke, or it's because language as a whole is a model that approximates reality.
Why does it make human-like predictions as if it knows about reality? Because it's essentially pattern-matching the consensus of the literature it was trained on. Literature as a whole will tend to settle on a consensus sentiment, albeit one that is probably significantly behind the current consensus sentiment (it takes time to accumulate enough mass to move the weights). If your interactions with GPT-3 fall into the rather sizeable consensus that most people either subscribe to or are familiar with then it will certainly prove a decent mimic of understanding. If, however, you attempt to teach it a novel concept or if you dig into its interactions long enough to test for depth of understanding you run into the gaps and GPT-3 either begins to mimic that bullshit artist everyone knows that claims to know things but is only regurgitating platitudes and buzzwords or it begins to mimic behavior that would make you question whether it was sober and/or sane.
There's little doubt that GPT-3 could hold its own quite well in a bout of polite conversation and/or small talk that features in many social situations but that's more a commentary on the limited area of knowledge and behavior that etiquette expects for interactions in such settings.
It could also be trained on the canon of Shakespeare and behave as a prior work of Shakespeare, but if you left one play out of that canon and then attempted to have GPT-3 generate that missing work it wouldn't... at all. It doesn't approximate how Shakespeare thought, it approximates the literature he produced that you used for training.
None of this is to denigrate the achievement that GPT-3 represents, it is merely to point out that it is unfair to GPT-3 to attempt to hold it to the standards of AGI.
It's not that philosophical. If you visualize the GPT-3 embeddings using an embeddings projector then you can see the knowledge with your own eyes. That bypasses the need for GPT-3 to use language to communicate its knowledge to you. Computers aren't a black box like we've traditionally thought of brains.
Humans are very good at visual memory, thanks to millions of years of evolutionary pressure to hunt game and gather for food. Hence why the best memory champions in the world mostly use the concept of "memory palaces." When you think of a car, do you see a car in your head or does a dictionary definition (i.e. purely language knowledge) of a car pop in your mind?
Tell that to all the other animals on Earth. Do they not also have knowledge? Do you really think they encode their knowledge in language?
Do you really think that humans are so special as to encode all their knowledge in language? Watch a movie. Listen to a song. Examine a piece of art. Feel sculpture. Play a guitar. Dance.
There is a segment of the software community that is highly language centric/adept. But that community is often blind to other forms of understanding.
Just look at the language of Shakespeare. Much of the language is visual and experiential. How much would you actually understand without your senses and imagination? Your knowledge encompasses your being.
>Tell that to all the other animals on Earth. Do they not also have knowledge? Do you really think they encode their knowledge in language?
Well, actually, yes, they do. Many animals have elaborate languages encompassing many concepts. Crows can explain to one another what a human looks like, for example.
If GPT-3 has a consistent position on anything, it's only because the corpus it was trained on was consistent about it. So, for example, it will reliably autocomplete Jabberwocky because there are a lot of copies of this poem in the corpus and they are all the same.
If there were two versions of this poem that started the same way, it would pick between the variations in the corpus randomly. In other cases it might choose based on the style of prose or other stuff like that.
GPT-3 can get some trivia right, but it's only because the editors of Wikipedia already came to consensus about it and Wikipedia was weighted more. It doesn't have a way of coming to a consistent conclusion on its own.
Without consistency, how can it be said to know or believe anything? You might as well ask what a library believes. Sure, the authors may have believed things, but it depends which book you happen to pick up.
I agree with you in that I would make a strong distinction between what a model like GPT-3 does and whatever it is that humans do.
But I do think you're missing the point just a bit. When we speak and think, we use all kinds of metaphors that express judgements about the world, usually without realizing it. In other words, the way we use language encodes concepts in a deep way.
To borrow an example from George Lakoff, we, in English, use war-metaphors to talk about arguments. Of arguments and of wars you can say things like "he's marshalling his forces," "they're ceding their territory," or "she's girding her defenses". In fact, almost anything you can say about a war you can also say about an argument. In American politics, with regard to partisan squabbling and the filibuster, we talk about "the nuclear option". The fact that these metaphors make sense to us indicates a judgement, something like "arguments are like wars". That judgement shows up in billions of lines of English scraped from the internet and can be fed into a model, allowing GPT-3 to "make that connection" via purely statistical methods.
Yes, this is a bit like asking "what a library believes". But a lot of these metaphors show up in our languages and, in a way, they express judgements, which is something akin to a belief. Does that mean a library has beliefs? Is this all knowledge is? I wouldn't go that far. But the argument is an interesting one and worth raising.
Well, it's certainly interesting that it can learn metaphors, and this can be useful for creative purposes, so it's fun to play with.
But a sophisticated understanding of metaphors could be used to tell the truth or to lie. In the case of GPT-3, it doesn't know the difference. Telling the truth and lying come out of the same autocompletion process.
If you consider the use of a metaphor to be showing judgement, it means that a particular metaphor seems to be appropriate to use in a particular context.
If it's only philosophical, then me saying that Hacker News Website itself has 'knowledge' of everything we discuss about is also philosophical. Same can be applied to plain paper books.
How about any web application? A for loop? anything which can generate something for you?
> metaphors only refer to other metaphors, i.e. language is circular.
Except we (humans) have real-life experience that gives meaning to those metaphors.
> Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
Just want to point out that he's saying the people on the upper end of the expectation distribution are wrong, not the people in the middle of it. So if you're takeaway from this is that GPT3 is nothing special, that's probably the wrong message.
His next paragraph claims that Nabla "debunks" the idea that "large language models" can be used in healthcare.
That's not just "some people have unrealistic expectations" it's "this tool, when when more advanced and find tuned, will never be appropriate to use in a very broad class of use cases".
He also says "GPT-3 has no knowledge of how the world works", which is clearly an overstatement meant to clear up hype, but is untrue. For example, GPT-3 knows more trivia than I do.
I'm having trouble wrapping my head around LeCun's thinking regarding the Nabla reference. The Nabla link is just a blog post by three people without any technical details provided at all. How can this possibly "debunk" anything?
> which is clearly an overstatement meant to clear up hype, but is untrue
It all depends on your definition of knowledge. Under a certain definition you could say that GPT-3 knows basically nothing.
If someone teaches me to repeat perfectly something very smart in a language I don't know, without explaining to me what that thing is, do I have knowledge about this?
The same argument can be made about those kind of models, the knowledge they have is about the structure of the language and what word is most likely to come next, but they have no way to ground those words in actual relation with the world.
Aka. the Chinese room argument. However, I'm not so sure us people are little more than just pattern matching machines. When I start to talk (or write, as I'm doing now), the words kind of just flow out. I can make the argument, that I understand the "real" world, but do I really?
> I'm not so sure us people are little more than just pattern matching machines
Yes that's what leads to multiple definition of what knowledge really is. Yann LeCun believe we are more than just that, hence why he is saying GPT-3 would have no knowledge.
no it doesn't, GPT-3 is a very sophisticated parrot. it doesn't know any trivia, it knows how to put the most likely string of characters next to the one it just saw, it doesn't matter what the text represents. That's the difference between you and the model.
It's basically the Chinese room. You can make an analog GPT-3 by asking a question, recording your answer, handing someone who doesn't understand a word of your language the giant box of tapes, and she tries to match them together until she appears to make sense to listeners
AlphaGo doesn't know anything about the game of go. It "just" manipulates symbols, runs instructions on its CPUs and GPUs, illuminates pixels on the screen for a human to see.
It just also "happens" to be the case that if you interpret those pixels as go moves and play those moves against the world champion human go player then eventually that human will hold a press conference announcing to the world that AlphaGo has won the match.
Yes that is entirely true, but the world of Go is very small and unambiguous in terms of goals, rules and so on. A little less than chess, but it's still a game with a well structured world and simple rules. AlphaGo understands as little about Go as Stockfish does about Chess or your TI-83 understands about maths. That they're good at it has almost nothing to do with (generalised) intelligence, and it's no coincidence that the performance of RL say, in games is insanely better than it is when applied to real-world, open-ended problems.
Deep Mind made a 'superhuman' Breakout bot, but if you messed with the panel at the bottom it sucked. It hadn't learned what the panel is or what the game is about, it had just learned to rearrange pixels. You can get away with this in one version of Breakout, but in the real world it's brittle and useless
Ok, fine. It doesn't "know" any trivia. It'll still crush me in a trivia contest:
"Q: Who was president of the United States in 1801?
A: Thomas Jefferson was president of the United States in 1801.
Q: What city is in the northwest corner of Ohio?
A: Toledo is in the northwest corner of Ohio.
Q: What highway do you take to get to North Lake Tahoe from Oakland?
A: Highway 80 is the highway you take to get to North Lake Tahoe from Oakland.
Q: What are Darius's abilities in League of Legends?
A: Darius's abilities in League of Legends are: Apprehend, Decimate, Noxian Guillotine, Crippling Strike, Apprehend [sic], and Hemorrhage.
Q: Who did Capablanca defeat to become world chess champion?
A: Capablanca defeated Lasker to become world chess champion."
Of those 5 questions, I could answer #1 after deploying a mnemonic and some math, and #2 if you gave me multiple choice of the top 5 cities in Ohio, and I would miss the rest.
Yes, I think the focus on "getting to the moon,", to use his analogy, ignores the fact that GPT-3 is an SR-71 in a world of 19th century balloons. It may not get to the moon, but it definitely points the way to lots of useful stuff. There is a lot of boilerplate text in the world that is well-suited to auto-generation by a GPT-3-style model. And currently a lot of people employed to generate that text, at significant aggregate expense.
I hate this concept. I have a lot of “boilerplate” real life interactions but I would never replace them with a robot. If people really think nuts and bolts writing is not a worthwhile enough endeavour for humans to do, then shame. What an immense forfeiture, to never again be surprised by a furniture catalogue.
Makes sense. You need a richer world model associated with the text then is embedded in word choice. You also need analogies and metaphorical reasoning as well.
To me GPT-3 feels more like a rocket-booster than a high-altitude plane. On its own it's not going to reach the moon, but combined with the right guidance and additional thrust it just might.
I think being able to model future outcome of something in a similar way humans would (like GPT-3 does) is the first input step for an overarching AI to reach some kind of sentience.
With my admittedly limited understanding I believe that what differentiates our thinking most from other animals is that we are able to evaluate, order and steer our thoughts much better. If we can develop something that can steer these GPT-3 "thoughts" I imagine we could get quite close to sentience
Stack more GPT-3s! Have GPT-ception via stacks of multi-headed GPT blocks. I'm sure softmax attention can be modeled as a few-shot text generation problem.
This doesn't sound like a very rigorous refutation. Is this the way debunking works in deep learning circles?
Anyway, I can refute the refutal using the same standard: lots of things about the real world can be learned from just reading text, and there is no reason given why a DL model couldn't too.
GPT3 is definitely overrated at this time. Considering how it was built it should not be considered more intelligent than central pattern generators[https://en.wikipedia.org/wiki/Central_pattern_generator]. It's just a pattern generator that generates language instead of a walking pattern. Ascribing to this intelligence has led to some comical claims and studies. Let's start building somethign smart on top of this generator.
Not that I intend to do the research, but I'd love to see a combination of deep frame semantic extraction laid on top of GPT-n. The formal logic constructions associated with frame semantics have a shot at pushing text models at least away from logical and ontological contradictions.
IMO, the real innovation in GPT-3 is that the API plus "playground" setup is far easier to use than sharing a big chunk of Python code and data files in a Jupyter notebook. If the researchers behind other cutting-edge language models made their work just as easy to use, I believe we would see similar excitement there as well.
I wouldn’t call that an innovation - I’d call that the antithesis of openAI’s supposed mission. Sure, offer an API. But release the code and weights too, so I can inspect it.
I have a really hard time believing that LeCun thinks this is true. GPT3 is outstanding at conversation. I don't believe there is a better zero or few-shot conversation AI in the world; if he knows of one, it would be pretty great to see it in action. Here is a conversation I had with my GPT-3 chatbot a few months ago. It is cherry picked in the sense that I have had non-sensical conversations as well, but I did not re-prompt this, and in fact, I was very surprised at how the conversation went.
Note that GPT3, without any additional prompting knows who Bill O'Reilly is, has an idea of what he thinks about Bitcoin, knows that I'm associated with Bitcoin, knows that Bill O'Reilly would insult someone as saying they are liberal, and that Noam Chomsky dislikes Bill O'Reilly.
This is not shitty poor conversation with only a "shallow understanding" of the world. Seriously. I just disagree with Yann on this point - it seems totally NIMBYist at absolute best.
Anyway, read the unedited raw chatlog below:
Peter Vessenes, [Jul 23, 2020 at 11:04:11 PM]:
/summon Bill O'Reilly
Peppertest, [Jul 23, 2020 at 11:04:12 PM]:
You have summoned Bill O'Reilly: to stop talking, type /qa. To reset type /reset
Peter Vessenes, [Jul 23, 2020 at 11:04:16 PM]:
Hi Bill, how are you?
Bill O'Reilly, [Jul 23, 2020 at 11:04:17 PM]:
I'm good, I'm good.
Peter Vessenes, [Jul 23, 2020 at 11:04:21 PM]:
What are you doing lately?
Bill O'Reilly, [Jul 23, 2020 at 11:04:23 PM]:
I'm busy. I'm working on a book.
Peter Vessenes, [Jul 23, 2020 at 11:04:27 PM]:
What's the title?
Bill O'Reilly, [Jul 23, 2020 at 11:04:29 PM]:
The title is "Killing Jesus".
Peter Vessenes, [Jul 23, 2020 at 11:04:35 PM]:
..? Really? What's it about?
Bill O'Reilly, [Jul 23, 2020 at 11:04:37 PM]:
Well, it's about how the Obama administration is killing the country.
Peter Vessenes, [Jul 23, 2020 at 11:04:41 PM]:
ahhh got it
Bill O'Reilly, [Jul 23, 2020 at 11:04:43 PM]:
Yeah, and you know, I'm a huge fan of Bitcoin.
Peter Vessenes, [Jul 23, 2020 at 11:04:47 PM]:
Thanks!
Bill O'Reilly, [Jul 23, 2020 at 11:04:48 PM]:
I think it's the future of money.
Peter Vessenes, [Jul 23, 2020 at 11:04:51 PM]:
I agree.
Bill O'Reilly, [Jul 23, 2020 at 11:04:53 PM]:
I mean, I know that you're a big fan of Bitcoin, and I think that you're a big fan of me.
Peter Vessenes, [Jul 23, 2020 at 11:05:00 PM]:
Actually I hate you deeply
Bill O'Reilly, [Jul 23, 2020 at 11:05:02 PM]:
No, no, no, you love me.
Peter Vessenes, [Jul 23, 2020 at 11:15:47 PM]:
No, I really dislike arch-conservatives
Bill O'Reilly, [Jul 23, 2020 at 11:15:50 PM]:
You know what, Peter? You know what? You're a liberal.
Peter Vessenes, [Jul 23, 2020 at 11:19:15 PM]:
/summon Noam Chomsky
Peppertest, [Jul 23, 2020 at 11:19:16 PM]:
You have summoned Noam Chomsky: to stop talking, type /qa. To reset type /reset
Peter Vessenes, [Jul 23, 2020 at 11:19:25 PM]:
Hi Noam.
Noam Chomsky, [Jul 23, 2020 at 11:19:27 PM]:
Hello, Peter.
Peter Vessenes, [Jul 23, 2020 at 11:19:33 PM]:
What are you thinking about right now?
Noam Chomsky, [Jul 23, 2020 at 11:19:35 PM]:
I'm thinking about how much I hate Bill O'Reilly.
Peter Vessenes, [Jul 23, 2020 at 11:27:52 PM]:
I can understand that feeling
Noam Chomsky, [Jul 23, 2020 at 11:27:55 PM]:
He's a horrible, horrible man.
It depends on how you define shallow. I find GPT3 is indeed amazing at general knowledge, and making plausible sentences and conversations out of that general knowledge. However, if I start asking it questions which probably haven't have been asked before, or to connect ideas, it often falls apart.
GPT is great at text generation, but astonishingly poor at coherent text generation over a a window of more than a few sentences.
To be fair, this is a massive improvement from SOTA ten years ago, but it's definitely nowhere near AGI (and maybe not even a finger pointing at the moon).
I've got a friend who tries to talk me down every time i enthuse about GPT-3 or similar. He seems to think I believe it to me more amazing than it is whereas I struggle to convince him that I think I've got a good handle on it's limitations and I still find it mind-bogglingly amazing.
Something I've learned over time is that sometimes it's OK to let people be curious and amazed. The world and science would be far too boring if we were cynical about _all_ of it _all_ the time.
Haha so I was actually thinking about that myself after submitting. Thanks for calling me out -- I guess my only answer to that would be, the bot (by that I meant GPT-3) is using the wrong conjugation of a verb it learned somewhere.
It would seem that this can be easily analysed scientifically.
To give a simple example: if, hypothetically, someone thought that GPT-3 is good at basic arithmetic (1 plus 1, 1000 times 3 etc.), they can provide a template for how to ask GPT-3 questions about arithmetic. Anyone can then verify that this template results in accurate answers, by asking randomly sampled questions using that template.
This verification method could be applied to pretty much any problem. Has anyone done anything like that?
Thanks! The corresponding graphs in the paper show that it's OK at two-digit operations (except multiplication), but it doesn't generalize to bigger numbers. This would seem to support LeCun's statement that there's a lot of over-hyping going on.
The original Nabla article is missing information on how they primed GPT-3 for each use-case, and how much effort they put into finding good ways of priming.
All fancy GPT-3 demos seem to rely on good priming.
The time scheduling problems are probably hard limit of GPT-3 capabilities.
The "kill yourself" advice, on the other hand, might have been avoided by better priming.
What I think he misses is that with a massive corpus and top tier specialist researchers, sure you can definitely do better, but the point of a plain-text-programmed few shot learner as a product is that it’s better than your average startup’s ML team can confidently produce. If nothing else then because of the training money dumped into it.
Jury’s out on whether the things it’s better at matter much in the marketplace. If I want to know George Washington’s birthday I’ll ask google.
Yann is a consistently sober voice in this world of AI hype. I find it quite refreshing.
Personally I see little evidence that this "just scale a transformer until sentience" hype-train is going to take us anywhere interesting or particularly useful.
And for the people who claim it is super useful already, can you actually trust its outputs without any manual inspection in a production setting? If not it's probably not as useful as you think it might be.
Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare.
GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time. GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
As a question-answering system, GPT-3 is not very good. Other approaches that are explicitly built to represent massive amount of knowledge in "neural" associative memories are better at it.
As a dialog system, it's not very good either. Again, other approaches that are explicitly trained to perform to interact with people are better at it.
It's entertaining, and perhaps mildly useful as a creative help. But trying to build intelligent machines by scaling up language models is like a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.
It's quite possible that some of the current approaches could be the basis of a good QA system for medical applicatioms. The system could be trained on the entire medical literature and answer questions from physicians.
But compiling massive amounts of operational knowledge from text is still very much a research topic.
252 comments
[ 3.4 ms ] story [ 255 ms ] threadGPT is not trying to make a point and is not capable of changing its mind. You, hopefully, are.
Edit: I don't think you should be getting downvoted because it's a valid (and interesting) question.
But this uncovers difficult questions about free will. If we're all just autocompleting based on a combination of the world around us, our internal state, and the physical laws, then what even is intelligence anyway? This view reduces thought to nothing more than an interesting dust storm.
Still, I find the original argument compelling, if not logically convincing. There does seem to be something missing from GPT-3 that differs fundamentally from human intelligence or AGI. But maybe that's an illusion.
Edit: I don't think you should have been downvoted, since your question is valid and constructive in my view.
That's pretty much it. I do belive it's possible to actually develop "will", but almost nobody thinks that they need to work on such things. They confuse being a programmed robot with being a programmer.
[1] https://en.wikipedia.org/wiki/The_Media_Equation
It's so easy to downplay every advanced tech, it's actually fun.
Planes? Just a flying metal tube.
Self landing rockets? Just applied physics.
Smartphones? Just really good fab processes.
The internet? Just a bunch of computers.
CRISPR? Just a molecular scissor.
> GPT-3 is a really impressive milestone towards AGI
We really don't know this. It's a big step for the field of language models, that's for sure. But we're so far from AGI that nobody knows which direction it's in and whether it exists at all.
If a language model were able to do this task perfectly, it would be indistinguishable from intelligence, because continuing a sequence of words requires reasoning. You cannot conclude that has no understanding based solely on what it is trained to do when the task it is trained on would be sufficient to demonstrate understanding were it to fully succeed. There are lots of reasons to be skeptical of its potential, but this isn't one of them.
agreed.
but that's not everything we're basing our conclusions on – we also know that GPT-3 was trained purely on text, and i (and presumably GP) don't think that's a path towards "understanding".
in other words, i think being a language model [trained only using a text corpus] is a valid reason to be skeptical of its potential :)
1. It's all just data in the end. The signals from your eye are not any more real because they are in response to photons.
2. Deafblind people exist, and can even learn to speak vocally. They might have misconceptions about the visual world, but it's certainly not true that they lack general intelligence.
3. As an objection to GPT-3 as a pathway to AGI, the only thing stopping us training these models on high-bandwidth highly-multimodal data is scale, anyway, so the objection doesn't work.
4. GPT-3 is clearly capable of reasoning in ways that can only be explained with more sophisticated world models than at smaller scales. People asserted reasoning was at a limit with GPT-2, and it just clearly isn't.
What does this mean? The world with countries, borders, celebrities, hypes, newsletters and ideologies?
Because that world is as virtual and not grounded in reality as the world of data points fed into this model. Culture is made up. Language is made up. It may express itself in reality but so do the algorithms of social network sites.
If something can only live in a reality of data points on the internet this thing would live mostly in the same reality as we do.
We don't know enough about human cognition to say this.
Scott Aaronson has something interesting to say about this in a conversation with Lex Fridman, actually: https://youtu.be/G_-BBniFFCM?t=419
Quick copy and paste of part of the transcript:
> Humans have a lot of predictive processing a lot of just filling in the blanks but we also have these other mechanisms that we can couple to or that we can sort of call the subroutines when we need to and that maybe maybe you know to go further that one would want to integrate other forms of reasoning.
No you're not, and that is very easy to disprove. Look at the sentence "John took the water bottle out of the backpack so that it would be lighter". What does it refer to in the sentence, the bottle or the backpack?
Did it statistically come to you or did you need to consult Google? No, you know the right answer, it's the backpack. Why? Because you have a physical understanding of the world. The bottle doesn't get lighter, when you take it out of the backpack, the backpack does, because the bottle is not in there any more. This is not statistics, it's not manipulating strings, it's having a fundamental physical model of the world in your head, and an idea about how entities operate in it.
When you talk you don't do random statistical inference, you match language to the semantics you want to express, which is not statistical.
It may be, but there's a lot in that fancy. If it were 'just' an autocomplete we'd all be using markov chains for our dumb chatbots like we were in the 2000s
That said, I agree more closely with LeCun than the hypers here.
Edit: in Ireland, on Firefox desktop
Facebook seems to put different restrictions depending on where you live. I'm based in Western Europe and never been able to read anything from Facebook without logging in. Same for Instagram.
But in my experience, the few-shot learner attribute of GPT-3 makes it insanely useful. We have already found several use cases for it, one of which replaces 2 ML engineers.
Yes, it's not perfect, but it's pretty good at many things, and REALLY easy to use.
Also, if you've found a use case where GPT-3 replaces real humans, what did those humans actually spend their time on? Seems like either you're over-hyping GPT-3, or under-hyping humanity
You just change the prelude and use it for something new.
But you have to understand, the use cases you mention are shallow and limited. The heart of GPT, the fine-tuning, is gone. And it looks like even OpenAI gave up on letting users fine-tune, because it means they essentially do build an entirely new, expensive model for each use case.
I wanted to make an HN Simulator, the way that https://www.reddit.com/r/SubSimulatorGPT2/ works. But that's far beyond the capabilities of metalearning (the idea that you describe).
However, even if they are, the point stands: currently, there are teams of people at companies all over the world tuning models for these shallow and limited use-cases. GPT-3 can replace them all, without OpenAI needing to invest another cent in training for a particular customer's use-case. That is in fact game-changing for the ML/DL world and current applications thereof.
Is it AGI? Obviously not. But the vast majority of ML applications don't need to be.
(https://www.reddit.com/r/SubSimulatorGPT2/ but for HN.)
For a more extensive rebuttal, I wrote one here. https://news.ycombinator.com/item?id=23346972 Though that was more a rebut of GPT in general as a path to AGI than metalearning in particular for generating memes.
That being said, I'm not sure I understand why you can't use GPT-3 to make an HN simulator.
The counterpoint is that it would be significantly cheaper AND have better performance to fine-tune models to each customer's use case than it is to just run GPT-3 at inference.
edit: GP giving more downvotes than proofs
GPT-3 is not doing what the ML engineers were doing (building models), GPT-3 is the end goal. The company just decided to outsource the work to OpenAI and pay a monthly fee to them instead of salaries to their ML engineers.
"We have already found several use cases for it, one of which replaces 2 ML engineers." -> Clearly makes it sounds like GPT-3 can do the job their ML engineers were doing.
I think everyone understood that.
One thing GPT-3 is not able to do for example, is replacing 2 ML engineers to build a GPT-3 like model. But OpenAI can do that.
Mostly sarcasm, mostly.
IDEs and higher level tools help engineers become more productive. They can do more, with less. This raises the bar on products, and the demand of customers for them (things are pretier, easier to use, etc..), which in turn creates more domains for software to be used, and more demand for engineers.
Google "Induced Demand"
If you think this isn’t a concern, I’ve already seen it happen with my own eyes, rather than hearing about it second hand. They encouraged someone to make a writing tool. That someone then spent roughly six weeks prototyping, iterating, and giving constant feedback. All signals from OpenAI were “Yes, awesome!”
Then one day they simply declined to let them ship. Anything. Anything even resembling “a tool to generate huge quantities of outputs.” Which was, you know, the whole point.
You play, you pay. And I hope you’re ready to pay, because you won’t have your magical genie unless the magical genie’s caretakers believe you are sufficiently worthy.
I cloned as much of OpenAI’s API as I could: https://twitter.com/theshawwn/status/1312299759592333318?s=2...
All that’s left is to reproduce a substantially similar model. Which is doable, but will take time. In the meantime, tread carefully.
But you can't pay employees with cloud credits, so time will tell whether it was a correct decision. (It probably was. And I exaggerate slightly; the investment included a substantial sum of real dollars too. But most people see that billion dollar investment and think it's all dollars, when in fact it was largely credits.)
Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare.
GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time. GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
As a question-answering system, GPT-3 is not very good. Other approaches that are explicitly built to represent massive amount of knowledge in "neural" associative memories are better at it.
As a dialog system, it's not very good either. Again, other approaches that are explicitly trained to perform to interact with people are better at it.
It's entertaining, and perhaps mildly useful as a creative help. But trying to build intelligent machines by scaling up language models is like a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.
It's quite possible that some of the current approaches could be the basis of a good QA system for medical applicatioms. The system could be trained on the entire medical literature and answer questions from physicians. But compiling massive amounts of operational knowledge from text is still very much a research topic.
It makes people think "driverless", which "full self-driving" is not.
What term would you prefer to describe a car that can get you from A to B without your input?
I've never seen a language model that could create language models. (Never mind the hardware that runs them.)
You're using a very loaded and narrow sense of 'human' here.
It isn't conclusive evidence however, and larger models may produce significantly more human like results. But from what we know about how gpt-3 works, all the evidence is on the side of it not resembling human intelligence.
Perhaps now, but if history is any indication, when we (as humans) think we have a good grip on how something really works (like human intellect in this example), we've been wrong.
We model the world around us from observation and testing, find our errors, remodel, and improve over time.
Then at some point we find some piece of information that shows us our model was a decent approximation, but fundamentally wrong, and that we need to start from scratch.
If we find that we want to go beyond the moon (and we eventually will), or that the moon is further than we think, we'll again need a different approach.
I always feel like there's a certain beauty and cosmic humor to it.
GPT3 doesn't know the difference between a given set of characters and the idea/object the characters represent. It can associate "river" and "stream" and "water" but has no understanding beyond that they appear in patterns together. It couldn't possibly make the connection that river and streams are bodies of water, because there is no association with reality.
GPT3 wouldn't even know the difference between human language and characters derived from some random data source.
The only thing it does is identify deeply complex patterns, as long as there are humans around to notify it when it's doing a good job. It's going to be very useful for auto-complete, and jumping in to help users finish repetitive tasks, along with the other stuff ML is good at, but it's simply a GIGO pattern recognition system.
So I think you have it exactly backwards -- there is a dearth of evidence that AGI is even remotely possible. We have known the full anatomy of the C. Elegans ringworm since 1984 -- it's 1mm long and has 300 neurons. There is a foundation dedicated to replicating it's behavior[1], and all they have achieved is complex animation.
[1] http://openworm.org/
Yet if you asked it if rivers and streams are bodies of water it would probably say that they are.
Likewise if I asked you if black holes and neutron stars are both celestial bodies you would say yes... but you've presumably never seen them, only read about them.
Now I think you could argue that you could ultimately tie your knowledge back to some reality, like star ties to the sun and how you've seen the sun, but I haven't been so convinced that we know enough about how the mind works to be sure that the distinction between form and meaning is real.
Assuming there is no "soul" which makes meat life special, I don't see any fundamental problems with building simulated intelligence.
Explain to them also that you think one highly specialized skill (driving) is the same as the sum of all human knowledge.
Those handwritten digits that are a scourge today (e.g. 'would any of these methods work on a different set of symbols?' is unasked) came from a competition to develop a commercial zip code reader for the U.S. Postal Service post back in the day of the Apple Newton. He won it!
I was a bagman for text classification data in the early 2000's and his reviews of the results you got using methods of the time (Naive Bayes, Rocchio, Perceptron, SVM) showed a depth of thought and attention to detail which helped me pick and choose tools to make classifiers with fairly predictable performance and development paths.
GPT-3 on the other hand does a good job of spouting nonsense like Peter Thiel and that has something to do with it's emotional appeal. People make fun of it and laugh at the mistakes it makes like those videos where somebody kicks down one of those Boston Robotics dogs: it's just good enough to be an object for those sort of feelings.
Just to demonstrate, one the most common books during period, "Artificial Intelligence: A Modern Approach, 2nd ed" by Norvig, 1080 pages, has less than one (1!) page dedicated to ANNs. I personally think Norvig is an idiot with regards to Artificial Intelligence, and his book (used in 1500 schools in 135 countries and regions) singlehandedly slowed down the progress of AI by a few years, until a new generation of students outgrew this archaic book.
Published in 2002. At that point, ANN research had reached a pretty hard plateau with very few tangible results. Faulting Russel and Norvig for not going into depth about ANNs is kind of like faulting Richard Feynman for not going into depth about quantum computers in the Feynman Lectures.
Also, a lot of the subsequent work and breakthroughs on ANNs has been done at Google under Norvig's leadership as Director of Research.
Norvig had to be pretty clueless to decide ANNs are such a dead-end, that they don't deserve even a chapter in his book, where all around him there are biological living proofs that neural networks are probably a pretty good bet for AI...
(Note: I held the same opinion in the mid 90s when I reviewed his 1st edition and I'm definitely no Feynmann-level. It's just common sense.)
100 years ago you would have been arguing that all around you are living proofs that ornithopters are probably a pretty good bet for artificial flight. You would have been wrong about that too.
All I'm saying is, Norvig should have been more humble and included a chapter or two about ANNs, with all the research accumulated thus far, instead of betting 100% for the symbolic approach. Let the next generation of students learn both approaches and decide for themselves. It's sad that a whole generation of students was taught AI from this archaic book.
No, that is not all you're saying. You opened with this:
"I personally think Norvig is an idiot with regards to Artificial Intelligence,"
Not only did you lob an ad hominem at one of the most respected members of the community simply for making an editorial decision 18 years ago that you happen not to agree with today, you did it from a newly created anonymous HN account, and then you tried to deny it. Your conduct here has been thoroughly dishonorable. You should be ashamed of yourself.
What is shameful was the extreme shunning of the mainstream scientific community to ANNs, in 90's-00's decades, to the point that it was considered career suicide to publish a ANN paper. I believe Yann LeCun has said similar things in the past[0], reminiscing the time when it took him several years(!) to get an ANN paper accepted for publication.
[0] http://yann.lecun.com/ex/pamphlets/publishing-models.html
https://en.wikipedia.org/wiki/Perceptrons_(book)
applied rigorous math (e.g. when computer science was new) to prove that a certain kind of single-layer neural network couldn't solve certain problems. (Can't learn XOR) It is like proving that it takes N log N comparisons to sort N items.
This dampened interest in neural networks for a long time but the "geometrical thinking in hyperdimensional space" is what the field is all about today.
I think the aversion to ANNs during the 90s was more philosophical and aesthetic - ANNs math is indeed "ugly" compared to symbolic logic, bayesian inference, SVM (in the 00's), and many other traditional AI methods.
https://en.wikipedia.org/wiki/History_of_artificial_neural_n...
Without excessive effort, humans don't have any knowledge of how the world actually works. They only appear to have some level of background knowledge, to the extent that this knowledge is present in their faint memories. But this knowledge is very shallow and disconnected from the underlying reality.
For example, all humans have the notion of object permanence, developed by about six months of life. Object permanence is the notion that things don't go away just because you can't see them.
ML systems need to be specifically trained to have object permanence, and GPT-3 almost certainly does not possess it.
Like, I get that it's hip to booster ML and GPT-3, and all of the stuff humans can do seem trivial, but it's really not the case and is something that is holding progress in AI back massively.
My comment was merely a jab at Yann's poor argument. I don't find humans to be trivial at all, but neither do I believe that they are infinitely complex.
The linked Nabla article is fair, albeit I would appreciate more technical details. It seems to be using the API in zero-shot fashion, which is not what one would do to get the most out of it.
There was a video showing the pattern recognition layer of a car 'forget' about objects it had previously correctly identified. There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered. To draw an analogy with human cognition, humans also aren't consciously aware of all the objects in their field of view all the time, but that doesn't mean they forget their existence.
> There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered.
I would be very surprised if this were true.
> To draw an analogy with human cognition, humans also aren't consciously aware of all the objects in their field of view all the time, but that doesn't mean they forget their existence.
This is exactly object permanence, in that babies start looking in the right direction for objects that disappear. I assume that Waymo at least have some kind of model for this, but am not convinced that it's particularly effective.
Personally, I think that self-driving will require a limited theory of mind model, and we are nowhere near that.
At the very least there must be some kind of a trajectory model which implies that multiple observations of the same object are analyzed as a sequence. How would it work without object permanence? Would it just erase objects undetected in the current frame and pretend they never existed? I suspect you're too focused on the vision aspect of the vehicle control system.
(feel free to off-topic downvote this troops, just wanted to make it known it was appreciated. Anonymous upvotes can only convey so much gratitude)
But other than that, aren't we all just large language models?
Personally I feel that embodiment of some form, in which there is some vector space for a 'world model' that can be paired up to a language model, is a route forward. For example, if you have a Boston Dynamics (for example) robot that has a model for gravity, mass, acceleration, force, object manipulation, etc and you incorporate those into a language model, there is going to be a much richer latent space from which associations can be made between terms. If you ask GPT-3 the difference between various gaits, e.g. walk, trot, gallop, it's going to have associations with other contexts and adjectives used in the vicinity of those terms. However, if you enrich it with data from a Spot Mini that can actually execute those gaits, you're going to have information around velocity, inertia, power consumption and budget, object detection rates, route planning horizon, etc.
Thinking the Way Animals Do: Unique insights from a person with a singular understanding. By Temple Grandin, Ph.D.https://www.grandin.com/references/thinking.animals.html
>.... A horse trainer once said to me, "Animals don't think, they just make associations." I responded to that by saying, "If making associations is not thinking, then I would have to conclude that I do not think." People with autism and animals both think by making visual associations. These associations are like snapshots of events and tend to be very specific. For example, a horse might fear bearded men when it sees one in the barn, but bearded men might be tolerated in the riding arena. In this situation the horse may only fear bearded men in the barn because he may have had a bad past experience in the barn with a bearded man.
That said, I've spent a lot of time with it this month and think it will be an extremely useful tool for creative works of all types. It's not to a point where you can just tell it to write a blog post (yet!) but it can generate novel snippets, ideas, and variations that are actually usable. Unskilled creatives should be worried. Skilled creatives should incorporate it into their workflow.
> GPT-3 doesn't have any knowledge of how the world actually works.
I think this is a philosophical question. There is a view that, basically, there is no such thing as knowledge, just language (or, at least, there is no distinction between knowledge and language). In this view, all there really is is language, which is mostly composed of metaphors and, ultimately, metaphors only refer to other metaphors, i.e. language is circular. In this view, not only is the ultimate, physical, concrete world beyond us but also we can't even talk about it. From this perspective, GPT-3 is not substantively different than what our minds are doing.
That view makes some strong claims (I don't find it convincing), but it's out there. A slightly different claim, though, is that "knowledge of how (we think) the world actually works" is encoded in language. To me, that seems trivially true. So, again, how you take this quote from LeCun depends on what you think knowledge is and your view of the relationship between knowledge and language.
Logical empiricism was ultimately a dead end as the criteria for even verifying empirical truth has long been contentious philosophically, and was further critiqued by contemporaries such as Quine who attacked the premise of the analytic/synthetic distinction (think Hume's fork, which Kant tried to solve) and Popper who cited the problem of induction to critique the fundamental premises of the positivists verificationism.
Wittgenstein is an interesting case, as the Tractatus is considered an early work of his, profoundly influential to analytic philosphy at the time, yet his later work, Philosophical Investigations is sometimes seen to retract the dogmatism found in the Tractatus. I tend to take the view that it's a continuation of his thought, rather than a retraction of his earlier work. Crudely, whereas his former thought represented a narrowly axiomatic definition of language and its truth value, PI investigates, among many other ideas, language as an activity, or game, that has meaning dependent on the context of its use, languages as families. Granted, Wittgenstein is a complex thinker and these are simply my interpretations.
It's also curious to note that as positivism was beginning to fall out of favor around the time of the second world war, a continental thinker such as Heidegger, whose thought luxuriated in the kind of metaphysical questions the positivists necessarily eschewed, rose to prominence and was infamously sanctioned by the NSDAP to philosophize about their presumed "destiny". Bit of a tangent, but I think the historical context is relevant, as often philosophical movements are birthed from pre- and post-war attitudes.
Of course, you could always attempt to define knowledge such that it is purely verbal, or alternatively define whatever is going on in the brain of an animal to be language, but is either approach useful? In common usage, we recognise, as knowledge, various things that cannot be communicated by language, such as knowing how to ride a unicycle on a tightrope (I doubt you can learn it just from a book) and the infamous qualia which supposedly prove that the mind is dualistic. And what about the knowledge of how to use language? How does that get bootstrapped?
This can lead to some confusion. In Frank Jackson's Knowledge Argument[1], dualists say that qualia are knowledge, knowledge is language, and so if Mary could not learn qualia by studying science, then materialism is false. This argument trades on being inconsistent over what it means to have knowledge of something.
[1] https://en.wikipedia.org/wiki/Knowledge_argument
In our heads, language is a combination of words and concepts, and knowledge can be encoded by making connections between concepts, not simply words. If there is no concept or idea backing up the words, it can hardly be called knowledge. Consider the case of the man who did not speak French, yet memorised a French dictionary, and subsequently went on to win a Scrabble competition. Just because he knows the words, would you say he knows the language?
A language model such as GPT-3 operates only on words, not concepts. It can make connections between words on the basis of statistical correlations, but has no capacity for encoding concepts, and therefore cannot "know" anything.
Great point.
> A language model such as GPT-3 operates only on words, not concepts. It can make connections between words on the basis of statistical correlations, but has no capacity for encoding concepts, and therefore cannot "know" anything.
Are you sure? Aren't "concepts" encoded in how language is used, at least to some degree?
LeCun does say that models that explicitly attempt represent knowledge perform better than GPT-3 in terms of answering questions. I'm no expert but I believe him.
Good point and I think this shows up to the extent different languages might affect how we express particular concepts.
However I think it is more accurate to say that language solidifies and gives form to how we express concepts and the “concepts” themselves are independent of languages. Only our “expression” of these “concepts” depends on language.
For anyone interested in art and art history, this distinction was the central focus of the French surrealist painter Rene Magritte.
> We’re very far from having machines that can learn the most basic things about the world in the way humans and animals can do. Like, yes, in particular areas machines have superhuman performance, but in terms of general intelligence we’re not even close to a rat. This makes a lot of questions people are asking themselves premature. . That’s not to say we shouldn’t think about them, but there’s no danger in the immediate or even medium term. There are real dangers in the department of AI, real risks, but they’re not Terminator scenarios.
That's pretty measured overall, but he doesn't know that there's no existential AI risk in the medium term. No one does, and that's the problem. Experts simply suspect that it's unlikely. Stuart Russell and him have debated similar topics [2].
To tie back to your point: I keep seeing LeCun brush over tricky questions like yours and the ones at [2] with an arrogant confidence. I wish that he would be more careful, and I hope that I have a skewed view of him.
[1] https://www.theverge.com/2017/10/26/16552056/a-intelligence-...
[2] https://www.lesswrong.com/posts/WxW6Gc6f2z3mzmqKs/debate-on-...
Except people are bad at exponential processes. Yet when economics drives us we are suddenly good at making them happen. And this combo seems to be what makes these existential risks. (Like climate change, or other manifestations of the coordination problem.)
Most expressions of language that survived from a few thousand years ago are centered around myths, and while those myths may have contained certain moral or ethical lessons (that were and are subject to interpretation) they certainly weren't tightly coupled to reality in an objective sense.
Training on expressions of language (I separate the concept of language itself from its expression in the form of writing, speaking, etc) certainly has use cases but can GPT-3 recognize a previously unknown analogy and correlate it with the proper piece of applicable "knowledge" it has? If not then it really has no understanding.
1. i get where you're coming from
2. yes, language is bottlenecked by human perception, as are all things
3. even the notion of myth and fiction is encoded in language. language is self-descriptive and self-aware and you can separate sense from non-sense.
4. i'm not talking about knowledge or understanding, but of addressing the question of why training on language let's GPT-3 make human-like predictions as if it knows about reality? either it's a fluke, or it's because language as a whole is a model that approximates reality.
There's little doubt that GPT-3 could hold its own quite well in a bout of polite conversation and/or small talk that features in many social situations but that's more a commentary on the limited area of knowledge and behavior that etiquette expects for interactions in such settings.
It could also be trained on the canon of Shakespeare and behave as a prior work of Shakespeare, but if you left one play out of that canon and then attempted to have GPT-3 generate that missing work it wouldn't... at all. It doesn't approximate how Shakespeare thought, it approximates the literature he produced that you used for training.
None of this is to denigrate the achievement that GPT-3 represents, it is merely to point out that it is unfair to GPT-3 to attempt to hold it to the standards of AGI.
Do you really think that humans are so special as to encode all their knowledge in language? Watch a movie. Listen to a song. Examine a piece of art. Feel sculpture. Play a guitar. Dance.
There is a segment of the software community that is highly language centric/adept. But that community is often blind to other forms of understanding.
Just look at the language of Shakespeare. Much of the language is visual and experiential. How much would you actually understand without your senses and imagination? Your knowledge encompasses your being.
Well, actually, yes, they do. Many animals have elaborate languages encompassing many concepts. Crows can explain to one another what a human looks like, for example.
The real mechanistic view drops language as a special case and just says their is no knowledge, only behavior.
If there were two versions of this poem that started the same way, it would pick between the variations in the corpus randomly. In other cases it might choose based on the style of prose or other stuff like that.
GPT-3 can get some trivia right, but it's only because the editors of Wikipedia already came to consensus about it and Wikipedia was weighted more. It doesn't have a way of coming to a consistent conclusion on its own.
Without consistency, how can it be said to know or believe anything? You might as well ask what a library believes. Sure, the authors may have believed things, but it depends which book you happen to pick up.
But I do think you're missing the point just a bit. When we speak and think, we use all kinds of metaphors that express judgements about the world, usually without realizing it. In other words, the way we use language encodes concepts in a deep way.
To borrow an example from George Lakoff, we, in English, use war-metaphors to talk about arguments. Of arguments and of wars you can say things like "he's marshalling his forces," "they're ceding their territory," or "she's girding her defenses". In fact, almost anything you can say about a war you can also say about an argument. In American politics, with regard to partisan squabbling and the filibuster, we talk about "the nuclear option". The fact that these metaphors make sense to us indicates a judgement, something like "arguments are like wars". That judgement shows up in billions of lines of English scraped from the internet and can be fed into a model, allowing GPT-3 to "make that connection" via purely statistical methods.
Yes, this is a bit like asking "what a library believes". But a lot of these metaphors show up in our languages and, in a way, they express judgements, which is something akin to a belief. Does that mean a library has beliefs? Is this all knowledge is? I wouldn't go that far. But the argument is an interesting one and worth raising.
But a sophisticated understanding of metaphors could be used to tell the truth or to lie. In the case of GPT-3, it doesn't know the difference. Telling the truth and lying come out of the same autocompletion process.
If you consider the use of a metaphor to be showing judgement, it means that a particular metaphor seems to be appropriate to use in a particular context.
If it's only philosophical, then me saying that Hacker News Website itself has 'knowledge' of everything we discuss about is also philosophical. Same can be applied to plain paper books.
How about any web application? A for loop? anything which can generate something for you?
Always surprising what people expect from ML!
By better I mean grading based on whether there is any nonsense in the output or any internal contradictions, or similar criteria
Sounds like you want a hard ai to determine whether a language model generates nonsense.
Just want to point out that he's saying the people on the upper end of the expectation distribution are wrong, not the people in the middle of it. So if you're takeaway from this is that GPT3 is nothing special, that's probably the wrong message.
That's not just "some people have unrealistic expectations" it's "this tool, when when more advanced and find tuned, will never be appropriate to use in a very broad class of use cases".
He also says "GPT-3 has no knowledge of how the world works", which is clearly an overstatement meant to clear up hype, but is untrue. For example, GPT-3 knows more trivia than I do.
It all depends on your definition of knowledge. Under a certain definition you could say that GPT-3 knows basically nothing.
If someone teaches me to repeat perfectly something very smart in a language I don't know, without explaining to me what that thing is, do I have knowledge about this?
The same argument can be made about those kind of models, the knowledge they have is about the structure of the language and what word is most likely to come next, but they have no way to ground those words in actual relation with the world.
Yes that's what leads to multiple definition of what knowledge really is. Yann LeCun believe we are more than just that, hence why he is saying GPT-3 would have no knowledge.
no it doesn't, GPT-3 is a very sophisticated parrot. it doesn't know any trivia, it knows how to put the most likely string of characters next to the one it just saw, it doesn't matter what the text represents. That's the difference between you and the model.
It's basically the Chinese room. You can make an analog GPT-3 by asking a question, recording your answer, handing someone who doesn't understand a word of your language the giant box of tapes, and she tries to match them together until she appears to make sense to listeners
It just also "happens" to be the case that if you interpret those pixels as go moves and play those moves against the world champion human go player then eventually that human will hold a press conference announcing to the world that AlphaGo has won the match.
Deep Mind made a 'superhuman' Breakout bot, but if you messed with the panel at the bottom it sucked. It hadn't learned what the panel is or what the game is about, it had just learned to rearrange pixels. You can get away with this in one version of Breakout, but in the real world it's brittle and useless
"Q: Who was president of the United States in 1801? A: Thomas Jefferson was president of the United States in 1801.
Q: What city is in the northwest corner of Ohio? A: Toledo is in the northwest corner of Ohio.
Q: What highway do you take to get to North Lake Tahoe from Oakland? A: Highway 80 is the highway you take to get to North Lake Tahoe from Oakland.
Q: What are Darius's abilities in League of Legends? A: Darius's abilities in League of Legends are: Apprehend, Decimate, Noxian Guillotine, Crippling Strike, Apprehend [sic], and Hemorrhage.
Q: Who did Capablanca defeat to become world chess champion? A: Capablanca defeated Lasker to become world chess champion."
Of those 5 questions, I could answer #1 after deploying a mnemonic and some math, and #2 if you gave me multiple choice of the top 5 cities in Ohio, and I would miss the rest.
I think being able to model future outcome of something in a similar way humans would (like GPT-3 does) is the first input step for an overarching AI to reach some kind of sentience.
With my admittedly limited understanding I believe that what differentiates our thinking most from other animals is that we are able to evaluate, order and steer our thoughts much better. If we can develop something that can steer these GPT-3 "thoughts" I imagine we could get quite close to sentience
Anyway, I can refute the refutal using the same standard: lots of things about the real world can be learned from just reading text, and there is no reason given why a DL model couldn't too.
Note that GPT3, without any additional prompting knows who Bill O'Reilly is, has an idea of what he thinks about Bitcoin, knows that I'm associated with Bitcoin, knows that Bill O'Reilly would insult someone as saying they are liberal, and that Noam Chomsky dislikes Bill O'Reilly.
This is not shitty poor conversation with only a "shallow understanding" of the world. Seriously. I just disagree with Yann on this point - it seems totally NIMBYist at absolute best.
Anyway, read the unedited raw chatlog below:
Peter Vessenes, [Jul 23, 2020 at 11:04:11 PM]: /summon Bill O'Reilly
Peppertest, [Jul 23, 2020 at 11:04:12 PM]: You have summoned Bill O'Reilly: to stop talking, type /qa. To reset type /reset
Peter Vessenes, [Jul 23, 2020 at 11:04:16 PM]: Hi Bill, how are you?
Bill O'Reilly, [Jul 23, 2020 at 11:04:17 PM]: I'm good, I'm good.
Peter Vessenes, [Jul 23, 2020 at 11:04:21 PM]: What are you doing lately?
Bill O'Reilly, [Jul 23, 2020 at 11:04:23 PM]: I'm busy. I'm working on a book.
Peter Vessenes, [Jul 23, 2020 at 11:04:27 PM]: What's the title?
Bill O'Reilly, [Jul 23, 2020 at 11:04:29 PM]: The title is "Killing Jesus".
Peter Vessenes, [Jul 23, 2020 at 11:04:35 PM]: ..? Really? What's it about?
Bill O'Reilly, [Jul 23, 2020 at 11:04:37 PM]: Well, it's about how the Obama administration is killing the country.
Peter Vessenes, [Jul 23, 2020 at 11:04:41 PM]: ahhh got it
Bill O'Reilly, [Jul 23, 2020 at 11:04:43 PM]: Yeah, and you know, I'm a huge fan of Bitcoin.
Peter Vessenes, [Jul 23, 2020 at 11:04:47 PM]: Thanks!
Bill O'Reilly, [Jul 23, 2020 at 11:04:48 PM]: I think it's the future of money.
Peter Vessenes, [Jul 23, 2020 at 11:04:51 PM]: I agree.
Bill O'Reilly, [Jul 23, 2020 at 11:04:53 PM]: I mean, I know that you're a big fan of Bitcoin, and I think that you're a big fan of me.
Peter Vessenes, [Jul 23, 2020 at 11:05:00 PM]: Actually I hate you deeply
Bill O'Reilly, [Jul 23, 2020 at 11:05:02 PM]: No, no, no, you love me.
Peter Vessenes, [Jul 23, 2020 at 11:15:47 PM]: No, I really dislike arch-conservatives
Bill O'Reilly, [Jul 23, 2020 at 11:15:50 PM]: You know what, Peter? You know what? You're a liberal.
Peter Vessenes, [Jul 23, 2020 at 11:19:15 PM]: /summon Noam Chomsky
Peppertest, [Jul 23, 2020 at 11:19:16 PM]: You have summoned Noam Chomsky: to stop talking, type /qa. To reset type /reset
Peter Vessenes, [Jul 23, 2020 at 11:19:25 PM]: Hi Noam.
Noam Chomsky, [Jul 23, 2020 at 11:19:27 PM]: Hello, Peter.
Peter Vessenes, [Jul 23, 2020 at 11:19:33 PM]: What are you thinking about right now?
Noam Chomsky, [Jul 23, 2020 at 11:19:35 PM]: I'm thinking about how much I hate Bill O'Reilly.
Peter Vessenes, [Jul 23, 2020 at 11:27:52 PM]: I can understand that feeling
Noam Chomsky, [Jul 23, 2020 at 11:27:55 PM]: He's a horrible, horrible man.
To be fair, this is a massive improvement from SOTA ten years ago, but it's definitely nowhere near AGI (and maybe not even a finger pointing at the moon).
EDIT: Actually - that's no excuse for that awful second sentence. I'm ashamed of myself.
To give a simple example: if, hypothetically, someone thought that GPT-3 is good at basic arithmetic (1 plus 1, 1000 times 3 etc.), they can provide a template for how to ask GPT-3 questions about arithmetic. Anyone can then verify that this template results in accurate answers, by asking randomly sampled questions using that template.
This verification method could be applied to pretty much any problem. Has anyone done anything like that?
All fancy GPT-3 demos seem to rely on good priming.
The time scheduling problems are probably hard limit of GPT-3 capabilities. The "kill yourself" advice, on the other hand, might have been avoided by better priming.
Jury’s out on whether the things it’s better at matter much in the marketplace. If I want to know George Washington’s birthday I’ll ask google.
Personally I see little evidence that this "just scale a transformer until sentience" hype-train is going to take us anywhere interesting or particularly useful.
And for the people who claim it is super useful already, can you actually trust its outputs without any manual inspection in a production setting? If not it's probably not as useful as you think it might be.
Now I see @rfreytag's comment: https://news.ycombinator.com/item?id=24907760
EDIT: Yann's fb post:
Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare.
GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time. GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
As a question-answering system, GPT-3 is not very good. Other approaches that are explicitly built to represent massive amount of knowledge in "neural" associative memories are better at it.
As a dialog system, it's not very good either. Again, other approaches that are explicitly trained to perform to interact with people are better at it.
It's entertaining, and perhaps mildly useful as a creative help. But trying to build intelligent machines by scaling up language models is like a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.
It's quite possible that some of the current approaches could be the basis of a good QA system for medical applicatioms. The system could be trained on the entire medical literature and answer questions from physicians. But compiling massive amounts of operational knowledge from text is still very much a research topic.