This blog post gives more details on the science of training large autoregressive language models that have recently been successful worldwide (ChatGPT, GPT-4). Some thoughts on the similarities between the biological brain and artificial neural networks are also presented.
I think just like there is a "/S" for sarcasm or the ">" to show a quote, we need a shorthand to denote where computer generated text is used like "::ChatPDF" at the end.
This has been around for quite some time, but never ceases to be entertaining.
Hello engineering, my old friend. Nothing inspired more confidence than a set of random numbers cast into a formula and then whimsically called a fundamental "law".
This space is moving so fast my jaw drops every time I read something about it. For example this line:
"... has discovered empirically that the capability for moral self-correction emerges at around 22B parameters. For more scientific details, the paper is here. [https://arxiv.org/pdf/2302.07459.pdf]"
Yah but do those words - as we would typically understand them - mean the same thing when applied to a computational model? A great deal of intellectual legerdemain is going on in many of these descriptions.
I don’t know that it’s sleight of hand. Seems to me that we are finally approaching a point where it might be possible to more precisely define terms like “morality” in a mathematical sense.
This would suggest that moral philosophy could be transformed - in much the same way that natural philosophy turned into physics.
I’m just a layperson and don’t really know what I’m talking about, but I find this all very exciting.
I think it's sleight of hand in that "moral self-correction" is a very complicated phenomenon, and proving that a computing system is doing it would require an incredible amount of detailed theoretical and empirical work. Some of which, yes, might include much more careful definitions of morality. Until that work is done, I think it's somewhere between foolish and negligent to anthropomorphize LLMs.
I agree with you that there are outcomes that would be less ideal. If it helps, I refer to them as Intelligent Tools. I do prefer the "tool" metaphor (and so does Bing Chat) and I hope that companies like Microsoft rethink their "copilot" and "assistant" metaphors.
I don't think they're "dangerous" per se, I think metaphors matter and we should choose the best ones.
I suppose they're talking about utilitarianism? But that's hand wavey math at best. Eg the whole "torture someone for 50 years or remove a speck of dust from trillions of eyeballs" debate. Those who chose the torture see utility as strictly additive and I'm not aware of any strict definition of utility which requires this. So any math in that instance is built on a shaky or even illusionary foundation.
I meant mathematically as in a proof. With LLMs knowledge has become - well, tokenised. We can now study it at an information processing level which likely to be at least similar in structure to how knowledge is organised in the brain. This in turn is likely to give us access to the way knowledge itself works, in a way that was not previously possible.
So we can actually look at concepts like “morality” and see how that is encoded. And I’m confident that this will give us empirical insight into these concepts in a way no philosophy has been able to do until now.
Morality is biologically and sociologically constructed. It's not consistent between people, social cliques, regions, or nations. It's a nebulous concept. And unlike, say, our understanding of disease processes, which may be fuzzy and inexact at times but has an external truth we can hope to discover, there is no ground truth that we are approaching in moral philosophy. There is no platonic morality that we approximate with our muddled intuitions. Morality is nothing more or less than those muddled intuitions. It cannot be distilled to cold logic.
What AI might enable however is a superior form of democratic process, wherein an AI surveys the entire population through a natural language interface and synthesizes the nuanced and conflicting desires of the entire society. This deliberative democracy process would ameliorate the distorting effects of campaign funding and issues like uninformed or misinformed voters and low voter participation. It could also allow a sort of citizen feedback line-by-line on proposed legislation and government action.
What assures is that this apparent morality is not a side effect of morally-aware or biased training data? The lack of adherence to the scientific process in this field is saddening.
1. Adenosine, i.e. the sleep molecule affects all outputs of dopamine including the two heterodimers of dopamine D1 and D2 that happen to be combined with Adenosine receptors A1 and A2 and also impacts the two optical computing components the eye itself as its bathed in dopamine and how we detect colored light and the other neuron component of the eye at the back of the human brain.
Okay, so in my ADHD research I got a bit focused on dopamine.
> To date, no neural network shows a capacity for reasoning and creativity worthy of an animal level. Current LLMs are autoregressive models that predict the probability distribution of the next token given the set of previous tokens called context:
This part seems unsubstantiated. The author implies that next-token-prediction does not lead to reasoning and creativity "of an animal level". But arguably GPT-4 is already at that level (can your dog predict what a novel Python function would output when it is executed, or write a rap song about autoregressive training where every word starts with the letter "a"¹?).
Regardless of whether GPT-4 is there already, it would certainly be a risky bet to say that it is impossible for a GPT-5 to get to "animal level" (whatever that means) reasoning and creativity while still using next-token-prediction.
As the article points out in the next paragraphs, there is no obvious limitation to next-token-prediction, because making the best prediction requires fully understanding the words in the context, as well as their true meaning and the reason they were written.
One of my big issues with these computational artifacts is that they are not modular -- in the sense that, if I wanted a studious snake without claws -- what things do I need to do to get it to produce a version of the snake, but "fixed" in the way which I describe.
E.g. here, when I say modular -- I mean that a "fix" can't be implemented in a local, small way. I need to try again with a new prompt, etc -- and it's not clear to me how to "fix" my prompt to get the right thing.
one way to support this sort of modularity might be with a GPT-4 / langagent integration which would query Midjourney until it gets a picture which satisfies my criteria, changing the prompt, etc.
The problem is that, if the integration or langagent has an issue -- how do I modularly fix those systems? And so on.
This sort of question "how do I figure out how to fix this prompt-based thing" makes me worry about a sort of "turtles all the way down" problem. Okay, so maybe I don't need to understand exactly how to use these systems to get what I want -- just let another LLM do that ... but what if that LLM has an issue ... and what if the LLM which is prompting that LLM has an issue ...
It concerns me. I'm not sure how well founded my concerns really are.
GPT-4 does fine with a reply prompt of "keep the rest as-is but fix this part". I imagine Midjourney would too if it supported conversational prompting.
There really is no arguably about it. There isn't a verifiable definition/criteria for general intelligence that GPT-4 fails that a significant chunk of the human population doesn't also fail.
GPT-4 performs nearly all tasks it's given at at least the average human level, usually well above the average baseline. In some cases (nlp, analogical reasoning - https://arxiv.org/abs/2212.09196, and some others it's top %)
You basically have to make up your own definition that can't be substantiated (don't worry, plenty seem to do this). It almost always boils down "Well it's not "real" understanding, the difference is huge, i can't show you this supposed huge difference but trust me bro, it's not real"
> GPT-4 performs nearly all tasks it's given at at least the average human level
yet, you will afraid to trust gpt-4 to file your taxes through installed browser plugin.
There is a huge gap between some narrow cherry picked (researchers have strong incentive to report mind boggling results) and diverse and multi modal real life situations
I wouldn't trust the vast majority of humans to file my taxes either, and I definitely wouldn't trust any below a certain intelligence level. Nevertheless that doesn't mean those humans are completely devoid of any intelligence at all.
I don't need to cherry pick, I use GPT4 daily. At this point naysayers can only be willfully ignorant or uncreative in their use of it.
Any artificial neural net has a finite set of inputs and a finite set of outputs. That makes it equivalent to some lookup table. You must therefore claim a lookup table has the ability to reason.
Me: Is it true that, in humans, one neuron has an infinite number of possible states (as far as we know)?
GPT-4: It is not accurate to say that a single neuron in humans has an infinite number of possible states. Neurons are complex cells that can exhibit a wide range of behaviors and responses, but the number of states is not infinite.
Neurons communicate with each other through electrical and chemical signals, and their primary functional unit is the synapse, where communication between neurons occurs. The strength and patterns of these connections can vary, resulting in a large number of potential states for a single neuron. However, this number is still finite, albeit extremely large.
Moreover, the activity of a neuron can be described in terms of its membrane potential, which is a continuous variable ranging from hyperpolarized to depolarized states. While there are many possible values for the membrane potential, it is still a bounded range, not infinite.
So, while the number of possible states for a single neuron is vast and complex, it is not infinite. The complexity and diversity of neuronal states contribute to the rich repertoire of cognitive and behavioral functions that humans possess.
That implies a lookup table requires me to provide a citation :) In fact, assuming space is not discrete, not only can a neuron be in any of infinite states, per quantum mechanics it can be in multiple at the same time.
That doesn't mean that neurons could be in multiple states at the same time which is what you were implying. That particular assertion is still horribly wrong.
again, that doesn't mean neurons can exist in multiple states. There is also no also indication higher level computational functions in the brain responsible for reasoning or whatever have you require any quantum computations.
We don't know that in any way that is relevant. A biological neuron is complex and made of lots of elements where very fine interactions can plausibly have effect, so in a sense, there is a combinatorially infinite space of neuron-states, but there's no evidence its upper-level computational function that leads to learning and producing adaptive outputs in response to inputs is not discrete. Indeed it seems that most of the complexity [that isn't just "piping" for life functions – don't forget these things aren't semiconductors in metallic sheets fabricated by TSMC, they have to migrate to their sites and branch out on their own, they consume, they excrete, they function for many decades!] is redundant and serves to compensate for various biological shortcomings like low transmission speed, limited bandwidth, noise and stochasticity, developmental abnormalities, cell death etc.
Analog computers also don't have infinite states, because of the noise floor.
A synapse can encode at most some low tens of states[1], synapses are discrete, spiking is discrete, receptor density is discrete, each neurotransmitter release is made up of discrete number of vesicles with a discrete number of molecules, even epigenetic expression is discrete. That it's not all perfectly algorithmically executed is not key to the function being performed – and anyway, at the limits of high-performance compute we also start to deal with esoteric processes, and ways of suppressing them. We didn't design the brain nor do we have perfect ways of observing it in vivo, so we can't always neatly distinguish the wheat and the chaff; but this cannot be seriously taken as a cause to think it's all wheat.
Non-human animals show that it's not magic either; even apes, with brains architecturally almost (not wholly) identical to ours but smaller, cannot learn any of the hard symbol-manipulating stuff that we or later GPTs can and that we colloquially refer to as intelligent behavior and reward economically. This, I think, is a very strong prior for that sort of intelligence not being dependent on some low-level expressivity of biological neural computational substrate.
Penrose-Hamerroff's "quantum effects in microtubules" thesis is crank science not supported by any direct evidence.
One of the reasons I think LLMs are somehow analogous to brains is that they scale based on size. This seems true for animal brains as well.
But you seem to be suggesting that all animal brains are equally powerful, because they have infinite neuron states, quantum superpositioned neurons, etc.
As far as I can tell, you seem to be suggesting that my old cat’s brain has the same complexity as my brain - right?
So, I’m interested in understanding why my cat is demonstrably stupider than I am, despite having infinite complexity, and why whatevrr you come up with doesn’t also apply to LLM size.
No, I’m saying having a finite number of neurons doesn’t imply a particular number of brain states. Scaling laws actually don’t apply very strictly to brain size. For example, whales have very large brains but aren’t generally thought to be more capable of reasoning than humans. Women generally have smaller brains than men but aren’t thought to be less capable of reasoning.
> No, I’m saying having a finite number of neurons doesn’t imply a particular number of brain states.
But of course it does imply that. It's vast but necessarily finite number of "brain states" however that's defined, which is itself a subset of the vast but also finite number of states of the finite number of component particles.
That clearly isn’t true, since a neuron that could be in infinite states would thereby be capable of storing infinite information, and (as the neuron itself is finite in size) have infinite information density, but there is an established physical upper limit on information density.
Mm not sure that bound applies because neurons are neither of fixed size nor fixed energy. But on the other hand one starts to stretch the definition of neuron.
Which animal body cells can be infinitely large then? This is foolery. "Neurons are of unbounded size" is a ridiculous statement, we any fool can see it's false. IDK what you're aiming at here, try making a case for something sensible?
We hit comment cooldown but it's not foolery exactly, more like reflecting the futility of trying to make these kinds of arguments. The problem with this whole chain is we haven't actually defined what a neuron is. Take the purported bound above. Suppose I move an associated potassium ion farther away. Have I not changed the state? If yes, what's the limit to how far away I can move that ion and have it still count as part of the neuron? That's just one of the limits of physical laws, you can make very precise statements, but only under very narrow constraints.
Here are some bounds, from most theoretical to most practical:
The size of the earth's biosphere
The size of the largest animal.
The size of a human skull cavity.
The size of a human skull cavity, divided by the number of neurons therein ( a few Billion).
Is this not obvious? There's nothing futile about stating these obvious facts.
Do not fool around with "if I move a potassium ion farther away, isn't that a different bound" ? correct, but moving the bound slightly, is irrelevant to the obvious statement that there are still bounds, and they are practical.
Stating that the neuron is not well-defined is similar irrelevance, if you want to precisely measure the bound, you will need to know precisely what is bounded, but I did not, merely stated the obvious truth that every such definition will come with a bound. One can say "definitely smaller than this" in a wide range of ways, without defining the exact edge.
What's the alternate argument: "I won't define it, therefor it might be larger than the solar system". This is not a serious thing to say.
And to think that such bounded systems "have infinite states would thereby be capable of storing infinite information" is pseudo-mystical foolery. Large, or even "vast", is not infinite. And as sibling comment has mentioned, there is a "noise floor" below which analog resolution means nothing.
Not even going to bother with the weird leap of logic you've made here but Memory augmented LLMs are computationally universal - https://arxiv.org/abs/2301.04589
Idk about “above average baseline”. Maybe, because quite a lot of people aren’t very smart. But even GPT4 struggles with complex logic problems, particularly ones with multiple steps, and when it makes logic errors it makes ones a human would spot. It also has trouble counting letters and words, though that’s largely due to its autoregressive nature. It has more knowledge and memory than any human, and I really think it’s emergent, but at its core it doesn’t seem very “intelligent”
>It also has trouble counting letters and words, though that’s largely due to its autoregressive nature.
It sees tokens not words or letters and there aren't many instances of counting occurring in text so it doesn't get learnt.
>and I really think it’s emergent, but at its core it doesn’t seem very “intelligent”
and we're back to the trust me bro argument. It's not intelligent because you say it isn't not because it doesn't perform a great many tasks intelligently. if you say so
If it's so smart already, why hasn't it replaced the most basic functionality in human society yet? Like, can a GPT4 drive a car because most people can drive a car.
The "basic functionality" may just be more complicated than intelligence. Intelligence and language are relatively new capabilities built on top of older, better honed ones, so it shouldn't necessarily come at a surprise that computers get smart before they can properly deal with the physical world.
> There isn’t a verifiable definition/criteria for general intelligence that GPT-4 fails that a significant chunk of the human population doesn’t also fail.
The usual broad indicia of intelligence (which, e.g., IQ is considered as a useful proxy measure for in humans because it predicts/correlates with them) are lifelong success in a wide variety of tasks GPT-4 can’t even begin to approach.
>are lifelong success in a wide variety of tasks GPT-4 can’t even begin to approach.
Right. and what are these tasks and why is intelligence the barrier? (meaning don't tell me something like driving, blind people are not unintelligent)
Well I've tried asking for lists of words that satisfy certain simple constraints, like a theme and a given number of syllables.
Almost no word satisfied the constraint, despite my many tries.
So if you can have a conversation that goes "give me a list of 3 syllables word", you get a list and then you ask for each word "how many syllables is that word?" and get answers like 2, 4, 5, sometimes 3. You can ask the question again and still get the same words with wrong numbers of syllables. Despite a very obvious logical implication of facts here.
This sounds quite similar to Bing famously explaining to a user that 2022 is in the future while we are in 2023.
It's something you would expect maybe a child to do (it reminds of "Patrick Star's Wallet" meme), but to me that clearly fails the notion of intelligence in a very fundamental way. What would be the counter-argument?
Language Models don't see words. They see tokens. So asking them to perform minute operations on letters/syllables/words etc don't work well. Doesn't really have anything to do with Intelligence.
Sentences look something like this to the model, [754 76 4752 67 456]
Here you have two predicates on words P: Sigma* -> N and T: Sigma* -> N. The question is give a list of words w such that for all w, P(w) and T(w) are both true. It has access to the correct values for P(w) and T(w), so there is a simple algorithm that can produce all words, you just enumerate them each one and verify that the word satisfies both predicates.
An intelligent agent should be able to understand this without any understanding of the underlying domain.
If I try to reformulate your answer, this is like words are a different domain than what GPT has been trained on (which is tokens), and so it is not unexpected that GPT cannot generalize to this other domain.
Is that a correct interpretation of your statement?
In my view failing to see the similarity in the relationship between class objects is a sign of a lack of 'intelligence', whatever that may be in a very precise sense.
>In my view failing to see the similarity in the relationship between class objects is a sign of a lack of 'intelligence', whatever that may be in a very precise sense.
Ok but not all relationship similarities transfer across representations like that.
It's like you give an instruction in English, "Find a word that rhymes with ingenious" to a french man. The french man sees, "Trouver un mot qui rime avec ingénieux". Guess what ? His response to "Trouver un mot qui rime avec ingénieux" would not fulfill the request in english that it might in French.
Ingenious and heterogeneous rhyme. ingénieux and hétérogène have no such property
The french man is not unintelligent to be unable to fulfill your request.
There still isn't any evidence that it can pass novel Winograd schemas (as in, ones it cannot possibly be trained on), which almost every human can pass. The abstract reasoning corpus is more of a "lots of people will fail this too" test, but if a human is approaching it carefully they will be able to do it and there's still no evidence that GPT-4 can do it reliably (I expect that to change once it's multimodal).
The systems are very impressive and I think there is a 'there' there, they're clearly intelligent to some significant degree. But it's incorrect to say that they pass every test.
I just tested GPT-4 on the Winograd schemas below, which I made up and (as far as I know) are novel.
It scored 3/4, incorrectly answering "the pouch" instead of "the baby kangaroo" for Question 2 of Schema 1. And now that I look at the question, one could easily imagine the kangaroo being afraid to jump into a pouch that's too large! In hindsight, if a human complained to me that this question was ambiguous, I'd have a hard time disputing them.
Granted these are just two examples, but they aren't cherry-picked. Moral of the story: before one makes a strong claim such as "there isn't any evidence a model can do X", it's often worth making a minimal effort to investigate one's own claim.
========================================
Winograd Schema 1:
Sentence 1: The baby kangaroo was afraid to jump into the pouch because it was too small.
Question 1: What was too small? (Correct answer: the pouch)
Sentence 2: The baby kangaroo was afraid to jump into the pouch because it was too large.
Question 2: What was too large? (Correct answer: the baby kangaroo)
Winograd Schema 2:
Sentence 1: The gardener couldn't grow the flowers in the greenhouse because it was too hot.
Question 1: What was too hot? (Correct answer: the greenhouse)
Sentence 2: The gardener couldn't grow the flowers in the greenhouse because they were too delicate.
Question 2: What were too delicate? (Correct answer: the flowers)
Moral of the story: before one makes a strong claim such as "there isn't any evidence a model can do X", it's often worth making a minimal effort to investigate one's own claim.
Do you think this was a strong refutation ? I just flipped a coin for fun and also got 3/4 correct.
I understand your point, but I mean, something that's intelligent really should've got the second question in that test correct.
I have tested them on novel Winograd schemas, they didn't pass; that's why I said what I said. Your second schema was incorrectly constructed, in both the first and second sentence the pluralisation of the pronoun in question gives away the answer. "It" won't apply to "flowers" because they're plural, and "they were" won't apply to "the greenhouse" because it's singular. As a result, those sentences aren't Winograd schemas because they're not testing common sense pronoun attribution specifically, they're testing the ability to maintain plural status consistency, which is doable by much less intelligent systems. It's also standard to ask "What does the pronoun '[pronoun]' refer to in that sentence?" rather than "What was too small?" or similar, as using descriptive words is also giving away information that you're meant to be testing the model on. But plenty of people make that mistake, so it's not that important.
Summing up, out of your four (not an acceptable sample size to form evidence) attempts to subject a SOTA LLM to Winograd schemas, two of them were actually Winograd schemas and only one of those got answered correctly. That isn't evidence that LLMs can pass novel Winograd schemas. If you actually come across such evidence, you should post it publicly on at least a blog page if you can't access the infrastructure to publish the results in a paper, even though the finding would certainly be novel enough to deserve a paper. As I said: there is no evidence that LLMs can pass novel Winograd schemas as of yet. We could get said evidence tomorrow, but it doesn't exist right now.
I would put a condescending moral of the story here, but I don't feel any particular need to belittle you, and instead hope you just learn from the information as presented.
> But it's incorrect to say that they pass every test.
Interesting! So there is published evidence, that it cannot pass novel Winograd schemas? Could you please provide me with the sources? I would like to have a look at the prompts. From my experience you can make GPT-4 pass novel Winograd Tests by giving GPT-4 additional context.
I'm not sure if you're joking or trolling, but on the chance that you're not:
>So there is published evidence, that it cannot pass novel Winograd schemas?
There is no published evidence that they can, which is what I said.
>Could you please provide me with the sources?
I cannot prove a negative to you with a source. You will have to review the literature like I did.
>From my experience you can make GPT-4 pass novel Winograd Tests by giving GPT-4 additional context.
This is why I am pretty sure you're trolling. This is a lot like saying "You can make GPT-4 do prime factorisation of a semiprime by noting the constituent primes." The point of testing the model is to see if the model can give you the answer, not to give it the answer. The point of Winograd schemas as a test is that they test the ability to attribute pronouns using common sense, without additional context. The reason it's a good test is that every competent human being can pass the test, and as of yet no computer can.
Not everyone is trolling on the internet my fellow human.
> This is why I am pretty sure you're trolling. This is a lot like saying "You can make GPT-4 do prime factorisation of a semiprime by noting the constituent primes." The point of testing the model is to see if the model can give you the answer, not to give it the answer.
It is not "giving the answer to the model" if you adjust the prompt with additional instructions like "this is a test question" or "use common sense". It is about creating the correct frame of context. With these kind of adjustments GPT-4 passes every novel Winograd scheme I have presented it with.
Something is odd to me about the amount of training data needed.
It seems as if animals and humans can learn on far less training data and on a much dirtier and noisier data set. The data an animal or a human trains on has not been curated to remove misleading data or noise.
It seems that one can produce models with amazing capabilities by training them on vast curated data sets, but I get the sense real brains still have a lot of tricks that we don’t understand.
If you want to put it terms of animals, maybe a session is an organism but the model is the DNA. We don’t “learn” from scratch, we have a model that develops our brain and processing centers and wires them to be able to learn from scarce data. This is somewhat what a session is like.
Our AI models are designed and tuned by engineers, which means they start with a blueprint. They are not randomly generated at the architectural level.
I don't agree that a human obviously experiences less learning stimulus than an LLM. For one thing, the human has direct and mutating access to the physical world through many senses. The LLM gets to experience written words, and if it's lucky, images.
That the pretraining can be compressed down into DNA, combined with a second source of DNA through sex, then decompressed/expressed to build a brain that wires itself up as it grows from a handful of cells, resulting in a conscious being at the end is nothing shorter of amazing.
The numbers involved are also an order magnitude greater. In a human brain, there are approximately 60 trillion connections between neurons. Made with a variety neurotransmitters and regulated by a host hormones as well, not single floating point weights.
The model "parameters" for the DNA encoded part can't be much more than about a billion. Basing this on the size of genomes and their information content (they are often repetitive).
Why is a constant stream of ultra high resolution input from ~5 senses for years not count as a huge volume of data? Not to mention all the pre-training data encoded in the brains blueprint via DNA through natural selection? If anything, LLMs are more impressive than humans and animals in that they only have access to text to build a world model from a tabula rasa.
While the input resolution is indeed high, it's not as if any animal on earth has access to photographic memory of all that data. It is so ephemeral you could scarcely tell me what the last 10 people you passed on the street were wearing, even if it was 10 minutes ago.
At inference time, a human being has access to 5 to 9 variables at most an average human can hold in their head at any given time. But I can feed pages and pages to an LLM and it has a perfect representation of every single word and letter in its working memory. It also has access to a look up table of billions of statistical correlations for what the next word or block of words in any sentence should be given the perfect priors.
If I'm talking to you, especially in person, I won't be able to tell you what the second word of three sentences ago was. In fact, I might not have even been paying attention, because I'm already thinking at a higher level about what you're communicating to me and the word wasn't all that important. But a LLM certainly uses this information to guide its responses (along with its ginormous lookup table).
People have pointed out that LLMs like GPT 3 and 4 can draw pictures, but they've been "blind since birth" and have never seen anything. They've just read descriptions of things. How capable would a human be, if they had grown up deaf, blind, and in all other ways insensate except for some sort of braille-like reading input?
Another salient quote is from Andrej Karpathy (ex-Tesla AI team lead) who said that a camera is a high-bandwidth input "that puts many constraints on the world". Children learn from multi-modal inputs, and vision especially provides a large number of constraints that they can use to learn how the world works.
I have a two-year old, and something I've noticed is that infants have a very strong instinctive urge to gain agency over the world. They try very hard to control things, to make things move, to make sounds, to be able to affect things in all sorts of ways. This must be a very critical part of learning, because they'll kick and scream if you remove their agency. It's as strong an instinct as wanting to eat or sleep. No current LLM has any kind of feedback loop, and has essentially zero agency.
That might be a feature, not a bug, depending on what you mean by "agency". At the tippy top of that concept might be an AI system that decides by itself (as perhaps a weapons system) whether to kill human beings without direct instruction to do so. Yeah, Skynet. And every other movie where the AI "goes crazy".
We have access to video training data for driving, absolutely tons of it, as we've been attempting to train AI cars for more than two decades now. If GPT4 is what you say it is, we should be able to train it on that video data and solve autonomous driving. There is nothing inherently about transformers that prevents them from taking in video data. They've already been used by some researchers (https://arxiv.org/abs/2104.09224).
And yet, you can take a 16 year old who's never driven, and teach them within a week to be decent at it and maybe 50-100 hours of driving training and they're competent. You don't need to show them a billion man-hours of driving footage. Even the first people to buy cars in the late 1800s when they were first invented, were able to pick it up almost right away (there weren't even driving licenses back then).
At any rate, driving is just one example. Despite being one of the oldest futuristic sci-fi examples, I don't see restaurants powered by AI. I don't see housekeeping powered by it.
Ok, those are embodied examples, so you say they're unfair. Fine. What remote-friendly jobs are being swept away by AI? Can we even do customer service with AI right now? No, outside of some "front line" chat bot (which just replaces phone trees and terrible localized search engines), we can't. Even if a GPT is trained on a business's proprietary documentation, it's wrong or unresponsive enough that it would cost you more than it would save you by firing your customer support staff.
A 16 year old has a decade of vision input, knows tens of thousands of words, has a coherent theory-of-mind, and has self-preservation instincts honed over hundreds of millions of years of evolution.
The equivalent scenario would be take a fully general AGI(!) that already has a body and has learned to manipulate the physical world, put that in the car and have it learn to drive.
A lot of people seem confused about these scenarios. The currently popular LLMs are like a child raised in a black box, and are weirdly retarded in the same way you would expect a child raised in a black box to be.
Similarly, driving AIs don't speak English, can't take instructions, and are simultaneously learning physics, theory-of-mind, the rules of the rode, and signage conventions without having agency during most of their training.
> A 16 year old has a decade of vision input, knows tens of thousands of words, has a coherent theory-of-mind, and has self-preservation instincts honed over hundreds of millions of years of evolution.
No they don't. Where were you driving 23 days ago? Or, if you could categorize your driving data in your head, what did you pass, precisely, in the car while you were driving 96 hours ago? AI training data has all of this. It has perfect timestamps with 25-30 frames per second (or more in some cases, with multiple cameras feeding frames in to the dataset) with full 360 view in many cases, adding up to millions of man-hours of driving data from thousands of drivers, which is longer than any human lifespan. A lot of times this data is supplemented with IR representations or LIDAR representations of the environment as well, something a human can't even see.
> Similarly, driving AIs don't speak English, can't take instructions, and are simultaneously learning physics, theory-of-mind, the rules of the rode, and signage conventions without having agency during most of their training.
They're not learning that on their own. They're being poked and prodded by humans in the loop (and indeed, automated tagging tools) who have tweaked the models by adding numerical weights to "bad outcomes" and "good outcomes". Or "good categorizations" and "bad categorizations", or "aligned responses" and "unaligned responses". And because it's just a dumb statistical model, if you tell it the color blue is bad and that it should resist answering any questions related to the color blue, it will agree, because its entire heuristic model for organizing the world was designed by humans.
Similarly, it doesn't have a theory of mind. If it did, it would also have agency and it would already be AGI. Instead, it has access to examples of data of humans exhibiting theory of mind with other humans. And it's got enough parameters in its statistical model that it can accommodate a weighting for this "theory of mind" impact on what the expected output should be.
LLMs are nothing like an intelligent child raised in a black box. I mean, we already have examples of this, vis-a-vis being both blind and deaf: https://en.wikipedia.org/wiki/Helen_Keller.
I mean everyday vision input, which generalises to novel scenarios because driving happens in the same world with the same rules of physics and optics. Shadow and light, depth and movement perception, etc… can all be reused.
This is like how LLMs are difficult to train to the point that they understand English, but then relatively easy to specialise.
Similarly, students don’t start their education at University, they spend nearly two decades getting to that point by learning the prerequisites.
> Similarly, it doesn't have a theory of mind. If it did, it would also have agency
These are unrelated concepts. There is evidence that GPT 4 has a rudimentary model of mind but it isn’t conscious itself and is a static model with zero agency over the world.
> No they don't. Where were you driving 23 days ago? Or, if you could categorize your driving data in your head, what did you pass, precisely.
Just because I cannot do that, still my brain had this input, and was "trained", and we know the real neurons of the brain work completely different in all regards (connecitvities forming, activations, weighting) than our gross simplification of one activation function with a weight.. so not getting your arguments at all.
I have been trained on all these inputs, and even if I cannot recall them now they manifested in a my superior brain somehow.
Nice that in theory the AI training data has all this available, still the resulting model is much simpler, and also cannot recite all of its inputs seen, too??
The raw volume of the data from the human senses is high but a lot of it is low entropy. The high resolution video from our retinas of reading the pages of a book over many hours, and the ASCII representation of said text are equivalent, modulo information about the physical object of the book.
Definitely. I'm thinking LLMs are capturing real world structure of some kind that the brain also taps into except with a different algorithm and also stores it differently.
I like how Max Tegmark frames it by considering Birds and Flight and how we built Planes that can also fly. It turns out Flight, as an action on its own, is not so difficult if you throw enough energy behind a particularly shaped collection of parts. Intelligence, as a behavior observable in the world, appears to be similar in that it emerges in stages at scales as we are discussing.
And just like inventing airplanes caused us to say 'all things that fly are not birds', now we may start saying 'all things that display intelligence are not us'
(where intelligence is => human level intelligence and "us" could be swapped out with alive/sentient/conscious/etc at least for now and perhaps in all cases)
Human intelligence is social (and some animals' as well). The majority of your knowledge was distilled by the previous generations, not you personally.
In the culture (symbols, then?). For example, a human raised alone in complete wilderness is unlikely to invent any useful numeric representation like our current base-10; they wouldn't even have any use for it without any societal pressure.
Similarly, many animals also pass the knowledge, like certain birds spread the songs among the population.
Besides, individual human knowledge is really specialized. LLM is expected to know everything.
I think my ignorance is showing here, but that paper's tldr to me seems to be: neural network performance is not a monotonic function of network width. Like the conclusion and the problem statement seem to be trivially equivalent.
They admit that this law is only useful if you already know where the 'breaks' are: "If an additional break of sufficient sharpness happens at a scale that is sufficiently larger than the
maximum (along the x-axis) of the points used for fitting, there does not (currently) exist a way to
extrapolate the scaling behavior after that additional break."
Imo, intelligence is not about words at all. A words processor, such as LLM, is just a gate thru which AIs will communicate. Intelligence works more like this: you throw a pile of semi-organized data at it, results of experiments or observations, and it finds a pattern behind yhe data. How it communicates with us, via LLMs or by other means, is a secondary question. Speculating further, I think that finding that pattern doesn't necessarily look like computation. It may look like this: our mind projects datapoints into a higher dimensional space, where symmetries are more apparent, and finds the simplest configuration that approximates the datapoints. The approximation happens automatically by a physical process based on quantum effects.
130 comments
[ 3.6 ms ] story [ 207 ms ] threadHello engineering, my old friend. Nothing inspired more confidence than a set of random numbers cast into a formula and then whimsically called a fundamental "law".
"... has discovered empirically that the capability for moral self-correction emerges at around 22B parameters. For more scientific details, the paper is here. [https://arxiv.org/pdf/2302.07459.pdf]"
This would suggest that moral philosophy could be transformed - in much the same way that natural philosophy turned into physics.
I’m just a layperson and don’t really know what I’m talking about, but I find this all very exciting.
I don't think they're "dangerous" per se, I think metaphors matter and we should choose the best ones.
Why would something being subjective imply it is meaningless?
So we can actually look at concepts like “morality” and see how that is encoded. And I’m confident that this will give us empirical insight into these concepts in a way no philosophy has been able to do until now.
What AI might enable however is a superior form of democratic process, wherein an AI surveys the entire population through a natural language interface and synthesizes the nuanced and conflicting desires of the entire society. This deliberative democracy process would ameliorate the distorting effects of campaign funding and issues like uninformed or misinformed voters and low voter participation. It could also allow a sort of citizen feedback line-by-line on proposed legislation and government action.
1. Adenosine, i.e. the sleep molecule affects all outputs of dopamine including the two heterodimers of dopamine D1 and D2 that happen to be combined with Adenosine receptors A1 and A2 and also impacts the two optical computing components the eye itself as its bathed in dopamine and how we detect colored light and the other neuron component of the eye at the back of the human brain.
Okay, so in my ADHD research I got a bit focused on dopamine.
This part seems unsubstantiated. The author implies that next-token-prediction does not lead to reasoning and creativity "of an animal level". But arguably GPT-4 is already at that level (can your dog predict what a novel Python function would output when it is executed, or write a rap song about autoregressive training where every word starts with the letter "a"¹?).
Regardless of whether GPT-4 is there already, it would certainly be a risky bet to say that it is impossible for a GPT-5 to get to "animal level" (whatever that means) reasoning and creativity while still using next-token-prediction.
As the article points out in the next paragraphs, there is no obvious limitation to next-token-prediction, because making the best prediction requires fully understanding the words in the context, as well as their true meaning and the reason they were written.
Related to creativity, here's a Midjourney image resulting from my spouse prompting "a studious corn snake": https://chris.printf.net/a_studious_corn_snake.png
¹: via GPT-4:
==
(Verse 1) Autoregressive, arranging, assessing answers, Adjacent activations, astounding, acting as anchors, Associating abstracts, analyzing antecedents, Aptly approximating, articulating, all adjacent aspects
(Chorus) Astonishing achievements, astounding accolades, Anomalous arrangements, ascending always, Audacious algorithms, autoregressive amaze, Applying ample artistry, as alliteration ablaze
(Verse 2) Accelerating acumen, anonymous aspirations, Anecdotal adventures, allegorical animations, Applied apparatus, assertive applications, Astral alliterations, assembling aggregations
(Chorus)
(Outro) Amplified abilities, augmenting atmosphere Adapting, acclimating, always aiming austere Autoregressive training, ascending, advocates Awe-inspiring artistry, as alliteration activates
(No, I don't know why.)
E.g. here, when I say modular -- I mean that a "fix" can't be implemented in a local, small way. I need to try again with a new prompt, etc -- and it's not clear to me how to "fix" my prompt to get the right thing.
one way to support this sort of modularity might be with a GPT-4 / langagent integration which would query Midjourney until it gets a picture which satisfies my criteria, changing the prompt, etc.
The problem is that, if the integration or langagent has an issue -- how do I modularly fix those systems? And so on.
This sort of question "how do I figure out how to fix this prompt-based thing" makes me worry about a sort of "turtles all the way down" problem. Okay, so maybe I don't need to understand exactly how to use these systems to get what I want -- just let another LLM do that ... but what if that LLM has an issue ... and what if the LLM which is prompting that LLM has an issue ...
It concerns me. I'm not sure how well founded my concerns really are.
inpainting.
There really is no arguably about it. There isn't a verifiable definition/criteria for general intelligence that GPT-4 fails that a significant chunk of the human population doesn't also fail.
GPT-4 performs nearly all tasks it's given at at least the average human level, usually well above the average baseline. In some cases (nlp, analogical reasoning - https://arxiv.org/abs/2212.09196, and some others it's top %)
You basically have to make up your own definition that can't be substantiated (don't worry, plenty seem to do this). It almost always boils down "Well it's not "real" understanding, the difference is huge, i can't show you this supposed huge difference but trust me bro, it's not real"
yet, you will afraid to trust gpt-4 to file your taxes through installed browser plugin.
There is a huge gap between some narrow cherry picked (researchers have strong incentive to report mind boggling results) and diverse and multi modal real life situations
I don't need to cherry pick, I use GPT4 daily. At this point naysayers can only be willfully ignorant or uncreative in their use of it.
It's not possible for me to believe it is anything else.
you should try to work on your limitations..
==
Me: Is it true that, in humans, one neuron has an infinite number of possible states (as far as we know)?
GPT-4: It is not accurate to say that a single neuron in humans has an infinite number of possible states. Neurons are complex cells that can exhibit a wide range of behaviors and responses, but the number of states is not infinite.
Neurons communicate with each other through electrical and chemical signals, and their primary functional unit is the synapse, where communication between neurons occurs. The strength and patterns of these connections can vary, resulting in a large number of potential states for a single neuron. However, this number is still finite, albeit extremely large.
Moreover, the activity of a neuron can be described in terms of its membrane potential, which is a continuous variable ranging from hyperpolarized to depolarized states. While there are many possible values for the membrane potential, it is still a bounded range, not infinite.
So, while the number of possible states for a single neuron is vast and complex, it is not infinite. The complexity and diversity of neuronal states contribute to the rich repertoire of cognitive and behavioral functions that humans possess.
(Would it bother you to be corrected by a lookup table?)
https://phys.org/news/2014-01-discovery-quantum-vibrations-m...
Analog computers also don't have infinite states, because of the noise floor.
A synapse can encode at most some low tens of states[1], synapses are discrete, spiking is discrete, receptor density is discrete, each neurotransmitter release is made up of discrete number of vesicles with a discrete number of molecules, even epigenetic expression is discrete. That it's not all perfectly algorithmically executed is not key to the function being performed – and anyway, at the limits of high-performance compute we also start to deal with esoteric processes, and ways of suppressing them. We didn't design the brain nor do we have perfect ways of observing it in vivo, so we can't always neatly distinguish the wheat and the chaff; but this cannot be seriously taken as a cause to think it's all wheat. Non-human animals show that it's not magic either; even apes, with brains architecturally almost (not wholly) identical to ours but smaller, cannot learn any of the hard symbol-manipulating stuff that we or later GPTs can and that we colloquially refer to as intelligent behavior and reward economically. This, I think, is a very strong prior for that sort of intelligence not being dependent on some low-level expressivity of biological neural computational substrate.
Penrose-Hamerroff's "quantum effects in microtubules" thesis is crank science not supported by any direct evidence.
1: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5247597/
But you seem to be suggesting that all animal brains are equally powerful, because they have infinite neuron states, quantum superpositioned neurons, etc.
As far as I can tell, you seem to be suggesting that my old cat’s brain has the same complexity as my brain - right?
So, I’m interested in understanding why my cat is demonstrably stupider than I am, despite having infinite complexity, and why whatevrr you come up with doesn’t also apply to LLM size.
But of course it does imply that. It's vast but necessarily finite number of "brain states" however that's defined, which is itself a subset of the vast but also finite number of states of the finite number of component particles.
https://en.wikipedia.org/wiki/Bekenstein_bound
Neurons are not of unbounded size nor energy.
The size of the earth's biosphere
The size of the largest animal.
The size of a human skull cavity.
The size of a human skull cavity, divided by the number of neurons therein ( a few Billion).
Is this not obvious? There's nothing futile about stating these obvious facts.
Do not fool around with "if I move a potassium ion farther away, isn't that a different bound" ? correct, but moving the bound slightly, is irrelevant to the obvious statement that there are still bounds, and they are practical.
Stating that the neuron is not well-defined is similar irrelevance, if you want to precisely measure the bound, you will need to know precisely what is bounded, but I did not, merely stated the obvious truth that every such definition will come with a bound. One can say "definitely smaller than this" in a wide range of ways, without defining the exact edge.
What's the alternate argument: "I won't define it, therefor it might be larger than the solar system". This is not a serious thing to say.
And to think that such bounded systems "have infinite states would thereby be capable of storing infinite information" is pseudo-mystical foolery. Large, or even "vast", is not infinite. And as sibling comment has mentioned, there is a "noise floor" below which analog resolution means nothing.
That's the minimum. It's far above average for most things.
>Maybe, because quite a lot of people aren’t very smart.
Lol okay..and ?
>But even GPT4 struggles with complex logic problems
So does everyone else
>and when it makes logic errors it makes ones a human would spot
people spot mistakes they're capable of making all the time
and so can GPT-4 - https://arxiv.org/abs/2303.11366 https://twitter.com/johnjnay/status/1639362071807549446
>It also has trouble counting letters and words, though that’s largely due to its autoregressive nature.
It sees tokens not words or letters and there aren't many instances of counting occurring in text so it doesn't get learnt.
>and I really think it’s emergent, but at its core it doesn’t seem very “intelligent”
and we're back to the trust me bro argument. It's not intelligent because you say it isn't not because it doesn't perform a great many tasks intelligently. if you say so
The usual broad indicia of intelligence (which, e.g., IQ is considered as a useful proxy measure for in humans because it predicts/correlates with them) are lifelong success in a wide variety of tasks GPT-4 can’t even begin to approach.
Right. and what are these tasks and why is intelligence the barrier? (meaning don't tell me something like driving, blind people are not unintelligent)
This sounds quite similar to Bing famously explaining to a user that 2022 is in the future while we are in 2023.
It's something you would expect maybe a child to do (it reminds of "Patrick Star's Wallet" meme), but to me that clearly fails the notion of intelligence in a very fundamental way. What would be the counter-argument?
Sentences look something like this to the model, [754 76 4752 67 456]
If I try to reformulate your answer, this is like words are a different domain than what GPT has been trained on (which is tokens), and so it is not unexpected that GPT cannot generalize to this other domain.
Is that a correct interpretation of your statement?
In my view failing to see the similarity in the relationship between class objects is a sign of a lack of 'intelligence', whatever that may be in a very precise sense.
Ok but not all relationship similarities transfer across representations like that.
It's like you give an instruction in English, "Find a word that rhymes with ingenious" to a french man. The french man sees, "Trouver un mot qui rime avec ingénieux". Guess what ? His response to "Trouver un mot qui rime avec ingénieux" would not fulfill the request in english that it might in French.
Ingenious and heterogeneous rhyme. ingénieux and hétérogène have no such property
The french man is not unintelligent to be unable to fulfill your request.
The systems are very impressive and I think there is a 'there' there, they're clearly intelligent to some significant degree. But it's incorrect to say that they pass every test.
It scored 3/4, incorrectly answering "the pouch" instead of "the baby kangaroo" for Question 2 of Schema 1. And now that I look at the question, one could easily imagine the kangaroo being afraid to jump into a pouch that's too large! In hindsight, if a human complained to me that this question was ambiguous, I'd have a hard time disputing them.
Granted these are just two examples, but they aren't cherry-picked. Moral of the story: before one makes a strong claim such as "there isn't any evidence a model can do X", it's often worth making a minimal effort to investigate one's own claim.
========================================
Winograd Schema 1:
Sentence 1: The baby kangaroo was afraid to jump into the pouch because it was too small. Question 1: What was too small? (Correct answer: the pouch)
Sentence 2: The baby kangaroo was afraid to jump into the pouch because it was too large. Question 2: What was too large? (Correct answer: the baby kangaroo)
Winograd Schema 2:
Sentence 1: The gardener couldn't grow the flowers in the greenhouse because it was too hot. Question 1: What was too hot? (Correct answer: the greenhouse)
Sentence 2: The gardener couldn't grow the flowers in the greenhouse because they were too delicate. Question 2: What were too delicate? (Correct answer: the flowers)
Do you think this was a strong refutation ? I just flipped a coin for fun and also got 3/4 correct.
I understand your point, but I mean, something that's intelligent really should've got the second question in that test correct.
Summing up, out of your four (not an acceptable sample size to form evidence) attempts to subject a SOTA LLM to Winograd schemas, two of them were actually Winograd schemas and only one of those got answered correctly. That isn't evidence that LLMs can pass novel Winograd schemas. If you actually come across such evidence, you should post it publicly on at least a blog page if you can't access the infrastructure to publish the results in a paper, even though the finding would certainly be novel enough to deserve a paper. As I said: there is no evidence that LLMs can pass novel Winograd schemas as of yet. We could get said evidence tomorrow, but it doesn't exist right now.
I would put a condescending moral of the story here, but I don't feel any particular need to belittle you, and instead hope you just learn from the information as presented.
Interesting! So there is published evidence, that it cannot pass novel Winograd schemas? Could you please provide me with the sources? I would like to have a look at the prompts. From my experience you can make GPT-4 pass novel Winograd Tests by giving GPT-4 additional context.
>So there is published evidence, that it cannot pass novel Winograd schemas?
There is no published evidence that they can, which is what I said.
>Could you please provide me with the sources?
I cannot prove a negative to you with a source. You will have to review the literature like I did.
>From my experience you can make GPT-4 pass novel Winograd Tests by giving GPT-4 additional context.
This is why I am pretty sure you're trolling. This is a lot like saying "You can make GPT-4 do prime factorisation of a semiprime by noting the constituent primes." The point of testing the model is to see if the model can give you the answer, not to give it the answer. The point of Winograd schemas as a test is that they test the ability to attribute pronouns using common sense, without additional context. The reason it's a good test is that every competent human being can pass the test, and as of yet no computer can.
> This is why I am pretty sure you're trolling. This is a lot like saying "You can make GPT-4 do prime factorisation of a semiprime by noting the constituent primes." The point of testing the model is to see if the model can give you the answer, not to give it the answer.
It is not "giving the answer to the model" if you adjust the prompt with additional instructions like "this is a test question" or "use common sense". It is about creating the correct frame of context. With these kind of adjustments GPT-4 passes every novel Winograd scheme I have presented it with.
It seems as if animals and humans can learn on far less training data and on a much dirtier and noisier data set. The data an animal or a human trains on has not been curated to remove misleading data or noise.
It seems that one can produce models with amazing capabilities by training them on vast curated data sets, but I get the sense real brains still have a lot of tricks that we don’t understand.
A few more breakthroughs every few years (not guaranteed) could mean better results with 1/100th the data.
The numbers involved are also an order magnitude greater. In a human brain, there are approximately 60 trillion connections between neurons. Made with a variety neurotransmitters and regulated by a host hormones as well, not single floating point weights.
If I'm talking to you, especially in person, I won't be able to tell you what the second word of three sentences ago was. In fact, I might not have even been paying attention, because I'm already thinking at a higher level about what you're communicating to me and the word wasn't all that important. But a LLM certainly uses this information to guide its responses (along with its ginormous lookup table).
Another salient quote is from Andrej Karpathy (ex-Tesla AI team lead) who said that a camera is a high-bandwidth input "that puts many constraints on the world". Children learn from multi-modal inputs, and vision especially provides a large number of constraints that they can use to learn how the world works.
I have a two-year old, and something I've noticed is that infants have a very strong instinctive urge to gain agency over the world. They try very hard to control things, to make things move, to make sounds, to be able to affect things in all sorts of ways. This must be a very critical part of learning, because they'll kick and scream if you remove their agency. It's as strong an instinct as wanting to eat or sleep. No current LLM has any kind of feedback loop, and has essentially zero agency.
And yet, you can take a 16 year old who's never driven, and teach them within a week to be decent at it and maybe 50-100 hours of driving training and they're competent. You don't need to show them a billion man-hours of driving footage. Even the first people to buy cars in the late 1800s when they were first invented, were able to pick it up almost right away (there weren't even driving licenses back then).
At any rate, driving is just one example. Despite being one of the oldest futuristic sci-fi examples, I don't see restaurants powered by AI. I don't see housekeeping powered by it.
Ok, those are embodied examples, so you say they're unfair. Fine. What remote-friendly jobs are being swept away by AI? Can we even do customer service with AI right now? No, outside of some "front line" chat bot (which just replaces phone trees and terrible localized search engines), we can't. Even if a GPT is trained on a business's proprietary documentation, it's wrong or unresponsive enough that it would cost you more than it would save you by firing your customer support staff.
A 16 year old has a decade of vision input, knows tens of thousands of words, has a coherent theory-of-mind, and has self-preservation instincts honed over hundreds of millions of years of evolution.
The equivalent scenario would be take a fully general AGI(!) that already has a body and has learned to manipulate the physical world, put that in the car and have it learn to drive.
A lot of people seem confused about these scenarios. The currently popular LLMs are like a child raised in a black box, and are weirdly retarded in the same way you would expect a child raised in a black box to be.
Similarly, driving AIs don't speak English, can't take instructions, and are simultaneously learning physics, theory-of-mind, the rules of the rode, and signage conventions without having agency during most of their training.
No they don't. Where were you driving 23 days ago? Or, if you could categorize your driving data in your head, what did you pass, precisely, in the car while you were driving 96 hours ago? AI training data has all of this. It has perfect timestamps with 25-30 frames per second (or more in some cases, with multiple cameras feeding frames in to the dataset) with full 360 view in many cases, adding up to millions of man-hours of driving data from thousands of drivers, which is longer than any human lifespan. A lot of times this data is supplemented with IR representations or LIDAR representations of the environment as well, something a human can't even see.
> Similarly, driving AIs don't speak English, can't take instructions, and are simultaneously learning physics, theory-of-mind, the rules of the rode, and signage conventions without having agency during most of their training.
They're not learning that on their own. They're being poked and prodded by humans in the loop (and indeed, automated tagging tools) who have tweaked the models by adding numerical weights to "bad outcomes" and "good outcomes". Or "good categorizations" and "bad categorizations", or "aligned responses" and "unaligned responses". And because it's just a dumb statistical model, if you tell it the color blue is bad and that it should resist answering any questions related to the color blue, it will agree, because its entire heuristic model for organizing the world was designed by humans.
Similarly, it doesn't have a theory of mind. If it did, it would also have agency and it would already be AGI. Instead, it has access to examples of data of humans exhibiting theory of mind with other humans. And it's got enough parameters in its statistical model that it can accommodate a weighting for this "theory of mind" impact on what the expected output should be.
LLMs are nothing like an intelligent child raised in a black box. I mean, we already have examples of this, vis-a-vis being both blind and deaf: https://en.wikipedia.org/wiki/Helen_Keller.
This is like how LLMs are difficult to train to the point that they understand English, but then relatively easy to specialise.
Similarly, students don’t start their education at University, they spend nearly two decades getting to that point by learning the prerequisites.
> Similarly, it doesn't have a theory of mind. If it did, it would also have agency
These are unrelated concepts. There is evidence that GPT 4 has a rudimentary model of mind but it isn’t conscious itself and is a static model with zero agency over the world.
Just because I cannot do that, still my brain had this input, and was "trained", and we know the real neurons of the brain work completely different in all regards (connecitvities forming, activations, weighting) than our gross simplification of one activation function with a weight.. so not getting your arguments at all.
I have been trained on all these inputs, and even if I cannot recall them now they manifested in a my superior brain somehow.
Nice that in theory the AI training data has all this available, still the resulting model is much simpler, and also cannot recite all of its inputs seen, too??
The raw volume of the data from the human senses is high but a lot of it is low entropy. The high resolution video from our retinas of reading the pages of a book over many hours, and the ASCII representation of said text are equivalent, modulo information about the physical object of the book.
And just like inventing airplanes caused us to say 'all things that fly are not birds', now we may start saying 'all things that display intelligence are not us'
(where intelligence is => human level intelligence and "us" could be swapped out with alive/sentient/conscious/etc at least for now and perhaps in all cases)
Similarly, many animals also pass the knowledge, like certain birds spread the songs among the population.
Besides, individual human knowledge is really specialized. LLM is expected to know everything.
They admit that this law is only useful if you already know where the 'breaks' are: "If an additional break of sufficient sharpness happens at a scale that is sufficiently larger than the maximum (along the x-axis) of the points used for fitting, there does not (currently) exist a way to extrapolate the scaling behavior after that additional break."
This clickbait in abstracts will hopefully, eventually, have the same effect as clickbait everywhere: people skip reading it.