If anyone needed to be convinced that LLMs are not accurate models of the human brain they need look no further than these numbers. The smallest model under discussion requires "more books than are in the Kindle store on Amazon U.S." Humans obviously acquire language proficiency with a lot less input than that.
True but the machinery (is the brain) that enables humans to acquire/process/understand that knowledge is a lot more complex and went through years of environmental conditioning?
Yes, but that environmental conditioning did not involve training on the contents of the Kindle library. And the results of that training are encoded in under a gigabyte.
For the record, I'm not saying LLMs are not a huge step forward. They are. But they are not -- and cannot be -- the whole answer.
No totally. Not an ml guy by long shot. My understanding is llms are foundational models which iiuc are "base" models representing all human knowledge until some point in time. Now I'd argue that you don't have to count all of evolution but just the point from when you were born. I'd say the all the sensory experiences + knowledge input from the day a brain is activated (DOB) would be in the order of PBytes if not more?
One argument (that I don't necessarily buy) is that inputs from sensory perception play a major role in building a language model in humans. A human might not read TB of books to learn language, but if you put them in a sensory deprivation tank that only displayed a massive series of Unicode characters it would take forever for us to learn language. If we did the same for an ML model, perhaps those non-language training materials would help.
(That said, DL architectures are obviously wildly different from how the human brain works. E.g. backprop is physically impossible.)
Yep totally agreed. One of the things I'm excited to see going forward is multimodal models (trained e.g. on text + video + audio + images).
I'm sure there's a lot more to it than this, but maybe one factor that makes humans a lot more data efficient is the multimodal input we receive.
If that's the case, imagine how much better things could get when we train with all the videos, podcasts, radio etc in the world, in addition to all the text out there!
1. You're mischaracterizing the parent's comment, to such a degree that it seems willful.
2. This tendency to police other people's behavior, telling them what they should or should not do, does not help your cause. It puts people on the defensive. No one, especially adults, like to be told what, or what not, to do unless it's coming from a well-qualified lawyer.
I can't believe you are serious. It is pretty clear what the context is here and I didn't mince my words. You are policing MY behavior and you should stop telling people how to be anti-ableist.
Agreed, i think the "telling other people what to do" thing is something i haven't seen brought up enough. Seems like the whole current social media is built on defending some group by telling other people that they're bad people for speaking the way they always have and that they must change themselves to fit your opinions, otherwise they're forever doomed to be "bad people". The bad person part is implicit in the command. When did people get so comfortable commanding others and demeaning their thoughts and agency?
Wasn’t she ‘boot-strapped’ by having had both for the first 19 months of her life? - Presumably this gave her an internal model of the world on to which she could map touch perceptions and language concepts.
It's all a combination of both explicit (traditional software that's faster/has better specific accuracy but is brittle) and implicit (machine learning/etc where fuzziness is better) optimized hardware (wetware?)/software all up and down the biological stack, so to speak.
Nature is good reference point but it's not by any means optimal. One example may be human eye - we can make cameras orders of magnitude better than eyes created by evolution.
You can look at it the other way around - dopamine and other neuro transmitters as poor approximation of backpropagation. It has many flaws for example tight harmful loops ie. addictions.
Majority of brain work is ignoring irrelevant information (attention) and small scale hallucinations (we don't see world as is but slightly hallucinated to keep it stable - ie. they way brain processes blinking <<turns off>>, you can peek at those nuances with ie. optical illusions etc).
One of missing bits in neural nets may be reusing its output as input (embedded in inference itself, not poor mans re-prompting).
Once it's sorted out I'd argue the performance will skyrocket and give opportunity to massive optimisations ie. embedding things like known functions - imagine brain which has known, very narrowed, available functions at its disposal - all mathematical functions on numbers, logic, optimal sorting etc. Imagine if as thinking human you'd have access to accurate functions - the sky is a limit.
> all mathematical functions on numbers, logic, optimal sorting etc
Bayes formula as a built in primitive. You don't need to know much of statistics to see how limited humans are at processing information because estimating posterior updates is so expensive for them.
Thinking in terms of raw probabilities would be very alien to most humans, but could easily be technically superior for making plans.
“Nature is good reference point but it's not by any means optimal. One example may be human eye - we can make cameras orders of magnitude better than eyes created by evolution.”
This is only true when considering single performance axis like pixel resolution. When you consider the corpus of power efficiency, jitter resolution enhancement, dynamic contrast, performance per volume, etc. We aren’t close to building something as capable.
Having a model of the world (predominantly from vision, but also hearing and touch) surely does wonders for language acquisition as it provides the learner a backdrop against which to infer the meaning of speech. When an infant hears “look, a dog” and “look, a cat” in two separate instances, their eyes alone provide enough information to infer, at least at a high level, the meanings of “look, a”, “dog”, and “cat”.
It's pretty clear that humans, unlike LLMs, use external sensory data (the only external data we have, when you cut through it all) when they produce speech, as evidenced by the fact that they don't speak falsehoods that don't mesh with their internal data model. LLMs have such a weak model of reality -- it's whatever “sounds right” -- that they speak falsehoods all the time. The only way to give an LLM sensory data would be to encode every sensory experience people have into text.
> It’s pretty clear that humans, unlike LLMs, use external sensory data […] when they produce speech, as evidenced by the fact that they don’t speak falsehoods that don’t mesh with their internal data model.
I don’t have access to any other humans internal data model, but the indirect evidence I do have suggests that they do, in fact, speak falsehoods that don’t mesh with their internal data model for a variety of strategic purposes.
Fair point, but accurate versus efficient are different. While humans do acquire language much more efficiently, these models "know" more than a human about most things - in that if you asked it to describe each of the 100 most popular subjects or books it would have no trouble doing so (in a dozen languages no less). So I don't think it's quite an apples to apples comparison.
Twitter is a madhouse, but as for HN, the closest I've seen anyone come to that here is claiming that the difference in underlying structure can/does lead to emergence of the same phenomena.
Not as strong an argument as you think as you're forgetting humans have 5 senses, are embodied, and millions of years of tuned weights from natural evolution codified into our brain structure.
Brain is nothing like attention networks at least in terms of dynamics and individual neuron complexity, my wild ass guess; there is not enough space in DNA to store even 10% of brain connections.
Cluster density, cluster location, eagerness to connect, distribution of cell types, etc. are all coded in DNA and contribute to creating "initial weights". Neuron "weights" need not be stored completely uncompressed.
Obviously newborns need to develop and take in stimulus before they "know" anything, so the initial conditions are not sufficient, but they are obviously necessary to make the limited learning useful.
It always seems naive to compare neural structures and functions to artificial networks, dynamics are simply not there, even single neural would require multi layered network (if not mistaken 7 layer network, and very long training time), it is fascinating how brain self-(re)organizes through out life time from small number of examples here and there.
As far my understanding goes to very large degree it is unknown how to model such dynamics, where it would be possible to start with no spiking neurons and evolve effective/stable learning behavior from small number of examples/experiences.
I accept this possibility, but because we don't understand, seems wrong to assume any other path, furthermore, there is some evidence that intelligence/cognition is convergent in neural structure and function, in addition to that most ANN are inspired from studying real neurons.
"Cluster density, cluster location, eagerness to connect, distribution of cell types, etc. are all coded in DNA and contribute to creating "initial weights""
Are they coded in DNA, or do epigenetic factors, the environment cells grow in, adjacent cells, their interaction, etc, play a role here as well?
That would depend on what you mean by "connection". The human genome has approximately 200GB of data in it. And this data is highly compressed as it takes 3 different layers of decompression before it gets to usable form.
Also, since you asked, I would like to mention protein interactions with cellular components and tissues. These can be argued as just proteins interacting with each other but the extreme complexity of these higher levels and the unique phenotypes arising from them make me think they are deserved to be treated as another layer of "data decompression".
A fruit fly has no meaningful learned intelligence yet is more capable than any trained neural net. It’s just a small bundle of nerves laid out according to the model encoded in their DNA. It was trained through millions of years of genetic algorithms, and is fully encoded in the DNA, along with the entire ability to specify the entire physical structure of its body. Why is it surprising our brain has instinctual ability encoded in its structure stemming from our DNA, evolved over the history of life?
Not sure about the 10% point. You could imagine that we got “lucky” and a handful of simple proteins manage to produce the brain (or at least enough of the brain’s connections that the rest can be filled in during infancy via sensory input). Or put another way, maybe evolution optimized for brains that are incredibly adept while only taking a small number of codons (relative to the number of neurons) to describe. The fact that we've managed to l describe the universe through a small set of laws shows that the laws of the universe require much less space to encode than the universe itself, so it's theoretically possible.
We have managed to describe a small subset of the universe through a small set of laws. The current incompatibility of gravity and QM and the resulting dark energy/matter explanation means we can’t see or interact with most of the universe.
Right. The nature trained all the nervous systems with genetic algorithms for millennia. Individuals rather fine tuned their own with natural noise and dropoffs and natural feedback. Fine tuning our brains are rather expensive.
There are a few innate reflexes but you lose them quickly and they are pretty basic. Our brains have way way more neurons/parameters than these models and are still not overfitting... the issue lies in the learning method.
It's not just about reflexes though I would hardly classify reflexes as "lost quickly". That's just not true.
It's deeper than that. Our brains are optimized for a lot general human functions. Learning language is one of them. There's a whole section of the brain dedicated to it and other things.
There's also a lot of vital biological information encoded in DNA/RNA.
We are not even close to starting from scratch.
True, but humans have a 4B element text book downloaded into each cell and some of that stuff encodes for a machine that already has a lot of language proficiency built in from day #1. I've always wondered how much of that genetic code is 'soft' in the sense that it pre-sets certain bits in the brains that it generates. Sort of a boot-strap ROM for a human.
That argument is utterly unconvincing, as you've conveniently left out the millions of years of "training" and "fine tuning" that has occurred through the evolutionary process.
It could be argued based on the extreme difficulty we're having in teaching animals human language, even primates. We can create a semiotic system and get a chimpanzee to communicate through it, but it won't learn English no matter how hard we try.
I think that's generally accepted to be that humans have a very large language center in the brain; IE: and inductive bias towards learning language (and I think there is something about our mouth and tongue being able to make specific sounds?)
It doesn't say anything about that knowledge being precoded.
Isn't a language center by itself precoded human toward the acquisition of human language (which has co-evolved to make use of our language center)? I don't understand how it could be any other way.
Your argument is utterly unconvincing too, a human left alone with no parents or way of learning can basically do nothing at all, i.e. what we start with isn't some trained neural net at all, it's an empty framework with no data. The OP is correct in that we are several orders of magnitude more efficient at learning than ANNs.
The fact that human brains are capable of learning things that animal brains cannot, is a clear indicator that it is already "trained" to a significant degree.
Human "learning" seems to be much more analogous to "fine tuning and memory storage/retrieval", than actual training.
Why can't a macaque teach a calculus class? Why can't its "empty" brain be taught to do something like that?
Try raising a child with no inputs and see what it ends up doing. We don’t start “trained” at all. We just start with a neutral network architecture with no data. The difference between us and other animals is the number of neurons and the network complexity, but it doesn’t start trained to any degree. Humans do not start with any ability to talk, reason, identify objects etc.
If you mean a lot of accuracy, that's obvious and doesn't really need argument. And this new fact doesn't change the argument.
If you mean a more moderate amount of accuracy, this isn't proof either way. Human brains take in less text but they put a lot more processing into it.
Yet we learning from 18 hours of ultra high resolution video per day (x2), along with 18 hours of reinforcement learning, along with 18 hours of audio and 18 hours of nerve stimulus data.
This is assuming we are learning nothing during sleep, which probably isn't true.
By the time a person is 21 years old, they have been trained on at least 1 petabyte of data.
By two years old, about 125TB of data. It makes LLMs look quite good in comparison.
By no means rigorous, an estimate for the number of neurons is around 10^10 [0] with an average number of connections estimated at around 8000 [1].
Call 10^10 \approx 2^40 for convenience, and 8000 \approx 2^13, which gives us a 2^53 entropy estimate, or about a petabyte of information as an estimate of what the human brain can store (discounting more exotic theories of memory stored in DNA or some such).
Their communication is much more complex too, and neither they nor their connections are static.
We don't understand how they work at the subatomic level simply because human understanding of the subatomic world is not complete, but even just at the atomic level a single neuron is massively more complex than anything humans have created.
Going up to the molecular level, even that is staggeringly more complex than the incredibly simple abstractions that make up a neural net.
Is what happens in the brain at the molecular, atomic, or subatomic levels relevant or necessary to intelligence and consciousness? We just don't know yet, but we do know all of that is far more complex and very different from the simple abstractions that are used for neural nets and LLMs.
The back of a napkin calculations in this thread don't even begin to do justice to the tremendous amount of "calculation" or "storage" that happens in the human brain.
And by far most of that data is completely meaningless. You just can't naïvely compare that with LLMs where all the data is curated and having meaningful content.
The majority of the data is a web crawl. It's not curated in the slightest and includes the whole of reddit.
In contrast the data we collect through our nervous system is rich and meaningful and far deeper than just raw text. We can even manipulate the environment as we learn to facilitate faster learning eg. pick up a ball and throw it, rather than just watch videos of balls being thrown.
The web crawling is curation, as even the most mundane entries into the dataset will still have a significant weight on the model, even if it's small. What separates the brain from any statistics-driven model is how well it swims through an ocean of information to find the relevant details, tossing out what's left, and filling in gaps with intuition. Sure, its messy process, and mistakes get made, but that's in stark contrast to the very trim way that ML models run.
> Yet we learning from 18 hours of ultra high resolution video per day (x2)
Our object recognition is trained pretty quickly. And we sleep (certainly as a child) more than 6 hours per day. But we don't learn much from just looking at pictures.
> 18 hours of reinforcement learning
You made that up.
> It makes LLMs look quite good in comparison.
So, exposing an LLM to a lot of video will make them understand language?
The poster above for some reason though encoded video is a good measure, which does not make sense actually. Brain gets raw video. With raw video it is about 100PB per year, > 200TB per day.
Even sound alone (uncompressed CD quality stereo) is 3TB/year.
Most of that data is the same all day e.g vision. There’s very little difference between frames.
The significant thing is the reinforcement. Without it you have to resort to the openai brainlet style 10tb of text training.
If their model had reinforcement built in then you could just plop it in front of a person or on reddit and it would rapidly self-learn on a fraction of the data. Their training model relies on it learning purely based on observations rather than interaction which is inefficient.
Why do they have to be accurate models? They just have to be possible to construct and have to perform. The construction method is irrelevant beyond feasibility.
I don't care if my model needs an exabyte of RAM if I can just go and buy that much RAM one day.
They aren't accurate models of the human brain just as airplanes are not flapping their wings and cars don't walk on legs. Our brain's way of achieving intelligence is not the only one out there, even if LLMs aren't there yet.
Yes, we use now magnitudes more oil than we used to 120 years ago. But back then oil was used for cooking and lamps, while now it is used for so many more things. Same goes for data. It is the new oil :).
I’d note that the models don’t just acquire language facility, but the ability to use language with alacrity about almost any subject. There’s no human alive that is remotely as capable, and most I’ve met hallucinate more.
While I agree with the conclusion, I don't think your argument actually supports it. Humans for the most part don't acquire language through books until they're mostly already fluent in their native language.
These LLMs are also trained on arbitrary books. When acquiring new languages, we use educational materials that are specifically created to facilitate an understanding of language. Not just arbitrary books in random order.
I'm curious how they keep LLM generated text from turning into future training input, and creating a loop that probably isn't good for quality. Or is that not a problem?
That's a problem for machine learning systems in general.
For instance, a recommendation system's output effects what users see, so it effects what they click on. The next training set is statistically dependent on the input of the previous.
There are strategies to deal with it, but in the case of a LLM it seems difficult apart from downweighing everything after 2023 that isn't from a vetted source.
OpenAI was talking about some kind of steganography so that previous outputs can be excluded from the training data. Not sure what's the progress on that.
That might protect them from their own generated content, but eventually other LLMs will catch up. Using some other undisclosed steganography they can't detect.
That makes sense only if LLMs will remain sub-human level for a very long time. But probably in 2-3 years generated content will be better than 99% of the human content.
The RLHF reward model is effectively a good/bad content filter. Human==good AI==bad doesn't always hold, there is good AI generated content and bad human written content out there, we should filter by content quality and novelty, not origin.
If I had the resources I would turn GPT-4 onto its own dataset to mine all the facts from all the source documents. Then flip the index grouping by fact, and for each fact it should write a page listing the supporting and contradicting evidence. This would be super useful in determining - 1. existence of a fact, 2. its controversiality level and 3. the distribution of answers.
With these pieces of information at hand GPT would get much better, it would at least stop hallucinating. This fact index would probably double the size of the training corpus as well. Google, with its years of search logs, is the best positioned to create this fact index. They already had simpler attempts at creating a knowledge graph for a long time. But if they don't move to exploit the data they sit on, others will take the lead anyway. It costs only money to run so much text over the model, no slow & expensive human involvement necessary.
I suspect this is a large reason why OpenAI trains GPT-4 using a dataset that is mostly clamped at September 2021.
It's almost certainly a problem for LLM development, just like it is for humans. Humans generate all sorts of stuff, some of it being absolute bullshit, and it does seem to cause problems for other humans. Yet with effort it still seems to be possible to cut through the bullshit in a lot of cases, so it's likely not an insurmountable problem for LLM development either.
It makes me wonder if long-neglected archives of old newspapers, internal corporate documentation, and government publications could have newfound economic value as training material. I know that the absolute volume is small compared to e.g. large scale document scraping from the Web, but intuitively I would guess that one old Bureau of Standards report written with a high degree of literacy has more value than 1000 SEO-chasing "how to make pancakes" guides. A lot of old documents were never digitized simply because people didn't think they had value. Is it time for a reinvigorated Google Books with a wider mission?
"A lot of old documents were never digitized simply because people didn't think they had value. Is it time for a reinvigorated Google Books with a wider mission?"
We have to be very careful with the "factual" content of old non-fiction works. So much of that, from history, to medicine, to biology, etc, turned out to be straight from their writers' imaginations. If an LLM considers such books to be no different from modern books on these subjects it would get a very skewed view of the world.
Imagine asking an LLM for a medical diagnosis and it responding with something about the humors.
That's a good point. I wonder how LLMs currently deal with the passage of time, if they can at all. Do they know that Satya Nadella is the current CEO of Microsoft and that Steve Ballmer no longer is simply because there are more training documents reflecting the present state of things, or is there an explicit time component that helps to resolve conflicting facts from different years? Based on what I've read so far about LLMs (and not actually working on any of these models myself), I wouldn't have thought they model time or facts in a way that you could expect them to resolve conflicting claims from a 2010 document and a 2020 document. Yet they're unreasonably effective at question answering anyway.
Something I've been thinking a lot about lately is the potential historical usefulness of LLMs which have been selectively trained on material from before a certain date, presumably resulting in a model exhibiting attitudes of the time in question. It would be extremely interesting to be able to ask for a 1950s take on a modern idea.
For this, large amounts of very old text would be extremely valuable.
I suppose the relative low value for low volumes of text makes this not a problem, but...
What about data that's sitting around and isn't supposed to be public? If training data gets scarce, does a market for small-medium sized data emerge? Like old homework papers, internal company documents, etc?
After those 33TB get tokenized and the tokens are encoded with basic frequency coding, I bet that much less than 1TB is left. This makes me think that LLMs are essentially LZW compression on steroids where text is indexed by meaning. It allows to query the data by meaning, but the catch is that every query needs to run a sort of matrix transform on the entire dataset (even though it's reoresented in a compressed form on the LLM weights).
Edit: Still, the idea of mapping symbols and words to a many-dimensional space of meanings is a great insight into how mind works. In that space, symbols with similar meaning appear next to each other, and a thought looks like a smooth but intricate shape that separates all symbols into the "insiders" and "outsiders" that, in practice, divide symbols into true/false, good/bad and so on. Such smooth intricate shapes appear in the frequency domain as a bunch of rational numbers, and that's the boundary of what a mind can imagine.
Thoughts to minds is what songs to birds: a way to communicate a relationships between observable things. Such relationships get reduced to a set of things "inside" and everything else that's outside. The boundary must be a smooth shape, that could be arbitrarily precise if the physical medium allowed it. A mind receiving a thought sees it as a finite set of resonant frequencies, even if the thought is more complex than that. Furthermore, the ultimate receiver of the thought is brain with its finite number of neurons, and even though different frequencies get mapped to different neurons, each neuron reduces amplitudes even further by activating a chemical reaction only when its input exceeds a certain threshold. Thus a continuous amplitude gets reduced to a sequnce of 0s and 1s, or a rational number. Hence the thoughts thinkable by brain are "rational". In their own realm thoughts are more like birdsongs with infinitely many harmonics and precise amplitudes.
That’s basically the claim of the “Blurry JPEG of the Web” article. I agree that to a large degree that’s probably what’s going on here. But it’s not all there is to it. Large language models apparently can build some world models. See Othello-GPT for a proof of principle.
In other words, once you “compress” to thoughts, there actually can be some degree of actual reasoning on “decompression”.
Has any one written the obvious reply to this paper yet?
It's false. It's not true.
The rules and the board state it "learns" are just functions of the token positions *by construction*; it is given the token positions. obviously it learns patterns in those positions.
The paper is such a misfire, it's absurd.
The problem with using a formal system to investigate this issue is that by construction the "distributional hypothesis" is true for that system. Ie., the pattern in distribution of the moves are (a very good model of) the rules and board-state.
But this is clearly untrue for non-formal systems not constructed this way. Eg., text tokens are not distributed like causal structures in the world -- and it's absurd to suppose so.
The distributional hypothesis is obviously false for almost every measurement system of interest: measures are not distributed like their causes. We invent measurement systems to encode information, not to be isomorphic to world-structure.
"That’s basically the claim of the “Blurry JPEG of the Web” article. I agree that to a large degree that’s probably what’s going on here."
I strongly disagree with that view, and encourage reading the "Sparks of Artificial General Intelligence" paper[1] and watching the associated video[2], then just experiment with GPT4 using novel information that has been created after its training data cutoff and you'll see it reason about it. It can also output novel creative works about things that didn't exist in its training data.
Sure, there's some "compression" going on in LLMs, but that's far from everything that's going on.
Every word is a mapping of meaning, no? Hunger is universal yet every tribe has a different word. Audio, visual, physical actions represent hunger.
Map the 70 emotions, dozen tenses, handful of modifiers and suddenly the only missing variables of state and desired state will bring about a framework to allow universal translation.
Am I the only one who is surprised how small the storage size is? I understand it’s a lot of text but to realize it easily fits on a few cheap HDDs raided together is still incredible to me.
For some reason, this article refers to the Chinchilla scaling laws as "data-optimal scaling laws." They are actually scaling laws that describe how to train the best model at a given computational cost, assuming that both the model size and the amount of data on which the model can be trained are constrained only by the amount of compute available. You can get an equally good model with less data if you make the model bigger, but such a model would require more compute to train than the compute-optimal model. It may also be possible to repeat the training set during training and get most of the benefits of training on more data as long as it isn't repeated too many times; this is a common thing to do in other subfields of ML but for LLMs the effect of doing so is not well-characterized.
- best score - don't care about efficiencies (GPT3, GPT4)
- best score for a fixed quantity of compute at training time - good for PhD's and people who make proofs-of-concept (Chinchilla)
- best score for a fixed quantity of compute at inference time - good for people who inference their models at scale (LLaMA, chatGPT turbo)
The article didn't mention the LLaMA scaling laws, where we use more than 20 tokens per weight, more precisely 142 tokens per weight for LLaMA 7B.
> The objective of the scaling laws from Chinchilla is to determine how to best scale the dataset and model sizes for a particular training compute budget. However, this objective disregards the inference budget, which becomes critical when serving a language model at scale. In this context, given a target level of performance, the preferred model is not the fastest to train but the fastest at inference, and although it may be cheaper to train a large model to reach a certain level of performance, a smaller one trained longer will ultimately be cheaper at inference. For instance, although Chinchilla recommends training a 10B model on 200B tokens, we find that the performance of a 7B model continues to improve even after 1T tokens.
What we care about is the best model we could run on our own hardware, not how efficient was its training, that doesn't cost us users anything.
> What we care about is the best model we could run on our own hardware, not how efficient was its training, that doesn't cost us users anything.
Training cost is a huge (but diffuse) cost because it limits which organizations can be train a model. Such concentration of power greatly affect users.
Even if you want to get best score overall, Chinchilla laws still apply. Any model is trained on finite amount of compute, and there is optimal (in a sense of minimal loss) model size for this amount of compute. So difference between 1 and 2 is only amount of compute basically.
As for inference if you want just bound from above possible model size, then just take largest model you can allow and train for as long as possible. There is no evidence (yet) that we can hit the ceiling with this one.
I have yet to see anyone explain or explore why text could not be trained on multiple times. And is there any reason a text passage wouldn’t be just as good run backwards as forwards? Predicting the previous word in a sentence seems just as relevant to grasping semantic meaning as the next word.
This is already how it works. When you train a deep learning model you show the samples of the training data hundreds of times. Of course depends on the size of your training set and your budget, etc.
You often train for 100s of epochs. One epooch means one pass over the training set.
The original T5 models were trained with masked language modelling similar to BERT and afterwards also trained on a autoregressive task similar to GPT.
Could the text size shrink using already trained LLMs? There is probably alot of irrelevant or wrong information in this data, and the LLMs could be used to remove this information.
SanderNL doesn’t know if he is just jealous, but he thinks it kind of odd to talk about one’s self on one’s own website like this:
“A contributor to the fields of human intelligence and peak performance, he has held positions as chairman for Mensa International, consultant to GE and Warner Bros, and memberships with the IEEE and IET.”
He doesn’t mean to degrade the guy as he doesn’t even know him. It’s just.. ah, well.
It's not that weird in a professional bio. For example if you read someone's bio in journal article (some ieee articles ask you for this) it will be 3rd person. Same for of you're a speaker at a conference.
To each his antidepressant. He probably (rightly) thinks that these are more effective than an equivalent helping of SSRIs. When you cannot degrade him, pity him!
In my personal experience anyone that claims they have anything to do with Mensa should be avoided. It's nothing but a pay some money per year club for bragging rights to claim you're superior to other human beings.
What companies say publicly, is different to what they are actually doing internally. Surely you don't believe everything that a company says straight out of their PR spin room?
O̶p̶e̶n̶AI.com knows that they cannot let the DALL-E 2 situation (with Stable Diffusion interrupting their rollout) happen again with the GPT releases and they have to cement their lead with another surprise announcement to throw everyone off.
I'm expecting them to acquire an AI accelerator and hardware company or at least add further investment in that chosen company.
This guy's site is hilarious. 20+ testimonials about how great he is - all from 2022 onwards.
Pre-2022 his books seem to be non-AI related. One of them is "Best: A practical guide to living your best life" and another is a book for gifted toddlers.
An older description of him says "Dr Alan D. Thompson is a world expert in the fields of child prodigies, high performance, and personal development. "
Now he is a "world expert in artificial intelligence (AI)".
counterpoint, the amount of hubris and gatekeeping in the entire AI/ML space has been a decade long setback. there is an assumption that PhD's in Machine Learning are necessary, like at all, for every single random AI startup, when that's not the case, at all. Until this year the same has been true for raising capital, since nobody could tell that off the shelf tools could be used by any junior dev to train these things but now we all know the emperor has no clothes. I'm glad that moat is getting disrupted faster than starving artists.
I would like to understand the role of quality vs quantity in these models. Is it better to train them on slight variations of what humans consider to be high quality input or just add a boatload of low quality input instead? If we somehow found an original Shakespeare book, wouldn't it be drowned by all the trashy Amazon erotica written every year?
I understand that there are reasons to expect improvement with a broader set of inputs - the model would never understand slang and other key language components by being trained only in academic papers. I wonder whether the Shakespeare example would be worked out by it somply occupyinf a novel high-dimensional space because of its uniqueness, or whether there is a signal to noise issue here.
163 comments
[ 2.7 ms ] story [ 213 ms ] threadFor the record, I'm not saying LLMs are not a huge step forward. They are. But they are not -- and cannot be -- the whole answer.
That helps tremendously compared to a tabula rasa.
(That said, DL architectures are obviously wildly different from how the human brain works. E.g. backprop is physically impossible.)
I'm sure there's a lot more to it than this, but maybe one factor that makes humans a lot more data efficient is the multimodal input we receive.
If that's the case, imagine how much better things could get when we train with all the videos, podcasts, radio etc in the world, in addition to all the text out there!
2. This tendency to police other people's behavior, telling them what they should or should not do, does not help your cause. It puts people on the defensive. No one, especially adults, like to be told what, or what not, to do unless it's coming from a well-qualified lawyer.
"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."
See: https://news.ycombinator.com/newsguidelines.html
Nor was anyone disparaged or disrespected.
When people stopped having "friends" and started accruing "followers."
You can bully anybody about anything when your gang is bigger than theirs.
https://en.wikipedia.org/wiki/Helen_Keller
And it's been optimized over billions of years.
You can look at it the other way around - dopamine and other neuro transmitters as poor approximation of backpropagation. It has many flaws for example tight harmful loops ie. addictions.
Majority of brain work is ignoring irrelevant information (attention) and small scale hallucinations (we don't see world as is but slightly hallucinated to keep it stable - ie. they way brain processes blinking <<turns off>>, you can peek at those nuances with ie. optical illusions etc).
One of missing bits in neural nets may be reusing its output as input (embedded in inference itself, not poor mans re-prompting).
Once it's sorted out I'd argue the performance will skyrocket and give opportunity to massive optimisations ie. embedding things like known functions - imagine brain which has known, very narrowed, available functions at its disposal - all mathematical functions on numbers, logic, optimal sorting etc. Imagine if as thinking human you'd have access to accurate functions - the sky is a limit.
Bayes formula as a built in primitive. You don't need to know much of statistics to see how limited humans are at processing information because estimating posterior updates is so expensive for them.
Thinking in terms of raw probabilities would be very alien to most humans, but could easily be technically superior for making plans.
This is only true when considering single performance axis like pixel resolution. When you consider the corpus of power efficiency, jitter resolution enhancement, dynamic contrast, performance per volume, etc. We aren’t close to building something as capable.
It's pretty clear that humans, unlike LLMs, use external sensory data (the only external data we have, when you cut through it all) when they produce speech, as evidenced by the fact that they don't speak falsehoods that don't mesh with their internal data model. LLMs have such a weak model of reality -- it's whatever “sounds right” -- that they speak falsehoods all the time. The only way to give an LLM sensory data would be to encode every sensory experience people have into text.
I don’t have access to any other humans internal data model, but the indirect evidence I do have suggests that they do, in fact, speak falsehoods that don’t mesh with their internal data model for a variety of strategic purposes.
So does Wikipedia. But that's not a good model of the human brain either.
> I don't think it's quite an apples to apples comparison.
Yes. That is exactly my point. Despite the superficially similar I/O behavior, the two systems are very different under the hood.
What weights are you referring to ?
Obviously newborns need to develop and take in stimulus before they "know" anything, so the initial conditions are not sufficient, but they are obviously necessary to make the limited learning useful.
As far my understanding goes to very large degree it is unknown how to model such dynamics, where it would be possible to start with no spiking neurons and evolve effective/stable learning behavior from small number of examples/experiences.
There may be many roads to intelligence. The path biology and evolution took may just be one such path.
Are they coded in DNA, or do epigenetic factors, the environment cells grow in, adjacent cells, their interaction, etc, play a role here as well?
Does this refer to DNA -> MRNA -> to protein or is there some other mechanism here?
Also, since you asked, I would like to mention protein interactions with cellular components and tissues. These can be argued as just proteins interacting with each other but the extreme complexity of these higher levels and the unique phenotypes arising from them make me think they are deserved to be treated as another layer of "data decompression".
A chimpanzee can listen to humans as much as it wants, and it will still not pick up much of the language.
It's deeper than that. Our brains are optimized for a lot general human functions. Learning language is one of them. There's a whole section of the brain dedicated to it and other things.
There's also a lot of vital biological information encoded in DNA/RNA. We are not even close to starting from scratch.
There's some reasons to suspect this is at least partially true, but to what extent is unknown and contraversial.
It doesn't say anything about that knowledge being precoded.
The fact that human brains are capable of learning things that animal brains cannot, is a clear indicator that it is already "trained" to a significant degree.
Human "learning" seems to be much more analogous to "fine tuning and memory storage/retrieval", than actual training.
Why can't a macaque teach a calculus class? Why can't its "empty" brain be taught to do something like that?
If you mean a lot of accuracy, that's obvious and doesn't really need argument. And this new fact doesn't change the argument.
If you mean a more moderate amount of accuracy, this isn't proof either way. Human brains take in less text but they put a lot more processing into it.
But regardless, the architecture of the human brain doesn't have to be the only way to get to AGI (not that LLMs are necessarily the way)
This is assuming we are learning nothing during sleep, which probably isn't true.
By the time a person is 21 years old, they have been trained on at least 1 petabyte of data.
By two years old, about 125TB of data. It makes LLMs look quite good in comparison.
Call 10^10 \approx 2^40 for convenience, and 8000 \approx 2^13, which gives us a 2^53 entropy estimate, or about a petabyte of information as an estimate of what the human brain can store (discounting more exotic theories of memory stored in DNA or some such).
[0] https://en.wikipedia.org/wiki/Human_brain#Microanatomy
[1] https://psychology.stackexchange.com/questions/7967/how-many...
A single pyramid neuron in the neocortex might be more comparable to a multilayer neural net.
https://www.biorxiv.org/content/10.1101/2021.10.25.465651v1....
We don't understand how they work at the subatomic level simply because human understanding of the subatomic world is not complete, but even just at the atomic level a single neuron is massively more complex than anything humans have created.
Going up to the molecular level, even that is staggeringly more complex than the incredibly simple abstractions that make up a neural net.
Is what happens in the brain at the molecular, atomic, or subatomic levels relevant or necessary to intelligence and consciousness? We just don't know yet, but we do know all of that is far more complex and very different from the simple abstractions that are used for neural nets and LLMs.
The back of a napkin calculations in this thread don't even begin to do justice to the tremendous amount of "calculation" or "storage" that happens in the human brain.
In contrast the data we collect through our nervous system is rich and meaningful and far deeper than just raw text. We can even manipulate the environment as we learn to facilitate faster learning eg. pick up a ball and throw it, rather than just watch videos of balls being thrown.
Our object recognition is trained pretty quickly. And we sleep (certainly as a child) more than 6 hours per day. But we don't learn much from just looking at pictures.
> 18 hours of reinforcement learning
You made that up.
> It makes LLMs look quite good in comparison.
So, exposing an LLM to a lot of video will make them understand language?
Even sound alone (uncompressed CD quality stereo) is 3TB/year.
Some deaf/blind people can read braille — they learn fine too.
Much less data than you might think.
Bare in mind most people can run the brain on 2500kCal/day.
And so on.
Which is, in turn, 2900 W·h, or 2.5 times less than one A100 card working round the clock (300W·24h)
The significant thing is the reinforcement. Without it you have to resort to the openai brainlet style 10tb of text training.
If their model had reinforcement built in then you could just plop it in front of a person or on reddit and it would rapidly self-learn on a fraction of the data. Their training model relies on it learning purely based on observations rather than interaction which is inefficient.
I don't care if my model needs an exabyte of RAM if I can just go and buy that much RAM one day.
Yes, we use now magnitudes more oil than we used to 120 years ago. But back then oil was used for cooking and lamps, while now it is used for so many more things. Same goes for data. It is the new oil :).
These LLMs are also trained on arbitrary books. When acquiring new languages, we use educational materials that are specifically created to facilitate an understanding of language. Not just arbitrary books in random order.
For instance, a recommendation system's output effects what users see, so it effects what they click on. The next training set is statistically dependent on the input of the previous.
There are strategies to deal with it, but in the case of a LLM it seems difficult apart from downweighing everything after 2023 that isn't from a vetted source.
https://en.m.wikipedia.org/wiki/Low-background_steel
tl;dr steel (except for low background steel from sunk battleships) has been pretty much forever contaminated by nuclear weapons.
I'd posit that data-wise, ChatGPT is the equivalent event.
The RLHF reward model is effectively a good/bad content filter. Human==good AI==bad doesn't always hold, there is good AI generated content and bad human written content out there, we should filter by content quality and novelty, not origin.
If I had the resources I would turn GPT-4 onto its own dataset to mine all the facts from all the source documents. Then flip the index grouping by fact, and for each fact it should write a page listing the supporting and contradicting evidence. This would be super useful in determining - 1. existence of a fact, 2. its controversiality level and 3. the distribution of answers.
With these pieces of information at hand GPT would get much better, it would at least stop hallucinating. This fact index would probably double the size of the training corpus as well. Google, with its years of search logs, is the best positioned to create this fact index. They already had simpler attempts at creating a knowledge graph for a long time. But if they don't move to exploit the data they sit on, others will take the lead anyway. It costs only money to run so much text over the model, no slow & expensive human involvement necessary.
It's almost certainly a problem for LLM development, just like it is for humans. Humans generate all sorts of stuff, some of it being absolute bullshit, and it does seem to cause problems for other humans. Yet with effort it still seems to be possible to cut through the bullshit in a lot of cases, so it's likely not an insurmountable problem for LLM development either.
We have to be very careful with the "factual" content of old non-fiction works. So much of that, from history, to medicine, to biology, etc, turned out to be straight from their writers' imaginations. If an LLM considers such books to be no different from modern books on these subjects it would get a very skewed view of the world.
Imagine asking an LLM for a medical diagnosis and it responding with something about the humors.
For this, large amounts of very old text would be extremely valuable.
What about data that's sitting around and isn't supposed to be public? If training data gets scarce, does a market for small-medium sized data emerge? Like old homework papers, internal company documents, etc?
Edit: Still, the idea of mapping symbols and words to a many-dimensional space of meanings is a great insight into how mind works. In that space, symbols with similar meaning appear next to each other, and a thought looks like a smooth but intricate shape that separates all symbols into the "insiders" and "outsiders" that, in practice, divide symbols into true/false, good/bad and so on. Such smooth intricate shapes appear in the frequency domain as a bunch of rational numbers, and that's the boundary of what a mind can imagine.
Can you explain why the thought divides symbols into good/bad and how it is projected to rational numbers? (What is the frequency domain of?)
In other words, once you “compress” to thoughts, there actually can be some degree of actual reasoning on “decompression”.
Has any one written the obvious reply to this paper yet?
It's false. It's not true.
The rules and the board state it "learns" are just functions of the token positions *by construction*; it is given the token positions. obviously it learns patterns in those positions.
The paper is such a misfire, it's absurd.
The problem with using a formal system to investigate this issue is that by construction the "distributional hypothesis" is true for that system. Ie., the pattern in distribution of the moves are (a very good model of) the rules and board-state.
But this is clearly untrue for non-formal systems not constructed this way. Eg., text tokens are not distributed like causal structures in the world -- and it's absurd to suppose so.
The distributional hypothesis is obviously false for almost every measurement system of interest: measures are not distributed like their causes. We invent measurement systems to encode information, not to be isomorphic to world-structure.
I strongly disagree with that view, and encourage reading the "Sparks of Artificial General Intelligence" paper[1] and watching the associated video[2], then just experiment with GPT4 using novel information that has been created after its training data cutoff and you'll see it reason about it. It can also output novel creative works about things that didn't exist in its training data.
Sure, there's some "compression" going on in LLMs, but that's far from everything that's going on.
[1] - https://arxiv.org/abs/2303.12712
[2] - https://m.youtube.com/watch?v=qbIk7-JPB2c
Map the 70 emotions, dozen tenses, handful of modifiers and suddenly the only missing variables of state and desired state will bring about a framework to allow universal translation.
Maybe there already is and I just haven’t looked hard enough.
https://pile.eleuther.ai/
- best score - don't care about efficiencies (GPT3, GPT4)
- best score for a fixed quantity of compute at training time - good for PhD's and people who make proofs-of-concept (Chinchilla)
- best score for a fixed quantity of compute at inference time - good for people who inference their models at scale (LLaMA, chatGPT turbo)
The article didn't mention the LLaMA scaling laws, where we use more than 20 tokens per weight, more precisely 142 tokens per weight for LLaMA 7B.
> The objective of the scaling laws from Chinchilla is to determine how to best scale the dataset and model sizes for a particular training compute budget. However, this objective disregards the inference budget, which becomes critical when serving a language model at scale. In this context, given a target level of performance, the preferred model is not the fastest to train but the fastest at inference, and although it may be cheaper to train a large model to reach a certain level of performance, a smaller one trained longer will ultimately be cheaper at inference. For instance, although Chinchilla recommends training a 10B model on 200B tokens, we find that the performance of a 7B model continues to improve even after 1T tokens.
What we care about is the best model we could run on our own hardware, not how efficient was its training, that doesn't cost us users anything.
Training cost is a huge (but diffuse) cost because it limits which organizations can be train a model. Such concentration of power greatly affect users.
What does "inferencing models at scale" mean?
The models produced have to be fast and efficient to support capacity/cost, which has some detriment to quality/accuracy.
As for inference if you want just bound from above possible model size, then just take largest model you can allow and train for as long as possible. There is no evidence (yet) that we can hit the ceiling with this one.
You often train for 100s of epochs. One epooch means one pass over the training set.
OpenAI's GPT-5 that they've said they're not making yet?
Does anyone actually read this guy or is this what they call puffery?
“A contributor to the fields of human intelligence and peak performance, he has held positions as chairman for Mensa International, consultant to GE and Warner Bros, and memberships with the IEEE and IET.”
He doesn’t mean to degrade the guy as he doesn’t even know him. It’s just.. ah, well.
I think these are important details.
We should probably make a new rule for AI article, where the (Month Year) is posted if the article is older than a month.
O̶p̶e̶n̶AI.com knows that they cannot let the DALL-E 2 situation (with Stable Diffusion interrupting their rollout) happen again with the GPT releases and they have to cement their lead with another surprise announcement to throw everyone off.
I'm expecting them to acquire an AI accelerator and hardware company or at least add further investment in that chosen company.
Pre-2022 his books seem to be non-AI related. One of them is "Best: A practical guide to living your best life" and another is a book for gifted toddlers.
An older description of him says "Dr Alan D. Thompson is a world expert in the fields of child prodigies, high performance, and personal development. "
Now he is a "world expert in artificial intelligence (AI)".
Looking forward to see his next world expertise.
I understand that there are reasons to expect improvement with a broader set of inputs - the model would never understand slang and other key language components by being trained only in academic papers. I wonder whether the Shakespeare example would be worked out by it somply occupyinf a novel high-dimensional space because of its uniqueness, or whether there is a signal to noise issue here.
Or would it? Are there estimations abut this?