Letting companies get away with applying token limits etc. is ridiculous. The electricity is not the bottleneck (taxpayers subsidize it). The knowledge was stolen from artists and writers. It's a logic gate on tool use determining if you're in the right economic class.
Here's the thing about economists... The loudest ones don't want to be correct, they want to be influential. The ones who can actually make good predictions work for banks and hedge funds lol.
- Many more mediocre papers written (mediocre ideas, implementation, claude-isms everywhere)
- Much easier to try every possible combination of a regression in order to show the result you want (same for theorists).
The one thing I'm happy about is it's now much easier to extract historical data from old documents from Google Books. Still not perfect, but takes you 95% there. And creating plots and datavis just for quick exploration is super fast.
Essentially all of economic theory is aimed at explaining, not predicting. The distinction between the two goals [0] is sometimes under-appreciated within the profession, and almost always under-appreciated outside of it.
Most predictive tools in economics and finance have “surprisingly” little economic content; but once you understand the distinction between the two goals, it should be unsurprising that predictive models tend to make few economic assumptions and tend instead to rely on basic statistical techniques. It then shouldn’t be too surprisingly that predicting the future is quite difficult in a context in which the relevant processes are continually seeking an equilibrium that implies unpredictability.
I’m not an economist, but I do a lot of applied financial-economic modeling. State-of-the-art LLMs are really, really terrible at economic intuition. They will hinder, not help, in formulating an economic model, which is mainly a process of coming up with a set of modeling assumptions that lead to a useful (implicitly, tractable) model. However, they’re good at math, so I’ve found them very useful in iterating through different sets of modeling assumptions to identify those that lead somewhere useful. Not having to work out all of the mathematical detail myself, and avoiding getting lost in the weeds in doing so, has accelerated my work immensely.
>Much of economics is built on models that were designed to provide a formal, logical basis for understanding the economic world, which is often quite unintuitive.
I disagree. It's purposefully unintuitive.
>(For example, many intelligent people uneducated in economics exhibit intuitions opposite of basic economic ideas like opportunity cost or comparative advantage.)
Most people don't believe in comparative advantage. They believe in something that economists can explain away as comparative advantage.
All unconsumed fixed size investments will result in something that is mathematically the same as comparative advantage. This is the intuitive view that people have. You go to university and get a 5 year degree. Now your cost basis for work that suits your expertise is much lower than for work that is out of expertise. A worker buys an expensive machine, now the cost basis for hiring the guy with the machine is lower than buying your own machine.
This also explains why specialization emerges: All specialization is basically a form of an investment that has some residual left over results that can be monetized in the future. If there was no residual it would be as if you forgot your education and at that point the investment is fully consumed and you turn back into a non-specialized worker.
All of this is incredibly intuitive, but economists instead insist on an invisible "factor" [0] to drive efficient production.
[0] The "factor" concept implies comparative advantage exists first rather than emerges as a result of past decisions.
Yes, Ricardo originally introduced the idea of comparative advantage in the context of international trade, in a model in which different countries had different endowments of resources.
Nothing that imtringued wrote above suggests he understands comparative advantage, which is the idea that it is relative productivity, not absolute productivity, that should determine what one specializes in. That’s precisely what I meant about people finding the concept unintuitive.
> Applied statistics is a
far more precise descriptor, “but no one wants to use that term, because
it’s not as sexy.”
This really hit me some time back when I was explaining AI to a friend. After about 10 mins of rambling about LLMs and mentioning the attention paper like I knew what I was talking about, it ended with “oh so it’s just a really advanced auto correct”
As an MLE I feel these takes are too reductionist.
You could say the (nearly) same thing about search. And content recommendation. And clustering. And topic modeling. And outlier detection. And spam filtering. And image diffusion. And dimension reduction. And...
There's a lot in common between these things, but there's also a lot cool and different!
For transformers in particular, it's pretty cool that you get some WILD emergent properties simply from scaling up.
So yes, it's just a next token predictor, but I'm just a bundle of nerves and meat. I don't get a lot out of those descriptions.
Some concrete facts about LLMs are explained by their next token predictor nature. Every time it says "wait, that's wrong." instead of generating the correct thing the first time.
It's a little different than that. Your bundle of nerves and meat is not static. It changes over time.
To me the heart of the "next token predictor" is that the distributions are static. You can manipulate what you feed into it through context (and a lot of interesting engineering has been applied there through CoT and other techniques). But these models as implemented will never be able try things and learn from mistakes or adapt. They are a set of weights frozen in time. A set of distributions derived from the original data that created them.
That's not quite true though. The fact that most models are in practice non-deterministic has been a huge point of contention.
And there's nothing inherently stopping labs from continuously fine-tuning the weights after every new invocation. It's just a difficult (not to mention expensive) software and ML problem.
I was not saying that they are deterministic, rather that the distributions (aka weights) are fixed. A model as deployed today at anthropic/open ai/etc is not learning beyond the context as far as I know.
What prevents continuous fine-tuning from what I understand is catastrophic forgetting. You can do things like RLHF which are built to minimize the damage but that is more about bringing out capabilities of the base model than incorporating new knowledge (at least from my understanding, I am obviously not a researcher at a lab).
Yes, catastrophic forgetting is absolutely one of the problems that needs to be solved to enable something like this.
My broader point is just that there's nothing inherent to the structure of LLMs that stops them from updating their weights and continuously learning from environmental feedback in the way humans do, and there's already solid templates for how they could push even further in that direction.
But as an assessment of the current state, I agree with you, LLMs lag humans severely in ability to self-update.
>My broader point is just that there's nothing inherent to the structure of LLMs that stops them from updating their weights and continuously learning from environmental feedback in the way humans do, and there's already solid templates for how they could push even further in that direction.
"LLM" is a branded model as a product. Of course it could be anything, as long as it fulfills the product category.
But we live in reality, we can only look at what models are out there and we see that they don't do any of those things and yet we're supposed to act as if these models already do.
What can a transformer not do that people say they can do?
The parent comment said, paraphrasing, "learn from interaction with the world", and I'm responding, they absolutely can already do this by taking their logs of interaction with humans and updating their weights through backprop.
The reason you don't see that done "live" is primarily an economics problem rather than a limitation of the model structure.
Alright let's assume your premise is true, that transformers can learn from interaction with the world by updating their weights - then why isn't this done?
Because backprop fundamentally wants the entire data set in every pass. It doesn't behave well and is destructive when you update after the pre-training phase. RLHF/LORA are attempts to work around that and effective at what they do, but it is not learning in the sense you are talking about and also do not fully address the catastrophic forgetting problem. This architecture as is - is not compatible with continual learning.
It IS done with every new model release. Why do you think SpaceXAI bought Cursor? And then immediately had a huge jump in capability with their next model version?
It's just done in large batches for economic reasons.
> backprop fundamentally wants the entire data set in every pass
I'm genuinely unsure what you mean, it's not even possible to run backprop in this way?
No, I'd have to guess that new model releases have either new from scratch or continued pre-training. This is not the same as continual learning. Starting a new pre-training session is a dramatically different affair and involves utilizing the entire source data set in some fashion. Not just continued training on new data. At least from my understanding.
Bolting on new data to an existing model (fine-tuning) is precisely what gets you catastrophic forgetting.
3. Ability of AGENT-HARNESSES to learn
with the help of the human user.
So agents learn, LLM already knows
everything it will ever know, and
ESPECIALLY it has already learned
how to understand human language.
No 3. above means there is no danger
of the LLM getting corrupted. But
the agents running on user's machine
learn on behalf of that user who
shares the machine with them.
But they routinely leverage web search in connection to providing responses which means they're leveraging their static intelligence on top of a dynamic context corpus. This is critically important and arguably similar to humans in other words the typical human might take a while to develop a new skill but they can change how they dynamically leverage an existing skill based on context
How are they non-deterministic? Ok there is random involved but for watermarking results the random is substituted by pseudo-random, otherwise it won’t work. If you control the randomness, you should be able to reproduce answers to prompts in equal context 100%.
It's not "not quite true", it's literally true because alternative architectures like RNNs and Mamba fully update their own internal states, whereas transformers only append to the context.
RNNs and Mamaba do not update their weights, but you could hypothetically scale the internal state to be as big as Fable's and GPT 6's parameters.
At least one mechanism to update transformers' internal states already exists, there is nothing stopping anyone from performing backprop after every session.
It just has big technical and economic challenges. But I expect advances there. There have actually already been big advances, though done in bulk fashion (RLHF).
Right or put differently you can say that all computer stuff is reduced to ones and zeros. And if you say that in the right context you might convince someone that computers are therefore not very powerful or interesting but of course you're hiding the power of abstraction generalization scale and for lack of a better word entropy in action.
Sometimes I feel we lack the words to describe what we're doing in a way that really conveys how all these things Ladder Up
41 comments
[ 0.23 ms ] story [ 6.0 ms ] thread- Many more mediocre papers written (mediocre ideas, implementation, claude-isms everywhere)
- Much easier to try every possible combination of a regression in order to show the result you want (same for theorists).
The one thing I'm happy about is it's now much easier to extract historical data from old documents from Google Books. Still not perfect, but takes you 95% there. And creating plots and datavis just for quick exploration is super fast.
Most predictive tools in economics and finance have “surprisingly” little economic content; but once you understand the distinction between the two goals, it should be unsurprising that predictive models tend to make few economic assumptions and tend instead to rely on basic statistical techniques. It then shouldn’t be too surprisingly that predicting the future is quite difficult in a context in which the relevant processes are continually seeking an equilibrium that implies unpredictability.
I’m not an economist, but I do a lot of applied financial-economic modeling. State-of-the-art LLMs are really, really terrible at economic intuition. They will hinder, not help, in formulating an economic model, which is mainly a process of coming up with a set of modeling assumptions that lead to a useful (implicitly, tractable) model. However, they’re good at math, so I’ve found them very useful in iterating through different sets of modeling assumptions to identify those that lead somewhere useful. Not having to work out all of the mathematical detail myself, and avoiding getting lost in the weeds in doing so, has accelerated my work immensely.
0. https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf
I disagree. It's purposefully unintuitive.
>(For example, many intelligent people uneducated in economics exhibit intuitions opposite of basic economic ideas like opportunity cost or comparative advantage.)
Most people don't believe in comparative advantage. They believe in something that economists can explain away as comparative advantage.
All unconsumed fixed size investments will result in something that is mathematically the same as comparative advantage. This is the intuitive view that people have. You go to university and get a 5 year degree. Now your cost basis for work that suits your expertise is much lower than for work that is out of expertise. A worker buys an expensive machine, now the cost basis for hiring the guy with the machine is lower than buying your own machine.
This also explains why specialization emerges: All specialization is basically a form of an investment that has some residual left over results that can be monetized in the future. If there was no residual it would be as if you forgot your education and at that point the investment is fully consumed and you turn back into a non-specialized worker.
All of this is incredibly intuitive, but economists instead insist on an invisible "factor" [0] to drive efficient production.
[0] The "factor" concept implies comparative advantage exists first rather than emerges as a result of past decisions.
Nothing that imtringued wrote above suggests he understands comparative advantage, which is the idea that it is relative productivity, not absolute productivity, that should determine what one specializes in. That’s precisely what I meant about people finding the concept unintuitive.
This really hit me some time back when I was explaining AI to a friend. After about 10 mins of rambling about LLMs and mentioning the attention paper like I knew what I was talking about, it ended with “oh so it’s just a really advanced auto correct”
You could say the (nearly) same thing about search. And content recommendation. And clustering. And topic modeling. And outlier detection. And spam filtering. And image diffusion. And dimension reduction. And...
There's a lot in common between these things, but there's also a lot cool and different!
For transformers in particular, it's pretty cool that you get some WILD emergent properties simply from scaling up.
So yes, it's just a next token predictor, but I'm just a bundle of nerves and meat. I don't get a lot out of those descriptions.
> They're Made out of Meat
https://web.mit.edu/people/dpolicar/writing/prose/text/think...
To me the heart of the "next token predictor" is that the distributions are static. You can manipulate what you feed into it through context (and a lot of interesting engineering has been applied there through CoT and other techniques). But these models as implemented will never be able try things and learn from mistakes or adapt. They are a set of weights frozen in time. A set of distributions derived from the original data that created them.
And there's nothing inherently stopping labs from continuously fine-tuning the weights after every new invocation. It's just a difficult (not to mention expensive) software and ML problem.
What prevents continuous fine-tuning from what I understand is catastrophic forgetting. You can do things like RLHF which are built to minimize the damage but that is more about bringing out capabilities of the base model than incorporating new knowledge (at least from my understanding, I am obviously not a researcher at a lab).
My broader point is just that there's nothing inherent to the structure of LLMs that stops them from updating their weights and continuously learning from environmental feedback in the way humans do, and there's already solid templates for how they could push even further in that direction.
But as an assessment of the current state, I agree with you, LLMs lag humans severely in ability to self-update.
"LLM" is a branded model as a product. Of course it could be anything, as long as it fulfills the product category.
But we live in reality, we can only look at what models are out there and we see that they don't do any of those things and yet we're supposed to act as if these models already do.
What can a transformer not do that people say they can do?
The parent comment said, paraphrasing, "learn from interaction with the world", and I'm responding, they absolutely can already do this by taking their logs of interaction with humans and updating their weights through backprop.
The reason you don't see that done "live" is primarily an economics problem rather than a limitation of the model structure.
Because backprop fundamentally wants the entire data set in every pass. It doesn't behave well and is destructive when you update after the pre-training phase. RLHF/LORA are attempts to work around that and effective at what they do, but it is not learning in the sense you are talking about and also do not fully address the catastrophic forgetting problem. This architecture as is - is not compatible with continual learning.
It IS done with every new model release. Why do you think SpaceXAI bought Cursor? And then immediately had a huge jump in capability with their next model version?
It's just done in large batches for economic reasons.
> backprop fundamentally wants the entire data set in every pass
I'm genuinely unsure what you mean, it's not even possible to run backprop in this way?
Bolting on new data to an existing model (fine-tuning) is precisely what gets you catastrophic forgetting.
1. Technical cost of updating the mode.
2. Inability to trust every user's "truth".
3. Ability of AGENT-HARNESSES to learn with the help of the human user.
So agents learn, LLM already knows everything it will ever know, and ESPECIALLY it has already learned how to understand human language.
No 3. above means there is no danger of the LLM getting corrupted. But the agents running on user's machine learn on behalf of that user who shares the machine with them.
RNNs and Mamaba do not update their weights, but you could hypothetically scale the internal state to be as big as Fable's and GPT 6's parameters.
It just has big technical and economic challenges. But I expect advances there. There have actually already been big advances, though done in bulk fashion (RLHF).
Sometimes I feel we lack the words to describe what we're doing in a way that really conveys how all these things Ladder Up
Didn't know that MIT press features open access books.