I truly don't think it does. AGI - a human level generally intelligent agent - is possible with today's technology. It'd take a lot of classical programming work - definitely man-centuries, but even if nothing new comes out and microchips stop progressing, it's possible. Look what can be done with the dumbest system imaginable - just raw input to raw output based on unfiltered conversations of internet idiots. I just can't see how it isn't possible if somebody invested the necessary man-centuries, which isn't that much - a mid level software house with few years of time.
You have thrown in your opinion and not countered the content of the poster.
That «y-axis» mentioned states that GPT-4 would be at the level of some «Smart High Schooler». I have been unable to be focused on LLMs in the past months, but unless the LLM revises its internal body of knowledge - which, if it happened, should be quite well advertised - it is not at the level of any relevant minor animal brain.
I don't want to counter it. They have their opinion and that's fine, I don't agree nor disagree.
I'm trying to say that we don't need any further advances in the science of ML or computers - we could already do it with a monumental lot of engineering. ChatGPT is what I meant by saying "the dumbest system - raw input to raw output based on conversations".
Several engineering steps are necessary IMHO:
- go through the training data and clean it up - likely what OpenAI does right now, but it's going to take a lot more than they have done so far
- make it a complex system with thoughts, memories, reasoning loops, planning, goal setting, self introspection... - this is what many amateurs are trying but the professionals seem to think their time is better used elsewhere right now
- careful machine code-level optimization of the software for purpose-built hardware
So, no new technologies or science are strictly necessary - "just" man-centuries of engineering.
Right now companies and researchers seem to think that there are more gains to be made at the fundamental scientific/technological level. But if that doesn't pan out, it doesn't mean AGI will never exist - just perhaps later, more energy consuming and less powerful than they thought.
Just imagine how dumb you would look if everybody heard the first thought your brain came up with, you couldn't stop yourself from saying it, think about it to consider the implications and meaning - or at least be aware you're saying it... I'm a sleep talker and the bullshit I say without any recollection of it is just insane, and frankly very similar to a hallucinating ML model.
> it is not at the level of any relevant minor animal brain
It's not at the level of any irrelevant animal brain. An LLM doesn't have to compete with mice. It doesn't even have to compete with ants. It's in symbiosis with human civilization, it's consuming our electricity and we consume it's output.
I have been speculating on Nvidia stock. It has been following an exponential curve. I have curve fitted the stock price, and determined that:
- in 2027, the market cap of Nvidia will be approximately $10 trillion - approximately half of US GDP
- in 2035, the market cap will be $1,500 trillion. Far greater than the total amount of value produced throughout human history.
It would be very unwise to bet against me; up to this point, my curve fit has been completely correct. Naysayers have been squashed. By my calculations, which are based on the laws of mathematics and completely indisputable -- the total market cap of Nvidia will be well over 10 quadrillion dollars by 2040.
I'm actually growing concerned. In our lifetime, Nvidia stock might become so uncountably valuable that even an attempt to calculate its value will consume most of the energy available in the known universe. Because I'm serious about my math, I've put some confidence intervals on my estimate; but, dear reader, even that doesn't save us. In the conservative case, it only forestalls the inevitable by just a few years.
I suggest we assemble a task force to ponder the inevitable problems that will arise from this set of facts. Humbly, I will submit myself as the leader of such a task force. Because of the gravity of the situation, I think a few million dollars, a small team, and perhaps a few hundred or thousand shares of Nvidia -- will help to fully understand the implications of this dire situation.
You forgot to mention that when Nvidia's market cap reaches infinity and beyond, by then we would surely be an intergalactic robotic species. Just sayin'.
> AGI by 2027 is strikingly plausible. GPT-2 to GPT-4 took us from ~preschooler to ~smart high-schooler abilities in 4 years. Tracing trendlines in compute (~0.5 orders of magnitude or OOMs/year), algorithmic efficiencies (~0.5 OOMs/year), and “unhobbling” gains (from chatbot to agent), we should expect another preschooler-to-high-schooler-sized qualitative jump by 2027.
The largest GPT-2 model had 1.5 billion parameters (each a connection weight between two artificial neurons). By some estimates, GPT-4 has on the order of 1 trillion parameters, or around three orders of magnitude (10^3) larger.[a] Another jump of three orders of magnitude would require a model with on the order of 1000 trillion = 1 quadrillion parameters.
It's far from certain that such scaling is possible over the next three years, and also whether existing model architectures, e.g., Transformers, can continue to improve qualitatively at similar rates as in the recent past, because there is some evidence they're hitting diminishing returns. Prominent experts like Yann LeCun think we will need new ideas and breakthroughs, i.e., new kinds of AI models not yet invented, the timing of which cannot be predicted in advance, by definition.[b]
The OP's "forecast" all but ignores these challenges.
16 comments
[ 3172 ms ] story [ 178 ms ] threadIt also requires believing the made-up labels you wrote on the y-axis.
That «y-axis» mentioned states that GPT-4 would be at the level of some «Smart High Schooler». I have been unable to be focused on LLMs in the past months, but unless the LLM revises its internal body of knowledge - which, if it happened, should be quite well advertised - it is not at the level of any relevant minor animal brain.
I'm trying to say that we don't need any further advances in the science of ML or computers - we could already do it with a monumental lot of engineering. ChatGPT is what I meant by saying "the dumbest system - raw input to raw output based on conversations".
Several engineering steps are necessary IMHO:
- go through the training data and clean it up - likely what OpenAI does right now, but it's going to take a lot more than they have done so far
- make it a complex system with thoughts, memories, reasoning loops, planning, goal setting, self introspection... - this is what many amateurs are trying but the professionals seem to think their time is better used elsewhere right now
- careful machine code-level optimization of the software for purpose-built hardware
So, no new technologies or science are strictly necessary - "just" man-centuries of engineering.
Right now companies and researchers seem to think that there are more gains to be made at the fundamental scientific/technological level. But if that doesn't pan out, it doesn't mean AGI will never exist - just perhaps later, more energy consuming and less powerful than they thought.
Just imagine how dumb you would look if everybody heard the first thought your brain came up with, you couldn't stop yourself from saying it, think about it to consider the implications and meaning - or at least be aware you're saying it... I'm a sleep talker and the bullshit I say without any recollection of it is just insane, and frankly very similar to a hallucinating ML model.
In fact, there is no question about revision of ideas being crucial. Some seem to ignore it. Others do not know exactly how to implement it.
It's not at the level of any irrelevant animal brain. An LLM doesn't have to compete with mice. It doesn't even have to compete with ants. It's in symbiosis with human civilization, it's consuming our electricity and we consume it's output.
- in 2027, the market cap of Nvidia will be approximately $10 trillion - approximately half of US GDP
- in 2035, the market cap will be $1,500 trillion. Far greater than the total amount of value produced throughout human history.
It would be very unwise to bet against me; up to this point, my curve fit has been completely correct. Naysayers have been squashed. By my calculations, which are based on the laws of mathematics and completely indisputable -- the total market cap of Nvidia will be well over 10 quadrillion dollars by 2040.
I'm actually growing concerned. In our lifetime, Nvidia stock might become so uncountably valuable that even an attempt to calculate its value will consume most of the energy available in the known universe. Because I'm serious about my math, I've put some confidence intervals on my estimate; but, dear reader, even that doesn't save us. In the conservative case, it only forestalls the inevitable by just a few years.
I suggest we assemble a task force to ponder the inevitable problems that will arise from this set of facts. Humbly, I will submit myself as the leader of such a task force. Because of the gravity of the situation, I think a few million dollars, a small team, and perhaps a few hundred or thousand shares of Nvidia -- will help to fully understand the implications of this dire situation.
You forgot to mention that when Nvidia's market cap reaches infinity and beyond, by then we would surely be an intergalactic robotic species. Just sayin'.
Don't be shy with your forecasts!
;-)
Maybe we need browsers with quick visual tweakers (past CSS editing in the "developer's panel").
The largest GPT-2 model had 1.5 billion parameters (each a connection weight between two artificial neurons). By some estimates, GPT-4 has on the order of 1 trillion parameters, or around three orders of magnitude (10^3) larger.[a] Another jump of three orders of magnitude would require a model with on the order of 1000 trillion = 1 quadrillion parameters.
It's far from certain that such scaling is possible over the next three years, and also whether existing model architectures, e.g., Transformers, can continue to improve qualitatively at similar rates as in the recent past, because there is some evidence they're hitting diminishing returns. Prominent experts like Yann LeCun think we will need new ideas and breakthroughs, i.e., new kinds of AI models not yet invented, the timing of which cannot be predicted in advance, by definition.[b]
The OP's "forecast" all but ignores these challenges.
---
[a] https://en.wikipedia.org/wiki/GPT-4
[b] https://www.reddit.com/r/singularity/comments/1d5b57b/lecun_...
Have you also produced an argument somewhere?