Ask HN: What if AGI isn't that close?
What if transformer models are the totally wrong approach? What if we're about to spend ten years iterating on a fancy parlor trick? We might invent something that almost seems like AGI but really it's just calling a bunch of APIs invented (and maintained) by humans.
What should we have spent the last ten years researching instead?
50 comments
[ 2.9 ms ] story [ 68.9 ms ] threadSo far, my assessment is that human intelligence is much less interesting than we thought.
Also, I don't think GPT-4 is conscious. If you ask it whether it is, it will say no. It does not appear to exhibit preferences or anything like suffering.
I realize it’s a little absurd to be upset over a nickname but it somehow like really triggers me to see “sama” (lowercase, too!) so many times in one weekend.
"Some" is quite the weasel word. You could be referring to 3 total people in the world, but that wouldn't be worth spending energy considering. You can find people out there who believe all sorts of weird and stupid shit.
Who's saying this and what is their reasoning?
Yesterday I had a totally reasonable conversation with it about the bugs I found in my flat... and the day before that I was pair programming with it on the best TypeScript interface for the library I'm building... nothing like this existed six months ago and to be honest it's pretty mind blowing. It makes me excited for the future, and even though I know the same demonstration of skill makes some people existentially worried for the future, I can't help but optimistically look forward to what we're all going to build with this thing...
Frankly, I say "so what?" For this to be "wrong" implies that the only useful outcome is AGI. I posit that that is clearly not the case. Current approaches are obviously useful and create value. It's like trying to invent television and getting radio instead. OK, fine, you still got something useful, so what are you complaining about? And the other will still come in time.
What if we're about to spend ten years iterating on a fancy parlor trick?
If it keeps getting better at doing things that people find useful, then that's fine.
We might invent something that almost seems like AGI but really it's just calling a bunch of APIs invented (and maintained) by humans.
Not really relevant, IMO. If the thing we invent behaves in ways that can be classified as showing intelligent behavior, then it's intelligent. If it reaches the bar that most people are willing to say "that's general intelligence", then it doesn't actually matter how it works. OK, to be fair there are senses in which it matters (getting into things like explainability, alignment, etc. yadda, yadda) but in terms of saying "is this intelligent or not?" we don't really have to know or care about the inner workings. I mean, we consider other humans intelligent (well, sometimes) and we don't know all the details of how human intelligence works either.
There is always progress to be made. The goal posts can always be moved forward. The standards for what is possible raised.
There is no limit to how high we could raise the global standard of living.
True, but per GP there is also an unimaginable amount of money and natural resources sent towards those technical problems. And unlike programmers, money and resources are reallocateable.
If, in an alternate history without the gravitational money-pull of the software industry, even 1% of the resources lent to that industry in our timeline had been put towards addressing one of the problems listed above, it would create a vastly different world. A better one, I suspect, in many ways.
You can't solve healthcare by throwing tech, money, etc at it anymore than you could solve a broken heart with those things. Sure, money could help you get therapy or pay for lobbying for whatever your solution to healthcare is. OTOH, perhaps the person cutting the checks thinks the solution to heartbreak is a raise ("it shows we care"), and the solution to healthcare issues is to lobby for criminalizing generics ("they're probably cutting corners and making people worse off"). People don't even agree on what are the problems in healthcare (and other things in that list), let alone how to solve them
Intelligence implies an environment and some sort of self-conservation. So being able to predict the environment AND using the prediction to survive/gain an advantage. The leaf to a self and consciousness is not that big from this but made murky by not knowing what consciousness really is to begin with.
The problem when talking about AI, AGI, intelligence is that we are always doing that using an anthropomorphized lens.
If we discard this lens and take these developments at face value for what they are, tools, the conversation can be honest and we can talk about pros/cons and where these technologies can be used. To pretend that the machines are going to take over any minute now and what we are seeing is "intelligence" will only help drive certain discourses and in the worst case will make these tools available for a small minority or have them banned altogether.
Even if it’s just a better tool, that’s a big deal.
It’s US tax season, I thought adjusted gross income first.
But ChatGPT is probably more useful as a tool than either.
So although I am profoundly skeptical of AGI metaphysics, I have come to the conclusion that "How do we make an AGI?" is a sound engineering approach when the goal is building useful tools for the benefit of other people.
YMMV.
What we currently have is useful and that’s a big deal; the time and effort spent on transformers wasn’t wasted. I personally don’t care if/when we have AGI and whether that’s on the same research path that we’re on now.
Right now, all transformers really do is distribute data probabilistically over a certain space, and the retrieval function maps itself to the "pattern" that looks the most similar. It's an advance in lossy data compression. It's a black box with a single door for output and input, it can necessarily never gain consciousness.
The only respite has been that we have been free from "Open"AI's marketing for the last 24 hours, and most of the "hype" has died down, normal topics have returned to the front page.
We are nowhere near AGI.
I’m genuinely asking. I think I disagree with your certainty, because my overall impression is that we don’t really understand the foundations of intelligence or consciousness. But perhaps that’s ignorance on my part.
Sure. I think LLMs map to the associative horizon in humans. Usually very pronounced in young children and geniuses[1]. They can intuit details and structures (patterns) with minimal instruction, when given enough time (geniuses discover new details in old tomes, children learn how to function in the world, children can learn multiple languages at once just by hearing people talk, etc). Mentation of this sort is highly abstract and is great for solving novel problems, but doesn't translate well to functioning in a environment where the rules are almost always arbitrary and not easily discernable from dysfunctional behavior. A lot of these issues stem from a primarily "inward" approach to new information. LLMs, being just a probabilistic distribution of data, suffer from similar issues.
LLMs don't have any structure of their "own", little to no "guiding principle" (consciousness), instead, their "wiring" is based on the data they are fed. This data is already tuned to our human mode of thinking and our judgement. Even the hardware is made possible by our own biases ("ability"). So, tend to anthropomorphize it very easily. A LLM doesn't have an "inside" or an "outside", it's a large pattern from which parts are "picked" and "shown" to us. So this act of "picking" and "showing" would arguably be a better candidate for being "alive" than any large pattern it's used on. This act is not sophisticated enough; it makes "hallucination" possible, and also doesn't discriminate between the dataset and query, so that if the dataset has a "count to five" or "ask what my name is", the result will be influenced by it as if it was part of the query. Because of a lack of a structure from first principles, every result is a novel result.
Also, because a LLM is effectively queried as an egregore of whatever its trained on, there's only a technical difference between the different ways of changing the weights. I guess this is just a fancy definition of version increment.
We don't just process facts or pick and choose alternatives, our brain and the rest of the body work in tandem; the brain has many sources of real-time information - 5 senses, internal body "feel", "mystic" intuition, almost telepathic connection with close kin, etc. We have a certain degree of autonomy from what we consider the individual parts of our surrounding ecology, but we can't really leave it for long periods of time. The mind-body split is sidestepped by people that claim something digital can be "conscious". I think to really have a "conscious" machine, it needs to be built from first principles and not as a black box; and then, it would also need to co-exist with the environment, which would necessarily mean that it would only be able to exist in a closed, artificial environment. After all, the easiest way to make intelligent life is to have a biological child. I think people have really fuzzed the lines between digital tech / real life and because most of the digital economy exists as an analogue of the real (irl) economy.
I also don't think letting the LLM somehow change itself will work. Without human intervention on a large scale, it would be like coaxing a liquid to act as a solid without any change in its environment. And with human intervention, it'd just be regular research / tech work. I think, given any sort of "freedom", the LLM will gradually implode upon its own data. We can try to hypothesize about what constitutes true "randomness" for a LLM+"pick"/"show"system but in the end it's all in our own heads. It's not a structure that's so pervasive in our environment or so far past our scope that we'd discover something adjacent to it that could dwarf the current understand...