Right now I'd say what's missing is sufficient time to build complex systems with LLMs, that will be AGIs with various features and components. We'll need to benchmark them as is currently done with LLMs.
It seems like people want it to mean not just general utility but a full simulation of a person (which then sometimes automatically becomes godlike). But they don't say that.
> Online non-greedy planning technologies that can operate in the presence of a priori unseen tasks, states, and actions.
> A world model technology that robustly predicts state-action-state transitions for all states and actions that might be encountered while performing any task in any environment.
People use greedy strategies all the time, and the intractability of considering 'all states and actions' is the Frame Problem. It is lack of generality that avoids the Frame Problem, and makes planning tractable for animals.
In my view, AGI is an anti-goal. Specialization is what makes intelligent systems feasible. On the other hand, there are technologies like DNNs, gradient descent strategies, even computation itself, that are generally useful for application to specialized intelligent systems.
By analogy, all animals are made of cells, and developed by evolution. But there's no General Animal that can do what all the others can, even if they share core technology.
edit: I wish I hadn't given the analogy. Quibbling about it seems to be irresistible, and has has distracted from my point here.
Unless that 'etc' is working very hard, I don't think you are seriously claiming that humans and their tech can do everything another animal can do. And if you were, we would only have to consider energy efficiency to save the analogy.
What can animals do that humans with tools can't? Energy efficiency only matters in comparison to the energy you can obtain: for animals, that means eating stuff. For plants, it means absorbing sunlight. For us, it means harnessing forms of energy no other living thing has ever deigned to.
This is in the context of AGI, not artificial life. We're not concerned with making an AGI that can photosynthesize or lay eggs. That's not what general intelligence refers to.
> There are benefits and downsides to AGI. On the upside, AGIs can do most of the labor that consume a vast amount of humanity’s time and energy. AGI can herald a utopia where no one has wants that cannot be fulfilled.
This is not a utopia, but a juvenile and unthinking definition of utopia, or a dystopia in disguise. Most people thrive on some sort of challenge to at least do basic tasks for themselves. A world where AGI fulfils every need is more like being in a senior's home where everyone does everything for you.
It sounds good at first but is actually a nightmare where your autonomy is stripped by the lack of goals to work towards.
Immense surplus and easy access to everything all the time means human beings will no longer need each other much either, so it means a world full of people uncaring towards each other. Fulfilling any want without hard work to get it means becoming effectively like a bacteria, consuming every resource in sight, transforming us into even more of a plague than we already are.
I certainly would not want to live in such a world.
Doesn't it bother you to describe humanity as a plague? The overall point is fine but these are people you're talking about, and NOT bacteria. It's just so anti-human to think of people that way. Souls. But a plague on the Earth, I guess.
Well, if you spend a lot of time looking at the damage we cause every day, studying reports and seeing innocent creatures like sharks being killed by the hundred millions, thousands of species being rendered extinct, endless garbage being dumped left right and center, habitats being destroyed endlessly by the millions of square kilometers, yeah, we do look like a plague.
And now we're trying to use AI to become even more resource-intensive, burning up endless energy to be even more powerful consumers...give me a break. It's sick. As individuals there's nothing seriously wrong with most people but en masse with modern technology, that is exactly what we are, a plague.
Perhaps trying implies too much deliberateness, but it sounds like an inevitable outcome to me.
One thing that the market will reward handsomely is bringing costs down. When costs go down, people consume more. If you're 40 or older, think about how many computers your family owned in your house growing up. We owned some hand-me-downs, and purchased 1 computer in my first 15 years. We bought about 4 in the next 3 years because they were advancing rapidly, but more importantly, because they had become much cheaper than they had been 5-10 years before. Today, many families might have a laptop for each person, replaced every 3-4 years.
One of the first things an AGI would be put to work doing is manufacturing, as we're already finding that it's hard to get people to do these stressful manufacturing jobs except in places where they're really desperate. Witness the moving of jobs from US -> China -> Southeast Asia.
A plague? A plague to what? The problem with our pollution and garbage and global warming is that it makes the world worse for us. The planet doesn't suffer, it just is. Plants and animals kill each other in a ruthless cycle of violence, too. We're just really good at it.
There is no good without humans. Any philosophy that asserts otherwise fails Camus's fundamental question.
@OkayPhysicist: You are wrong..what we are doing is making the world worse for us and other creatures. Because of us, the extinction rate is much higher than the background extinction rate.
I wouldn't be so quick to make that call. It's definitely a serious possibility, but at the end of the day I think it's hard to imagine exactly how people would fill their time and adapt. I think there are a lot of opportunities for challenges / goals outside of the typical profit-based economy.
I actually think that the disadvantages of having few goals would become immediately obvious to almost everybody, and a whole new type of skill / creative / athletic based "economy" might emerge. But with lower stress and lower stakes
So instead we should just roll the dice, wait to see what happens, and then what? With every new technology, it is hard to go back once it's out there. Maybe the professors and computer scientists will find new things to do but the vast majority of people certainly won't.
> So instead we should just roll the dice, wait to see what happens, and then what?
As opposed to what? Trust your hunch and halt technological progress. Wrap it up folks, we're all done with R&D now.
Your concern about purpose strikes me as odd. When people derive purpose from their work, it's often because they feel they're playing a role in a community. To deprive people of purpose would require that we make communities fully automated and impersonal. I don't think anyone wants or is proposing that.
Absent a requirement to spend one's time earning money, some people would probably degenerate into sloths, but plenty of people are otherwise motivated by self-expression, socialization, competition, etc. People would still interact and do things because people like interacting and doing things.
> Your concern about purpose strikes me as odd. When people derive purpose from their work, it's often because they feel they're playing a role in a community. To deprive people of purpose would require that we make communities fully automated and impersonal. I don't think anyone wants or is proposing that.
It is really not about wanting or proposing anything. It is about the effect of technology and what it will mean. Don't you think that communities are already becoming a bit more impersonal and automated? With ChatAI replacing customer service, self-checkout aisles, online dating, etc...
It seems to me that communities are indeed becoming more and more impersonal, especially if you compare them to communities thousands of years ago.
Definitely these interactions have become less personal, but aside from online dating, which is its own can of worms, the interactions are all transactional. I actually much prefer to interface with a predictable machine when purchasing things or making an RMA request or whatever. I don't want to stop to chit chat or worry about maintaining a relationship with the village cobbler's son to ensure he'll do his best fixing up my field boots.
The ongoing disentanglement of socialization and daily necessities is liberating. It allows one to associate with others by choice. I'm not a huge fan of the tools we have for this sort of voluntary community building at present, but I see no reason why they couldn't be improved, especially if people were able to spend their time and skill doing so free of profit motives.
Toil for toil's sake is some Sisyphean bullshit. Our goal should absolutely be to eliminate the necessity of work. If you can't find reason to live in such a scenario, you've whole-heartedly bought into Capital's propaganda, and frankly, lack imagination.
You're always free to set your own goals, and pursue them as you see fit. Hell, look at the leisure rich: They're liberated from work already. Their lives don't seem particularly miserable.
I have no objection to eliminating some toil, like working 9-5 in an office. That's indeed bullshit. And of course, I could find reasons to live. Going against AI would already be enough reason for me.
My post was more subtle than that anyhow. I don't advocate work for work's sake, especially working FOR some worthless corporation. But swamping the world with AI-created things will also make most leisurely creative activities difficult for humans to do. So even purposes you create will be difficult to share with other people.
I create things to express my own feelings, for my own satisfaction in the act of creation. Nobody else can do that for me, including an AI. So far I just share with people I know, and AI won't make that more difficult.
It may not make it more difficult for you, but AI will make it MORE difficult for other people to even make friends. Friendship and even family bonds are partially dependent on needing other people, and with advanced AI and other technologies, we are approaching a world where we need less people all the time.
AI also floods the world with a surplus of random content, which will make people less likely to appreciate what you share with them. Perhaps you are old enough to have developed a good network of people, which gives you some shielding against it, but for babies born today, they will have a much harder time developing such a network and they will not grow up right.
The people I share with can already find plenty of better stuff online. So what? They're friends with me so they care about seeing what I've been working on, and what I have to say.
I'm not sure what you mean by "needing other people." If you're talking about emotional need, I don't think AI really substitutes. If you mean pragmatic needs, then that's not what real friendships are based on.
When I go to see an obscure band playing in a rundown club I go for being there, not for the quality of their riffs or for the acoustics of the space (which I expect both to be bad). This can't be replaced by a future AI as it cannot be replaced by the current radio playing in my car, this is the human part mentioned above.
I think it might actually be beneficial for artists, in the long run. Assuming that AI will remove the need to work for a living[1].
I recently saw a “modern art” exhibition and one of the things on display was quite literally a black square in a white wall. That’s it. AI could have produced that long ago. But presumably the artist, and at least some of the people viewing it, got something out of this as a means of expressing “something” (I don’t know what, I clearly didn’t “get it”). If AI liberates artists from having to make money from their art, then they can focus on expression and “art for arts sake” without having to worry about appealing to whoever is likely to pay them.
Sure, the AI could create the same art, but it could create that black square too, long ago. That didn’t stop this particular artist.
[1] I don’t think it will, automation has already improved productivity and production a thousand fold over the past 100 years, yet we still have to work just as hard while a handful of rich people are richer than ever before, with more personal wealth than sone nations. More likely, AI will make a handful of rich people even richer still leaving the rest of us in the dust, probably being put to work on meaningless grunt work — don’t want to waste valuable AI time on flipping burgers now do we!
The black square may have been Malevich’s famous painting.[0] Even if you got nothing out of it, it has long since become part of the canon. A great deal of literature and later painting makes reference to it, so one benefits from seeing it in a museum and learning about its historical context regardless of what one thinks about the painting itself.
I tend to agree with you, but people like us need to face the fact that many people do live through work. I've seen it happen to a number of people where they are healthy, retire, and then their health goes downhill and they sometimes die. Depression rates can be terrible. If all we do for these people is say "you've whole-heartedly bought into Capital's propaganda, and frankly, lack imagination," we're going to have blood on our hands. I don't want to slow or halt the progress toward lack of scarcity, but we do need to figure this problem out and it won't be easy because millions of years of evolution have made us creatures who get satisfaction from accomplishing tasks that help us survive and thrive.
> Our goal should absolutely be to eliminate the necessity of work ... You've whole-heartedly bought into Capital's propaganda.
It is not just capitalism that demands hard work but even some challenges to capitalism. I am again reminded of a bit of 1960s Maoist literature translated from Chinese into English that I once found in a Chinatown bookshop: in discussing the Peking Man fossil, it claimed that what set Man apart from his ape forebears was that he had the capacity to labor. Those who did not wish to dedicate their lives to labor, the book claimed, were reverting to primitive ways.
It's safe to say the average HN reader has been exposed to more Capitalist propaganda than ML/Maoist propaganda, but it is a good point. Harping on the virtue of work is a pretty common motif between authoritarian regimes of all manner of political ideology. It's an easy way to make people do what you want them to.
I'm retired with enough money to hire someone to do any task I don't feel like doing, and buy anything I actually want (which isn't all that much). At first it was actually a bit depressing. Then I started focusing on tasks I enjoy doing myself, and challenges that nobody can do for me. Gratitude for my good fortune, and appreciation of the world's natural beauty, are also pretty helpful. Altogether it's a fantastic life and if AI and robots make it possible for everyone, I'm all for it.
AI certainly won't do that. They will make resource use even more efficient, promoting the death of the world's natural beauty. Do you think all this AGI shit will be for free? AI will want to consume everything natural to construct a disgusting technological system.
And can you really say that you would have appreciated getting to where you are, and FINDING your purpose if your WHOLE life were provided for by AI all along the way??? Sounds like you might have actually worked a little to get where you were. Imagine a life where you didn't have to do any of that....would you have the same WISDOM to appreciate where you are now??
I thought that Deep Blue and Alpha Zero would be the demise of Chess and especially online chess. Why play when a machine can trivially beat you, or worse, score how "correct" your moves are?
Your final statement is grossly misleading. People do mind. I actually follow the chess world.
It is much easier now to cheat at chess than it ever was, and it even happens at the grandmaster level. In some ways, the most recent controversy with Magnus Carlsen is due to computers and cheating. Moreover, many people cheat at online chess.
Finally, chess is becoming less interesting for sure, now that computers have introduced such a high level of mechanization in the study of it.
SURE, people STILL enjoy chess, but one could CERTAINLY argue that computers, Deep Blue, and Alpha Zero have made chess less interesting due to cheating and over-analyzing. That is why some people have even got to playing random chess (random starting position), because the overemphasis on highly-studied opening plays, especially by computers, makes the whole process a little too machine-like.
You would be amazed at the amount of unsolved problems that Intelligence as a mechanism helps solve. Goals would still be abound as that is the Consequence of maintaining a highly entropic system. It would be more interesting bluntly.
I certainly don't think solving problems is really helpful any more. It's become an industrialized enterprise towards mindless economic growth. We need to focus less on solving complex technical problems to make resource-use more efficient and focus more on sustainability, which certainly does not require AI.
But can help and be accelerated by such. As this has been the tune for a long time as things have been presented as being solved. A wild west is how you open back up marketplaces to solve these issues. Especially as those issues are fundamental towards handling a rapid rise of productivity/entropy. This is the challenge of now. And intelligence as a mechanism is an actionable route, versus waiting for people to change perspective. Especially as the question can be whether the current now is capable of even handling an increased amount of Entropy. In addition that rise of Productivity is Unavoidable.
I think humans will always create their own goals, drama, and social structure.
I look at Burning Man for an inspirational example of what a post-capitalist economy could look like; a bunch of people bringing cool art together, not for any monetary gain, but simply because it’s cool to watch people enjoying something you built.
Now, if AI gets better at doing even that (rather than just freeing us from toil by producing radical material abundance) then things could get fraught.
For me the most scary end state (that still involves humans in control and existing) is mainlining / wireheading on generated VR content that is in a feedback loop with brain scanners to maximize hedonic enjoyment. Plausibly this could displace our more-human desires and drives.
But I think there is a big gap between “automation frees us from toil” and “automated maximization of uniquely human art”.
To be FAIR, I started my original comment quoting the author of the article, who didn't say AI freeing us from toil would be a utopia. He specifically said, "provide for our every need". Big difference.
I see, that wasn’t how I read “wants” in that passage, but I agree it’s a valid interpretation. Maybe the author does intend to include everything up to the top of Maslow’s hierarchy of needs as “wants”.
Will AI meet our needs for love, friendship, and self-actualization? That’s more problematic. Though I could still see room for utopia if it does; would the world not be a better place if there was “universal basic friendship” and nobody was completely abandoned? In that world we might still prefer human companions, yet never fall into complete loneliness.
I’m not sure Burning Man today is a good example for post-capitalist idealism. Longtime veterans of the event have complained that Burning Man today (and for years now) is largely attended and driven by people who spend most of the year acquiring wealth through cutthroat capitalist means, and then only come to the event to blow off steam.
Good point, I should have clarified on time period, my experience was circa early 2010s, and I hear it’s more commercialized now.
I didn’t intend to claim that BM is actually solving these problems or going in the right direction, just that in a specific time and place and community, there was a glimpse of something very different. (I understand it’s a lot to do with community size and selection effects.)
back when we were scaling a saas offering in the late 2000s, we were told by some sales 'engineer' at a certain company well known in those days with lots of hype marketing (that died explosively a year later, big shock), that ML would solve all our engagement problems. It was hyped up, just like AGI is now. the reality? it has made a difference, no doubt, mainly in the form of incremental improvements at best. this feels eerily similar.
I've sat in countless pitches and meetings with bright-eyed founders touting the imminent rise of AGI. There's a palpable excitement around the topic, but from a purely business perspective, there's an equally strong air of naiveté.
I often wonder if proponents of AGI have truly considered its commercial implications. let's indulge the idea for a moment and say that it becomes a reality. how do you price it? how do you convince industries to adopt and trust something that mirrors human cognition, with all its opacity, when we still grapple with the biases and imperfections of simpler AI models?
add to this the astronomical R&D costs. the sheer capex required is a massive barrier. even with vc backing, the path to profitability seems tenuous at best. and in our current startup landscape and inflation reality, such ventures are money holes.
to me, AGI feels like childish pursuit of the shadows of sci-fi dreams. there's merit in pushing the boundaries of technology, but mistaking theoretical marvels for immediate commercial viability is a classic pitfall. for now, I'll reserve my enthusiasm until someone can convincingly bridge the chasm between agi's promise and its pragmatic business value.
> Because non-greedy online planning can be slow, offline planning can be used to generate a policy that maps any state to the best action to take. The resulting policy can be run very fast at execution time; no matter what state the agent ends up in after executing an action, the next action to execute is available as quickly as π can be queried.
The article claims this is insufficient for AGI, but to me it sounds a lot like Kahneman's distinction between fast and slow thinking. Slow thinking is the offline planning, the trained network does the fast thinking. If humans work that way, then a human-level AGI probably can too.
The article has a flavor of trying to find a role for good old fashioned AI, with planning and search, in this new world. It points out that to some extent large language models can plan, but only for things where the steps required are roughly described on the Internet. It's basically splicing together bits of known sequences of steps. That can handle many cases. As usual, the problem is lack of evaluation - totally bogus plans can emerge unnoticed. This is the lack of a model problem.
The big lack remains manipulation in unstructured situations. Although that Google project mentioned a few days ago has some promise. They train on simple manipulation tasks with a force feedback teleoperator, and then use an LLM for the higher level task sequencing. But the simple manipulation tasks remain rather simple. Too simple to, say, assemble flat-pack furniture, let alone repair a car. LLMs just don't seem to be relevant to such non-language tasks.
I say the biggest missing link remains a complete lack of modeling whatsoever for some time domain. Representation of time delay between neurons and other elements can encode a lot of information. As it currently stands, everything in popular AI architectures is evaluated in a completely timeless manner.
I believe very deeply that spiking neural networks are the path to AGI. I also believe that the amount of compute ultimately required to realize practical SNNs is going to turn out to be shockingly small.
I thought there were NN architectures where some layers’ outputs are sent to layers not directly in front of them. Wouldn’t that essentially capture some simulation of a time delay?
Is there any inherent delay modeled between any of the layers in your hypothetical NN architecture (assuming non-SNN)?
If there are no actual time delays modeled within the network, then whatever it evaluates is representative of the outside state at some instant in time.
I think the problem is that SNNs aren't suited to standard von neuman architecture. I'm in total agreement. That amount of power and compute thrown at language models, and ML in general is shocking relative to the capabilities. For GPT4, it's staggering. Probably on the order of KWHs in power alone for basic tasks.
I think a properly modeled architecture would look something like the actor model represented in hardware, where each actor is basically a small extremely power efficient cpu but unfortunately you wouldn't be able to pull that off the shelf.
> I think the problem is that SNNs aren't suited to standard von neuman architecture.
This is where we begin to disagree.
If you are content with moving from actual real time to synthetic time, all of the potential frustrations fall away again. Think about online vs offline rendering in computer graphics. Theoretically, you could render all of Toy Story on a RasPi4.
Most of us would be willing to wait days for answers, assuming the batch size is acceptable and the answers are correct.
Philosophy of mind. It is impossible to create an artificial version of something that we cannot yet define. People need to dial back the hype and take some more time to consider why we think and what constitutes a 'sense of self'. Spending some more time figuring out how biological brains 'store data' wouldn't hurt either, but impressive MRI party tricks (with monetization paths) seem to win the day every time.
Side note is that language models are extremely inefficient for the benefit they provide. Most humans don't need to ingest terabytes of domain specific data in order to engage in basic reading comprehension and reasoning.
I'd argue they excel at things like designing molecules, where we don't have the mental capacity or speed to ingest as much information as we would need to reason quickly, but the amount of information and processing necessary to replace a secretary is laughable.
A child doesn’t process “data” in the form of a sequence of tokens. A child associates visual objects with similar traits to pronounced words, heard from their parents. It also acquires a basic feeling of temporal sequentiality and physical causality by throwing toys, for example. It learns the rules of the language grammar by subconsciously noticing the basic structure of sentences with objects, verbs and qualities.
> A child doesn’t process “data” in the form of a sequence of tokens. A child associates visual objects with similar traits to pronounced words, heard from their parents.
Objects and words are tokens. At least, objects and words are not fundamental types of things. Before they are tokens, they exist only through light and sound waves that we perceive (and other sensations such as touch, smell). Our eyes and ears need to process this information before it becomes objects and words.
No contest about the 'bitrate' for eyes and ears, but we don't have to feed kids the entire library of congress in order for them to reason about text. This is what gives me cold feet about the current wave of LLMs. Where is this going? We'll have something that people will dub 'AGI'. It will seem intelligent and may or may not be catastrophic, but whether or not it actually has a sense of self will be an open question, second only to the fact that it requires the power output of Argentina to survive?
Seems like a dumb hack, not an advancement indicative of a thorough understanding of the problem.
If and when, then maybe we can use it to bio-engineer our way to a better future?
I think you are plainly wrong here. Imagine a thought experiment where you'd plug newborn's ears and cover eyes until they are 6yo and see how much data you'd have to feed to them before they can reason about text from this point. I imagine it will be in hundreds of terabytes or even petabytes.
The idea that children learn from little is naive.
I did not state that children learn from little. I did state that they are able to achieve impressive outcomes quickly with very small amounts of domain specific knowledge (movement not included).
Our best language models at this point appear to be narrow and deep where the child model is wide and shallow. For example, how many pictures of apples do I need in order to train a robot to recognize apples? How many apples (or representations thereof) does a child need to see to identify an apple? I won't argue about image density, but human eyes aren't that high resolution and people don't seem to have excellent high resolution multi-frame memory.
This just makes we wonder if we should be spending more time working on how we utilize our wide but shallow data corpus rather than playing I can't believe it's not statistics with hugely inefficient and focused data sets.
You are wrong again. There's no difference between ChatGPT and a child in that regard. That's like half of the point of GPT's zero shot performance on the tasks that involve made up language that you describe directly in the query.
You are doubly wrong because somehow for a robot you are counting pictures of apples, and for a child you are counting distinct apples. The answer to "how many pictures of apples does child need to see" is somewhere in thousands. Remember, brain does an equivalent of 24 fps, so a child holding an apple for 5 seconds gets an equivalent of over 100 pictures.
> Philosophy of mind. It is impossible to create an artificial version of something that we cannot yet define
Not true. It's a genuine fear of mine that we will create artificial consciousness without knowing we've done so, and doom countless minds to endless suffering before we realize the err.
I think, with regard to AGI, like a lot of A.I. research, implementation will come before explanation.
This is all very superstitious. Much of what's said about AI in tech circles has a superstitious, irrational character. There's a very obvious absence of any real philosophical analysis or depth of understanding of the subject matter, only a crude, superficially materialistic, hand-wavy sort of assumption that any of this makes sense.
The idea that computers and intelligence have anything to do with one another, in the manner they are often assumed in the context of AI to have anything to do with one another, is profoundly flawed from the start. A formal system by definition lacks features that are characteristic of intelligence (semantics and the ability to abstract from particulars), and physical computers, strictly speaking, aren't a natural kind nor do they, again strictly speaking, even embody or deal with formal systems per se anyway. Computation is a mathematical construct that is simulated with physical machines, and this model is completely agnostic about the physical means by which it is simulated. You could use wooden gears if you want. The assignment between physical state transformation and a computation is entirely conventional and in the eye of the beholder.
Now, if you want to define "artificial intelligence" as the simulation of intelligence, then I could concede a loose version of this claim, but it would be closer to a marketing claim than a scientific or philosophical one. Practically speaking, such "AI" could still have major consequences as a sophisticated means of automation, but theoretically, there's no intelligence involved except on the part of the artificer.
If you are prepared to worry about your computer becoming conscious, then you should equally be worried about your cup of coffee become conscious as you drink it and subject the contents to the torments of your digestive system. There's no reason to fetishize those artifacts we call computers.
> There's a very obvious absence of any real philosophical analysis or depth of understanding of the subject matter
Integrated Information Theory seems like a lot of thought has gone into it, and it would ascribe a good deal of consciousness to some of these systems.
> A formal system by definition lacks features that are characteristic of intelligence (semantics and the ability to abstract from particulars)
And quantum mechanics would have you believe Newtonian mechanics is nonsense, but there are such things as emergent properties, and rules changes as scales allow more complex compositions of simple rules.
> If you are prepared to worry about your computer becoming conscious.
That feels a bit disingenuous. You're dressing down some of the worlds largest supercomputers as "my computer". I'm not worried about my M1 Mac suffering. I'm worried about an emergent mind composed of thousands and thousands of computers working together to simulate intelligence suffering.
Technological progress does not proceed from definitions, new concepts can unlock new domains of thought that can in turn yield knowledge and technology.
Humans and most other things we consider to be alive - even viruses - have an inherent motive to reproduce. Evolution is a tautology: that which survives survives. If you stop reproducing, over time you tend to stop existing. LLMs have a clock, and no motive whatsoever. They rely entirely on us to produce them so far.
They can do what, how, and where, but they lack a "why".
82 comments
[ 3.0 ms ] story [ 152 ms ] threadIt's just a matter of time.
It seems like people want it to mean not just general utility but a full simulation of a person (which then sometimes automatically becomes godlike). But they don't say that.
Anyway, decent article. Reminds me of what LeCun has been talking about. https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/
> Online non-greedy planning technologies that can operate in the presence of a priori unseen tasks, states, and actions.
> A world model technology that robustly predicts state-action-state transitions for all states and actions that might be encountered while performing any task in any environment.
People use greedy strategies all the time, and the intractability of considering 'all states and actions' is the Frame Problem. It is lack of generality that avoids the Frame Problem, and makes planning tractable for animals.
In my view, AGI is an anti-goal. Specialization is what makes intelligent systems feasible. On the other hand, there are technologies like DNNs, gradient descent strategies, even computation itself, that are generally useful for application to specialized intelligent systems.
By analogy, all animals are made of cells, and developed by evolution. But there's no General Animal that can do what all the others can, even if they share core technology.
edit: I wish I hadn't given the analogy. Quibbling about it seems to be irresistible, and has has distracted from my point here.
Humans can artificially travel fast, fly, swim deep, drill deep, move heavy things, kill very efficiently, etc.
This is not a utopia, but a juvenile and unthinking definition of utopia, or a dystopia in disguise. Most people thrive on some sort of challenge to at least do basic tasks for themselves. A world where AGI fulfils every need is more like being in a senior's home where everyone does everything for you.
It sounds good at first but is actually a nightmare where your autonomy is stripped by the lack of goals to work towards.
Immense surplus and easy access to everything all the time means human beings will no longer need each other much either, so it means a world full of people uncaring towards each other. Fulfilling any want without hard work to get it means becoming effectively like a bacteria, consuming every resource in sight, transforming us into even more of a plague than we already are.
I certainly would not want to live in such a world.
It's just a depressing viewpoint imo
And now we're trying to use AI to become even more resource-intensive, burning up endless energy to be even more powerful consumers...give me a break. It's sick. As individuals there's nothing seriously wrong with most people but en masse with modern technology, that is exactly what we are, a plague.
Who has that as a goal?
One thing that the market will reward handsomely is bringing costs down. When costs go down, people consume more. If you're 40 or older, think about how many computers your family owned in your house growing up. We owned some hand-me-downs, and purchased 1 computer in my first 15 years. We bought about 4 in the next 3 years because they were advancing rapidly, but more importantly, because they had become much cheaper than they had been 5-10 years before. Today, many families might have a laptop for each person, replaced every 3-4 years.
One of the first things an AGI would be put to work doing is manufacturing, as we're already finding that it's hard to get people to do these stressful manufacturing jobs except in places where they're really desperate. Witness the moving of jobs from US -> China -> Southeast Asia.
There is no good without humans. Any philosophy that asserts otherwise fails Camus's fundamental question.
It's not just us. Earth might be fine but a great many species won't
>Plants and animals kill each other in a ruthless cycle of violence, too. We're just really good at it.
It's not at all comparable lol. Not in scale or function.
I actually think that the disadvantages of having few goals would become immediately obvious to almost everybody, and a whole new type of skill / creative / athletic based "economy" might emerge. But with lower stress and lower stakes
As opposed to what? Trust your hunch and halt technological progress. Wrap it up folks, we're all done with R&D now.
Your concern about purpose strikes me as odd. When people derive purpose from their work, it's often because they feel they're playing a role in a community. To deprive people of purpose would require that we make communities fully automated and impersonal. I don't think anyone wants or is proposing that.
Absent a requirement to spend one's time earning money, some people would probably degenerate into sloths, but plenty of people are otherwise motivated by self-expression, socialization, competition, etc. People would still interact and do things because people like interacting and doing things.
It is really not about wanting or proposing anything. It is about the effect of technology and what it will mean. Don't you think that communities are already becoming a bit more impersonal and automated? With ChatAI replacing customer service, self-checkout aisles, online dating, etc...
It seems to me that communities are indeed becoming more and more impersonal, especially if you compare them to communities thousands of years ago.
Definitely these interactions have become less personal, but aside from online dating, which is its own can of worms, the interactions are all transactional. I actually much prefer to interface with a predictable machine when purchasing things or making an RMA request or whatever. I don't want to stop to chit chat or worry about maintaining a relationship with the village cobbler's son to ensure he'll do his best fixing up my field boots.
The ongoing disentanglement of socialization and daily necessities is liberating. It allows one to associate with others by choice. I'm not a huge fan of the tools we have for this sort of voluntary community building at present, but I see no reason why they couldn't be improved, especially if people were able to spend their time and skill doing so free of profit motives.
Or like being mega-wealthy.
There are plenty of folks at senior homes enjoying the ability to try new things they didn't previously have time for.
You're always free to set your own goals, and pursue them as you see fit. Hell, look at the leisure rich: They're liberated from work already. Their lives don't seem particularly miserable.
My post was more subtle than that anyhow. I don't advocate work for work's sake, especially working FOR some worthless corporation. But swamping the world with AI-created things will also make most leisurely creative activities difficult for humans to do. So even purposes you create will be difficult to share with other people.
AI also floods the world with a surplus of random content, which will make people less likely to appreciate what you share with them. Perhaps you are old enough to have developed a good network of people, which gives you some shielding against it, but for babies born today, they will have a much harder time developing such a network and they will not grow up right.
I'm not sure what you mean by "needing other people." If you're talking about emotional need, I don't think AI really substitutes. If you mean pragmatic needs, then that's not what real friendships are based on.
I recently saw a “modern art” exhibition and one of the things on display was quite literally a black square in a white wall. That’s it. AI could have produced that long ago. But presumably the artist, and at least some of the people viewing it, got something out of this as a means of expressing “something” (I don’t know what, I clearly didn’t “get it”). If AI liberates artists from having to make money from their art, then they can focus on expression and “art for arts sake” without having to worry about appealing to whoever is likely to pay them.
Sure, the AI could create the same art, but it could create that black square too, long ago. That didn’t stop this particular artist.
[1] I don’t think it will, automation has already improved productivity and production a thousand fold over the past 100 years, yet we still have to work just as hard while a handful of rich people are richer than ever before, with more personal wealth than sone nations. More likely, AI will make a handful of rich people even richer still leaving the rest of us in the dust, probably being put to work on meaningless grunt work — don’t want to waste valuable AI time on flipping burgers now do we!
[0] https://en.wikipedia.org/wiki/Black_Square_(painting)
It is not just capitalism that demands hard work but even some challenges to capitalism. I am again reminded of a bit of 1960s Maoist literature translated from Chinese into English that I once found in a Chinatown bookshop: in discussing the Peking Man fossil, it claimed that what set Man apart from his ape forebears was that he had the capacity to labor. Those who did not wish to dedicate their lives to labor, the book claimed, were reverting to primitive ways.
And can you really say that you would have appreciated getting to where you are, and FINDING your purpose if your WHOLE life were provided for by AI all along the way??? Sounds like you might have actually worked a little to get where you were. Imagine a life where you didn't have to do any of that....would you have the same WISDOM to appreciate where you are now??
But somehow people don't mind...
It is much easier now to cheat at chess than it ever was, and it even happens at the grandmaster level. In some ways, the most recent controversy with Magnus Carlsen is due to computers and cheating. Moreover, many people cheat at online chess.
Finally, chess is becoming less interesting for sure, now that computers have introduced such a high level of mechanization in the study of it.
SURE, people STILL enjoy chess, but one could CERTAINLY argue that computers, Deep Blue, and Alpha Zero have made chess less interesting due to cheating and over-analyzing. That is why some people have even got to playing random chess (random starting position), because the overemphasis on highly-studied opening plays, especially by computers, makes the whole process a little too machine-like.
I look at Burning Man for an inspirational example of what a post-capitalist economy could look like; a bunch of people bringing cool art together, not for any monetary gain, but simply because it’s cool to watch people enjoying something you built.
Now, if AI gets better at doing even that (rather than just freeing us from toil by producing radical material abundance) then things could get fraught.
For me the most scary end state (that still involves humans in control and existing) is mainlining / wireheading on generated VR content that is in a feedback loop with brain scanners to maximize hedonic enjoyment. Plausibly this could displace our more-human desires and drives.
But I think there is a big gap between “automation frees us from toil” and “automated maximization of uniquely human art”.
Will AI meet our needs for love, friendship, and self-actualization? That’s more problematic. Though I could still see room for utopia if it does; would the world not be a better place if there was “universal basic friendship” and nobody was completely abandoned? In that world we might still prefer human companions, yet never fall into complete loneliness.
I didn’t intend to claim that BM is actually solving these problems or going in the right direction, just that in a specific time and place and community, there was a glimpse of something very different. (I understand it’s a lot to do with community size and selection effects.)
I've sat in countless pitches and meetings with bright-eyed founders touting the imminent rise of AGI. There's a palpable excitement around the topic, but from a purely business perspective, there's an equally strong air of naiveté.
I often wonder if proponents of AGI have truly considered its commercial implications. let's indulge the idea for a moment and say that it becomes a reality. how do you price it? how do you convince industries to adopt and trust something that mirrors human cognition, with all its opacity, when we still grapple with the biases and imperfections of simpler AI models?
add to this the astronomical R&D costs. the sheer capex required is a massive barrier. even with vc backing, the path to profitability seems tenuous at best. and in our current startup landscape and inflation reality, such ventures are money holes.
to me, AGI feels like childish pursuit of the shadows of sci-fi dreams. there's merit in pushing the boundaries of technology, but mistaking theoretical marvels for immediate commercial viability is a classic pitfall. for now, I'll reserve my enthusiasm until someone can convincingly bridge the chasm between agi's promise and its pragmatic business value.
The article claims this is insufficient for AGI, but to me it sounds a lot like Kahneman's distinction between fast and slow thinking. Slow thinking is the offline planning, the trained network does the fast thinking. If humans work that way, then a human-level AGI probably can too.
The big lack remains manipulation in unstructured situations. Although that Google project mentioned a few days ago has some promise. They train on simple manipulation tasks with a force feedback teleoperator, and then use an LLM for the higher level task sequencing. But the simple manipulation tasks remain rather simple. Too simple to, say, assemble flat-pack furniture, let alone repair a car. LLMs just don't seem to be relevant to such non-language tasks.
1. How much space would it take up? GBs?
2. Would we have the computing power to run "it", say on a top of the line consumer laptop?
I believe very deeply that spiking neural networks are the path to AGI. I also believe that the amount of compute ultimately required to realize practical SNNs is going to turn out to be shockingly small.
If there are no actual time delays modeled within the network, then whatever it evaluates is representative of the outside state at some instant in time.
I think a properly modeled architecture would look something like the actor model represented in hardware, where each actor is basically a small extremely power efficient cpu but unfortunately you wouldn't be able to pull that off the shelf.
This is where we begin to disagree.
If you are content with moving from actual real time to synthetic time, all of the potential frustrations fall away again. Think about online vs offline rendering in computer graphics. Theoretically, you could render all of Toy Story on a RasPi4.
Most of us would be willing to wait days for answers, assuming the batch size is acceptable and the answers are correct.
Side note is that language models are extremely inefficient for the benefit they provide. Most humans don't need to ingest terabytes of domain specific data in order to engage in basic reading comprehension and reasoning.
I'd argue they excel at things like designing molecules, where we don't have the mental capacity or speed to ingest as much information as we would need to reason quickly, but the amount of information and processing necessary to replace a secretary is laughable.
How much data do you think a 6 year old has processed by the time they read a book? Eyes and ears are pretty high bitrate input devices.
Objects and words are tokens. At least, objects and words are not fundamental types of things. Before they are tokens, they exist only through light and sound waves that we perceive (and other sensations such as touch, smell). Our eyes and ears need to process this information before it becomes objects and words.
Seems like a dumb hack, not an advancement indicative of a thorough understanding of the problem.
If and when, then maybe we can use it to bio-engineer our way to a better future?
The idea that children learn from little is naive.
Our best language models at this point appear to be narrow and deep where the child model is wide and shallow. For example, how many pictures of apples do I need in order to train a robot to recognize apples? How many apples (or representations thereof) does a child need to see to identify an apple? I won't argue about image density, but human eyes aren't that high resolution and people don't seem to have excellent high resolution multi-frame memory.
This just makes we wonder if we should be spending more time working on how we utilize our wide but shallow data corpus rather than playing I can't believe it's not statistics with hugely inefficient and focused data sets.
You are doubly wrong because somehow for a robot you are counting pictures of apples, and for a child you are counting distinct apples. The answer to "how many pictures of apples does child need to see" is somewhere in thousands. Remember, brain does an equivalent of 24 fps, so a child holding an apple for 5 seconds gets an equivalent of over 100 pictures.
Too bad they are not saying how long it took to get to the recognition.
Not true. It's a genuine fear of mine that we will create artificial consciousness without knowing we've done so, and doom countless minds to endless suffering before we realize the err.
I think, with regard to AGI, like a lot of A.I. research, implementation will come before explanation.
The idea that computers and intelligence have anything to do with one another, in the manner they are often assumed in the context of AI to have anything to do with one another, is profoundly flawed from the start. A formal system by definition lacks features that are characteristic of intelligence (semantics and the ability to abstract from particulars), and physical computers, strictly speaking, aren't a natural kind nor do they, again strictly speaking, even embody or deal with formal systems per se anyway. Computation is a mathematical construct that is simulated with physical machines, and this model is completely agnostic about the physical means by which it is simulated. You could use wooden gears if you want. The assignment between physical state transformation and a computation is entirely conventional and in the eye of the beholder.
Now, if you want to define "artificial intelligence" as the simulation of intelligence, then I could concede a loose version of this claim, but it would be closer to a marketing claim than a scientific or philosophical one. Practically speaking, such "AI" could still have major consequences as a sophisticated means of automation, but theoretically, there's no intelligence involved except on the part of the artificer.
If you are prepared to worry about your computer becoming conscious, then you should equally be worried about your cup of coffee become conscious as you drink it and subject the contents to the torments of your digestive system. There's no reason to fetishize those artifacts we call computers.
Integrated Information Theory seems like a lot of thought has gone into it, and it would ascribe a good deal of consciousness to some of these systems.
> A formal system by definition lacks features that are characteristic of intelligence (semantics and the ability to abstract from particulars)
And quantum mechanics would have you believe Newtonian mechanics is nonsense, but there are such things as emergent properties, and rules changes as scales allow more complex compositions of simple rules.
> If you are prepared to worry about your computer becoming conscious.
That feels a bit disingenuous. You're dressing down some of the worlds largest supercomputers as "my computer". I'm not worried about my M1 Mac suffering. I'm worried about an emergent mind composed of thousands and thousands of computers working together to simulate intelligence suffering.
Humans and most other things we consider to be alive - even viruses - have an inherent motive to reproduce. Evolution is a tautology: that which survives survives. If you stop reproducing, over time you tend to stop existing. LLMs have a clock, and no motive whatsoever. They rely entirely on us to produce them so far.
They can do what, how, and where, but they lack a "why".
You cannot build an accurate world model without sensory input and the ability to gather data-driven descriptors of the physical world.