> Study Deep Blue’s 1997 defeat of world chess champion Garry Kasparov, and your excitement about how quickly this technology would take over other cognitive work would wane as you learned just how much brute human force went into fine-tuning the software specifically to beat Kasparov.
That was ~26 yeas ago. In the intervening 26 years humans have been curb-stomped so hard at chess that certain microwaves might be competitive against human grandmasters. Anyone who doesn't think that was an exciting moment for AI research has impossible standards, we're living in a tiny window of existence where humans were at the top of the intellectual food chain and are now dropping to also-ran status along with the rest of the meat-based animal kingdom.
All these moments are the real deal. The idea that somehow this particular step change is even unexpected is missing the forest for the trees. The trends that have been observed since the beginning of computers are chugging away so quickly the whole process may well happen in a single generation.
Even AlphaGo had a human moving the pieces, and nobody would say that was evidence of any 'lack' in the AI.
It's just an interfacing issue, engineering will eventually solve.
Edit> What I was getting at. Take Robots as the physical interface layer for the AI. AI advancements are currently outpacing advancements in Robotics. So potentially AI could use real humans as the 'hands and feet' to do things in the physical world, by giving directions. Something will be needed to bridge the gap until Industry can be done completely by robots. Until then, AI wont be able to manufacture itself.
This is the scary scenario. What if AI does out pace humans, but the physical side does not, we don't succeed in making vast quantities of easy to build and maintain robots. Thus to keep industry going, the AI does need humans, and it gives the directions.
If you have ever worked in a modern warehouse, it is very evident, the humans are just following instructions from the system. Now scale that up to a manufacturing facility, it is already happening.
Everybody with an ear-piece or 'phone' type device, following instructions.
> The problem here isn't automation, it's power. The workers whom the robots could benefit are instead harnessed to the robot to the benefit of the shareholders.
> Workplace democracy, AKA unionization, AKA the thing Amazon has pulled out every dirty trick to prevent, is the difference between centaur utopia and reverse-centaur dystopia.
Again, your information is tremendously out of date. AlphaZero crushes grandmasters and had no human training whatsoever as to board evaluation functions.
The are microwaves fully in on the curb-stomping https://www.tomsguide.com/news/ges-kitchen-hub-is-the-smarte...
My back of the envelope suggests most digital microwaves with a smart defrost but no connectivity are probably more in the 2000 ELO range (i.e. can beat most players but not grandmasters). Mainly due to not having enough storage for large opening/endgame books.
No, no it isn't. It's a very particular violent act designed to maximize ongoing physical pain and social disadvantage in the victim. Sort of like throwing sulfuric acid at someone's face.
Don't stop with merely hearing them; think about them.
> [Hobbes] even, through sheer force of imagination, was able to outline the main psychological traits of the new type of man who would fit into such a society and its tyrannical body politic. He foresaw the necessary idolatry of power itself by this new human type, that he would be flattered at being called a power-thirsty animal, although actually society would force him to surrender all his natural forces, his virtues and his vices, and would make him the poor meek little fellow who has not even the right to rise against tyranny, and who, far from striving for power, submits to any existing government and does not stir even when his best friend falls an innocent victim to an incomprehensible raison d'etat.
Policing language sounds a bit Orwellian (insert doublespeak quote). People like to use various metaphors. It doesn't mean they literally want to curb stomp a competitor.
> Policing language sounds a bit Orwellian (insert doublespeak quote).
And that only applies to criticizing phrases like "killing it", without even demanding it to not be used -- but not to bringing out Nineteen-Eightyfour? And doublespeak? No, doublethink means holding two contradictory things to be true at the same time, doublespeak is the verbalization of that. You're thinking of wrongthink, and either way you are doing what you criticize. Instead of saying "this is horrible" you say "this is dystopian", it changes nothing.
More importantly, your reply contains nothing specifically applying to my comment. Maybe you missed the point, which is that people delude themselves to compensate their lack of agency, being restricted to partake in a rat race that will leave them empty-handed in the end -- all of which Hannah Arendt explains in the context, but I didn't want to post a wall of text. That they don't actually mean what they say, because what they mean (and do) would be to pitiful to say without such flourish, and that they aren't actually violent and powerful, but "poor meek little fellows", is precisely the point.
It would be more accurate to say that some people use it as a simple turn of phrase without considering what it means. Language is made mostly of metaphor and analogy. Explaining something by comparison to something that is already understood is a technique that seems to fit our model of thought.
Over time the original meaning may fade or be forgotten, and the residual symbol in the language becomes the thing itself. Etymology is full of examples.
But that much history has not yet passed since the introduction of the term "curbstomping" and so using it is still a choice that conveys some meta-meaning about content and style.
As mentioned further up in the thread it is unnecessarily violent for conveying the meaning in the comment.
everything so far looks to me like improvements on the pocket calculator, bearing in mind that an order of magnitude difference in scale really still is a difference of kind, they're still orders of magnitude improvements on pocket calculators (which admittedly is something of a different kind). There's a very important particular kind of human intelligence that machines can't seem to do at all, like Einstein discovering general relativity or Fleming discovering antibotics...which sounds like I'm cherry-picking human intelligence at its best, but I'm really pointing out a kind of information synthesis on the basis of reacting to unexpected information.
I have trouble articulating it, but I can see that LLMs and chess engines don't do the slightest hint of it.
I'll be concerned when machines show the slightest hint of it, but we're playing a ball game in a sphere they can't even touch yet.
I'm genuinely continually boggled that people are not seeing this difference and feeling threatened by machine "intelligence", although they are encroaching on something thought uniquely human, and I don't think the brain is doing some kind of information processing that can't be done in any other medium, but the number of connections and kind of processing in the human brain may make it hard with current technology. The number of connections in the brain is staggering, and we certainly can't simulate anything of its scale on the scale of computer we have now, so I don't know what it will take to get any bootstrap on the kind of intelligence computers aren't doing.
I mean you might think AI can't currently affect humanity but it really does. A lot of people are now using AI to write code and to answer questions they probably should be asking a doctor or lawyer. A lot of people seem to think it's some all knowing and psychic thing and will take its advice blindly.
Where on the S-shaped curve are we? I am curious if one day the paradigm that brought forth the current chatbots has to be abandoned because it's reached his limits, and a new approach will replace it.
But certainly it has inspired the field of AI massively and I expect many new developments which will lead is into a future that resembles sci-fi books and movies.
I am also interested in what good sci-fi will look like in the future? It looks like "the singularity" is defined by the limits of our imagination. We cannot imagine what comes after it, so this means we cannot write sci-fi on it, right?
Because exponentials will eventually break the laws of physics.
For instance there is a relationship between information and energy, so a system processing an exponential amount of information will need an exponential amount of energy, which the universe will eventually run out of.
At this point we know very little and we have to leave it to the AIs to determine if physics has any limits.
That's a tall order. Despite all the fancy news headlines, which I must admit gave me existential anxiety at some point, it seems that AI still can't truly make a discovery. Also I really don't know if we know little or a lot, how can we really tell ? It could be argued we actually know a lot because it is ridiculously hard to make an actual big breakthrough in physics.
Yeah it sure would be convenient if this is the one time in all of nature that a curve that looked like it was going exponential before suddenly conforming to all past trends and becoming sigmoid was actually exponential after all because you really really want it to be.
Tangent: It has always annoyed me that a sigmoid curve does not look like a sigma, and it doesn't really look much like an S either. Just the middle part of an S. Am I missing something?
Old school handwriting, probably [0]. When people had to squiggle the glyphs on paper they had all sorts of weird and wonderful ways of scribing. Consider that the integral symbol is actually an S.
For LLMs and generative AI we’re likely at the start of the curve. Things take longer than you expect in the short-term and less time than you expect in the long-term.
After this hype cycle ends, things will probably bump along slowly for a decade or so, then suddenly you’ll look up and they’ll be everywhere, and you’ll wonder “when did that happen?”
At least that’s the way it felt for other technologies to me. Internet, Bluetooth, cell phones, etc.
> After this hype cycle ends, things will probably bump along slowly for a decade or so, then suddenly you’ll look up and they’ll be everywhere, and you’ll wonder “when did that happen?”
The reason this is incorrect is that “AI” isn’t one capability. It’s a huge swath of capabilities under the umbrella of “automate X”. All of them have been enabled by software and hardware improvements over the last decades. The underlying technologies aren’t that new in many cases either, it’s just the scale that’s different.
After we get bored of one huge new capability, another comes along. A year ago everyone was talking about automatic image generation. Now it’s automatic text generation. There are plenty of other things we’d like to automate. So the hype train is likely to continue for quite a while.
- Extrapolating the last couple of years of progress.
- Scientific studies of LLM quality scaling in proportion to input data size, parameter count, and total training compute ops.
- How much more training data is available.
- Reasonable budgets, especially the cost-efficiency of inference, which appears to be more limiting than training, especially for long-term usage and profitability.
- Efficiencies such as 4-bit quantization, algorithmic improvements, etc...
Based on the above, there is at least a factor of 10x scaling available in many directions within a year or two, but not likely all at once. E.g.: 10x context window size at the same intelligence level? Sure! 10x inference speed? Doable! 10x cost reduction? Coming soon!
All at once? Not yet, and not for a while. Everyone is hardware constrained, and demand is pushing up prices and limiting the training scale maximums.
Maximum intelligence is much harder to predict. The current generation of AIs are trained on human-authored text as their input data. They're trained to predict that text. That means that they're "blurry JPEGs of the Internet". More training might make them "sharper JPEGs", but not necessarily smarter because the Internet didn't get smarter.
They'll better model humans, but they'll still be modelling humans, not superhumans.
That can be fixed with self-training, etc... but that will take a lot longer. I'm guessing 5-10 years, which is in line with the predictions of AI futurists that seem to know what they're talking about.
It really stands to reason that at some point these systems will just be "too smart" and we won't find any value in something that much more intelligent. It's like a 4 year old trying to understand a seasoned professor explaining some concept, we just won't get it.
Imagine a hypothetical scenario where some ultra-intelligent beings delivered a book with all the answers humans ever wanted, but we needed an IQ of 500 t understand it. This IMO is where we're heading.
There might be some pretty sweet low hanging fruit to attain along the way, but at some stage, we'll hit a ceiling and the only way forward would be to augment our own intelligence.
That's why alignment is such a huge issue - the immediately obvious and tempting way to "solve" that problem is to give the machines control.
The child-parent analogy stretches to describe it. A young child has an extremely fuzzy picture about what is going on - how did this house get here, why is there food in the fridge, why is the food wrapped in this clear thing, why does Dad disappear for a few hours every day, etc. At a young enough age they're not even thinking about it.
Yet it all happens anyway and the child is provided for. Even in the best case of perfectly aligned AI, that's our future if we allow it to happen (I hope we don't...)
From what I've seen, the increases in LLM intelligence is not that type of smart. On the contrary, the better LLMs write in a more neutral and clear tone, a bit like a Wikipedia article.
Actually, Wikipedia articles are already sort-of a "distillation of the Internet" into a type of average. By having many people edit them, the individual personal quirks are smoothed out, leaving a type of average-of-humans result.
This reminds me of the "average faces" project about a decade ago where they blended photos of random people together. The results were often very attractive, because it's the small flaws that make normal people unattractive. The blending of many samples smooths out the flaws, resulting in a "perfect" but ordinary face. Not some post-human face, but a perfectly human face.
That's what I see LLMs doing: as smart as us, but no smarter. Great breadth certainly, but not greater depth.
That's kind of hard to say. You don't know how long the curve is until you're somewhat past halfway, unless you can very carefully measure progress.
That's easy with something like transistor size. That's hard with AI, because we can't put a number on how good it is. (The model size is probably not a good number for measuring AI progress.)
This is the Real Deal, but not in away you would imagine.
So far "nigerian email spam" was easy to recognize. Grammar mistakes, bad translation, very bad wording... But now every fringe group, can generate gigabytes of text and arguments that make sense! This is the end of debate, consensus and interactive communication, as we know it!
Similar moment was in Africa in 1960, after they got AK45 and other cheap weapons. It was easy to dominate Congo River Basin with a few Maxim machine guns mounted on boats, when opponents had only spears. But when every tiny village gets comparable fire power, there are no colonies!
> So far "nigerian email spam" was easy to recognize. Grammar mistakes, bad translation, very bad wording... But now every fringe group, can generate gigabytes of text and arguments that make sense!
"nigerian email spam" contains grammar mistakes, bad translation, vary bad wording... by design. To weed out those unlikely to end up sending money.
I don't know how you can make an argument that makes sense about being chosen by royalty of a country to help them in some way.
If someone is running an actual 419 scam in crypto, probably yes; they only want the really easy marks (because there's a non-zero incremental cost to each person engaging so they only want the ones who are likely to convert). Most crypto scams are either pump and dump schemes or some sort of Ponzi, though, and for those it really doesn't matter; there's no incentive to cut the number of participants.
> "nigerian email spam" contains grammar mistakes, bad translation, vary bad wording... by design. To weed out those unlikely to end up sending money
this theory is popular, but ignores the simple fact that mostly it weeds out the 90% of people whose email provider offers spam filters which are entirely uncorrelated with gullibility, as well as how low-risk, high reward email exchanges with the tiny fraction of people that bother to reply are to anonymous scammers in faraway low income countries.
I agree LLMs aren't that helpful though: the better scammers are able to generate leads even more easily through copy/pasting other people's classified ads or genuine business enquiries. And yes, having fooled a relatively smart person with someone else's prose into sincere interest in doing business with them, they then reply with poor English, unsettling disinterest in anything other than the money and repeatedly reassuring you it's safe, and a payment proposal that makes no sense (not as a filtering technique but because that's the best they can do). LLMs only solve the first bit.
> mostly it weeds out the 90% of people whose email provider offers spam filters
Do you thing it weeds out 90% of politicians who talk total gibberish?
At end, it will enforce people who are in your social network. Some random celebrity will be ignored. Dude who lives 1 km from your location will be boosted.
Thank you! LLMs have raised the floor; not the ceiling. It's much easier to do stuff that was already kinda easy, but not easier to do things that were hard.
Maybe LLMs free up intellectual bandwidth for some, but what will we do with that increased productivity? Mostly scams for now, but I'm sure it'll be able to find many other net-negative or wealth-extraction applications.
What's a large language model doing when it's not being queried?
Am I correct that they only compute information when dealing with a prompt? If so, that seems like a fundamental flaw. An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output.
You think you can just hook up a firehose of visual or auditory data to ChatGPT and have that produce anything meaningful? That's not even the basic format that ChatGPT operates in.
Furthermore, isn't part of the point of the dataset an LLM is trained on that it has to be at least moderately structured and tagged, or you end up with garbage in the output?
Also, based on its continuous experience, it would be able to prompt you (send multiple messages in a row after not getting a response) or it should be able to wait for you to send multiple messages before responding.
>it would be able to prompt you (send multiple messages in a row after not getting a response)
This already happens. When you play around with local LLMs you'll run into the situation where the model answers your prompt. After that it will generate a new query as a response to it and then reply to that too.
No, unfortunately it doesn't learn online. It forgets everything after each interaction. They can collect the data and retrain later, but the hard part is doing all the fine-tuning steps all over again and ensuring the new model has no regressions. GPT3.5 and 4 are years out of date, an unfortunate situation when generating code or asking about recent events.
And now they removed the search plugin, probably they got sued for copyright leaks from the search engine results into the generated text. Using copyrighted data in the prompt is not necessarily legal. So we have to deal with out-of-date AI that updates once every couple of years.
The cost to compute these language models should eventually lower. Will OpenAI then release more frequently, or release larger LLMs? They'll probably to try achieve both goals in some proportion.
Training on web pages created after ChatGPT (and equivalently Stable Diffusion) was released, has a strong risk of trying to create the next AI (of whatever type) on the barely-even-proofread output of the current models.
A big issue with constant retraining is going to be the self referential consumption of it's own generated material as training material. There's already been the studies that these models quickly break down when they're fed their own generated data as training data. It won't immediately degrade them as it's a small percentage but there's already a lot of people using these models to generated spam junk out there and it'll only get worse with time.
> A big issue with constant retraining is going to be the self referential consumption of it's own generated material as training material
That's just a hypothetical scenario. Usually LLMs don't generate text used indiscriminately to train new models.
In creative tasks, the generated texts are the result of multiple explorations and corrections, filtered by the user, then posted online and commented on by other people. So there is plenty of correction and feedback mixed into the process. A child model could learn something from this feedback.
Then there is code and math, where it is possible to run tests or double check by a different method. These tasks could be iteratively improved by LLMs by incorporating the feedback.
Another field ripe with feedback signals is using LLMs in games and simulations, where you can specify a goal easily but finding a solution requires search and learning.
And a related application was RLHF, where a preference model was trained on human judgments and used to fine-tune a LLM. That preference model can be used to filter bad training data as well.
To generalize, LLMs alone can't do it, but LLMs with external systems can get feedback and improve. The garbage-in-garbage-out scenario can be avoided with a bit of external feedback.
>> There's already been the studies that these models quickly break down when they're fed their own generated data as training data
Not "these models" if you're referring to large language models pretrained and fine-tuned with Transformers. The only study I am aware of that explored the effect you discuss [1] had to make do with very different models as proxies: Variational Autoencoders, Mixture of Gaussians and a smaller LLM with 125 million parameters that hardly qualifies as one. It is not safe to extrapolate those results to really Large-LMs trained with Transformers, as it is not safe to extrapolate any results on one neural net architecture on another.
Note the Mixtures of Gaussians aren't even neural nets at all, but a completely different statistical model.
The question of what will happen once a few generations of LLMs have trained on their ancestors' outputs is an interesting one that has not yet been clearly answered. In the long run, future developments may even mean it's not really a valid question at all because such feedback retraining can't happen in practice- who knows?
Ranking algorithms should help with this, working on human feedback. Pages that spew badly generated text likely won't rank well on the top search engines. The big LLM names, such as OpenAI, would probably partner (e.g. with Microsoft), to tap into this if they needed to. They may have their own ranking algorithms anyway.
That's really just a temporary bandaid as the whole goal of these spam sites is to rank on search and the goal of LLMs is to appear to be human generated. You're stuck in an arms race against you're own improving performance if you're counting on filtering to keep your own output out of your inputs as an LLM company/trainer.
Also there's not a singular search rank for a site it's rank depends on the search query so I'm not even sure how you'd apply that metric.
This is one of the questions far too few people seem to be paying attention to.
"Thinking" in any way that we truly understand the term requires consciousness, and consciousness requires much more continuity than LLMs have. It would need continuity of input as well as continuity of learning in order to even be able to begin to approach something we might recognize as consciousness.
Well, "consciousness", at least as we recognize it, requires a mechanism by which the entity being measured can continuously form new "thoughts" and "memories" (which requires continuity of learning, and at the very least continuity of input being fed back from its own output), and some form of continuous external input of information about the world to at least be available, even if it is not always on.
A standard LLM is a static bundle of trained data that sits, inert, on a drive, with a process waiting for discrete input. When that input arrives, it does nothing to modify the trained data—the LLM's "memory"—it simply triggers a computational process that reads both the input and the trained data and produces an output based on them. This does not resemble in any way the structure of something that could be reasonably described as a conscious mind.
Continuous processing may be putting undue weight on accidental features of humans to be a necessary feature. Consider that a conscious mind can't represent the gaps in its processing, and so has an appearance of continuity. But this appearance probably doesn't map onto a continuous reality. An example is anesthesia patients that report a seemingly uninterrupted time from counting down pre-surgery to waking up. So interruptions, gaps, discontinuities, and so on don't necessarily eliminate the possibility for consciousness. It may be the case that LLMs are conscious when they are engaged in active inference.
While I generally favor a requirement for recurrent processing, I have low but non-zero credence for certain feedforward networks being conscious. The point of recurrence is to allow information about itself to influence its processing. But It seems plausible that feedforward constructs can represent meta-information in a way that is computed as part of constructing the output.
Imagine a hypothetical black box that could correctly answer any question you ask it and perform any task that you instruct it to perform. In terms of the impact such a thing would have on the world, would it matter if it were conscious? Would it even be desirable for it to be conscious? IMO, discussions of consciousness and self-awareness are a complete red herring when it comes to the topic of AGI.
1) They may feel like a red herring to you, but they are a big part of what a lot of people are talking about, and coming into a discussion that is clearly talking about whether an AI is conscious or how we could decide if it was and saying "that's all pointless, you shouldn't even be having this discussion" is kind of rude.
2) I am deeply skeptical that it is possible to have anything we could reasonably deem an "AGI" without it demonstrating at least what appears to be consciousness. Certainly, the "singularity" that many people talk about as being either an inevitability or an actual goal seems impossible without an AGI that can self-analyze and self-improve, which seem, at least to me, like they probably require consciousness, at least at the level being talked about.
> They may feel like a red herring to you, but they are a big part of what a lot of people are talking about, and coming into a discussion that is clearly talking about whether an AI is conscious or how we could decide if it was and saying "that's all pointless, you shouldn't even be having this discussion" is kind of rude.
Don't silence people, answer them. Say why it's actually important. Don't say that by saying "when you say it's not important, you're disrespecting the people who say it's important." That kind of thinking will take us back to the middle ages.
> I am deeply skeptical that it is possible to have anything we could reasonably deem an "AGI" without it demonstrating at least what appears to be consciousness.
The meaning of the word "AGI" is not an interesting thing to talk about. Call it what you want. If there's a conventional meaning to the word AGI that you're using, explain it and explain why an algorithm that isn't constantly running fails that. If you do this using the word "consciousness," you're passing the buck. You might as well be talking about souls for all the precision that the word "consciousness" has.
> seems impossible without an AGI that can self-analyze and self-improve
Thought experiment. What if you yourself aren't an AGI that can self-analyze and improve? What if you are two not-AGIs (by your definition):
1) a learner that improves the other based on new information, and
2) a controller that produces output,
And that they switch roles between 1 and 30 times per second, based on the intensity of the input. Would that mean you weren't conscious?
Well, imagine a model of the apparent motion of the planets where each planet moved on a perfectly circular orbit with any number of smaller circles on top of it, with the main circle of the orbit centered on a point between the Earth, or even the Sun, and another point a bit further from it. So, you know, an epicyclical model of the motion of the planets [1].
"In terms of the impact such a thing would have on the world", as you put it, would it matter if that model was completely wrong, despite its great predictive power?
Did we gain something when we figured out how the planets really move?
"Wrong" doesn't make much sense here. The more inductive bias we think is appropriate which we try to shove into models, the worse they perform. There's also an awful lot of fabrication the brain does with sense data, rationales etc. all of this is to say we have no clue what makes us tick. This means there's absolutely no guarantee we would recognize a replication of "human"
Something different is not necessarily "wrong". a plane's flight is no less "true" than a bird. It's not flying the "wrong" way.
Trying to elevate our very poor and wrong understanding of "human" to be the same as "right" or "true" is very silly. Even Biology with its set of constraints does not always solve the same problem the same way. Who are you to dub one way "right" ? Makes no sense.
Consider for a moment that the language generation neural network in your brain may not have continuity, and that it may exist elsewhere in your brain.
I don't find that particularly relevant to the questions at hand.
Even if it's true, then at best it means that the LLMs that we've created are only a small part of what's necessary for an AGI that functions as an "artificial brain": that not only are they not conscious, they are fundamentally incapable of it, and in order to create artificial consciousness, we have now gone as far along that particular path as we can (roughly speaking), and need to be looking into how to create the other parts of the "artificial brain", and how to hook them together.
Why does thinking require consciousness? This just seems like goalpost-moving. A few years ago, if someone gave you a transcript of a GPT-4 conversation, you'd definitely say it thinks, but now "it's not thinking without consciousness" and "it's not thinking unless it learns continuously".
> What's a large language model doing when it's not being queried?
Nothing.
> Am I correct that they only compute information when dealing with a prompt?
Yes.
> If so, that seems like a fundamental flaw.
Flawed in what way? It clearly doesn't need to be like us to be useful, because it's useful and definitely not like us.
> An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output.
This might be good, but it's not clear if, or to what extent, we really do that ourselves — the differences between working/short/long term memories, between episodic and skill, even linguistically between knowledge of phonemes, words, grammar, and the connection between those and the things they represent all being impaired independently of each other by localised brain damage[0].
Then there's how much this changes with some stages of sleep, and meditation to clear your mind.
Given the number of users (what is it, 100 million?), having it always on, continuously integrating, would still be inhuman even if the architecture was a perfect mirror of the human brain.
Also, if the AI is structured to be a "thinking machine", does that make it murder to switch it off?
[0] Cognitive Psychology for Dummies, currently listening to it as an audiobook.
> It clearly doesn't need to be like us to be useful, because it's useful and definitely not like us.
For many people, the question is not—and has never been—"is it useful?", but "is it conscious?", or "when will it bring the singularity?"
Yes, it is useful as it is. But it is not conscious, and if it is a step closer to the still-very-hypothetical singularity, then it is not a particularly large one, because of these very real fundamental limitations.
But engaging with people who are clearly talking about whether AI is conscious by responding as if they're talking about whether it is useful could very easily be seen as engaging in bad faith.
How can we tell? We don't have a single working definition of what "consciousness" is, rather we have at least half a dozen wildly different definitions, some of which aren't testable[0], some of which include VHS players[1], some of which exclude many humans most of the time[2].
If consciousness exists purely in working memory and not short/long term, then LLMs may have it within any given context window, even if they're effectively amnesiac between contexts.
Possibly. Like I said, we don't know what consciousness is.
> But engaging with people who are clearly talking about whether AI is conscious by responding as if they're talking about whether it is useful could very easily be seen as engaging in bad faith.
It may seem clear to you, but it didn't to me; furthermore, the singularity doesn't seem to me to require any of the many definitions of consciousness that I'm aware of, merely the capabilities for problem solving that LLMs increasingly demonstrate regardless of if they're conscious or not.
[0] what even is qualia
[1] can record sensory experiences in memory and change responses based on this
Consciousness seems like a red herring. OP can speak for themselves, but what I took is the feature preventing singularity is statelessness. Whether or not a thinking system experiences qualia, it seemingly needs some amount of persistence and continued ability to learn. As it stands, the training phase of a GPT might meet these requirements, but the inference phase is just a stored program. People talk of "ChatGPT" as if it's a thing, but it's actually billions, probably at this point quadrillions, of things. Each request is served by a new instance loaded into memory that lives for a few seconds, produces a response, then dies. Memory is simulated by storing some limited context window in a secondary system (Redis? Browser cache? I don't even know) to play back the conversation history as part of the prompt to the next GPT that spawns to service the next request.
To analogize to humans, imagine if every person who was ever born was born with all of the knowledge humanity has ever produced (though no understanding of hierarchy of evidence or any way to verify the knowledge is still correct) and the ability to produce language as an adult. They each lived long enough to produce one continuous thought, which they could write down, and then they die. New people who are born can read one such note each and add to it, up to a fixed limit of words. All people are born with the same knowledge base, the same personality, and that could never change until their God updates the DNA from which they're born out-of-band.
Could such a people be expected to produce "all technological advances that will ever happen but all at once?" Or has the definition of the singularity become significantly more modest? It's not clear to me any way in which such things could produce science and engineering on their own, as opposed to aiding in the production as tools used by higher-level beings that stay alive, form memories, and continue learning.
> Could such a people be expected to produce "all technological advances that will ever happen but all at once?" Or has the definition of the singularity become significantly more modest? It's not clear to me any way in which such things could produce science and engineering on their own, as opposed to aiding in the production as tools used by higher-level beings that stay alive, form memories, and continue learning.
Depends on the size of the context window.
Humans don't, and can't, learn everything from the ground up — we seem to have started with peer learning (it predates humans), which became culture; then we got (complex) language and could share more complex discoveries about the world; then writing and the dead could share their thoughts with the not-yet born centuries later…
…and, apparently, with machines. (Imagine explaining to Plato that we'd made metal from sand and used amber to make it discuss his philosophies).
If the context window is enough to create part of something new, then yes, I'd say an LLM can likely invent (or at least contribute to inventing) ${interesting novel tech}.
That said, I don't buy the "everything all at once" meme of the technological singularity, as I think any model with infinities in it is wrong — my preferred analogy is the technological event horizon, which is more like "you can't make good predictions past this point", akin to the event horizon around singularities in relativity.
Imagine you are just sitting there when you see a message pop up from ChatGPT 9.2 saying, “You know, I’ve been thinking, an easy way to quickly solve the climate crisis would be to get rid of all the humans. What do you think?” That would be just great.
It's not like it was explicitly programmed for any of the answers it currently produces, only to filter out dangerous and illegal stuff (and then by Bay Area standards, mostly).
Our brain is made of interconnected systems but somehow expect LLM architecture to encompass the whole spectrum.
Nothing stops you from running a loop that involves other systems such as long term memory (vector /dev storage), visual pre-processor (CNN), auto lora, and more.
That’s the fundamental flaw with most of the criticism - the tech is out only a few short months in the hands of everyone. The disruption will come from plugging it into feedback systems.
Side note: this publication, The New Atlantis, has been a very satisfying subscription. It's not too technical to casually read and consistently mixes together an interesting collection of essays, opinions, reviews, etc. I've passed every copy I receive to friends and then regret not having them around six months later when the ideas I read are still lodged in my brain.
In case you were wondering (as I was) about the qualifications of the author to write about AI, here is what I found on the "New Atlantis" website:
Ari Schulman is editor of The New Atlantis, as well as editor of TheNewAtlantis.com and of the New Atlantis Books series.
His writing has appeared in the Wall Street Journal, the New York Times, The Atlantic, The Hedgehog Review, Commentary, First Things, and Slate. He has previously been a research assistant in the Opinion department at the New York Times, an ontological engineer at Cycorp, and has degrees in computer science and English from the University of Texas at Austin.
I'm still not certain. I have a degree in computer science, but it had no AI focus, and even if it had, it was so long ago as to be mostly irrelevant to what is currently called AI if I had not been keeping up independently in the field.
... ChatGPT is a bigger, fancier mechanical turk. It's still aligned by the manual labour of underpaid people given no benefits for their work. They get exposed to some of the most horrible content on the Internet every day as part of their job. It's trained on data mined from other humans. It regurgitates that content and if it's too close to verbatim with the training data we have manual labour fixing that. There's an army of people that are trying to make it look like they're keeping SkyNET in a box... just don't look in the box.
See God in it if you want. Believe that it has feelings. Believe that it can think for itself and develop agency over it's future... one day. People believe all kinds of things about the natural universe that make no sense but it makes them feel something.
LLM's are what they are. No more, no less. Don't believe the hype, stick to the science.
Combine an LLM with long-term memory formation/recollection and a protracted (~18 year?) incubation period where it interacts with the outside world, and... the line between AI and consciousness would be pretty blurry at that point.
> the line between AI and consciousness would be pretty blurry at that point
This is like saying the difference between Cillian Murphy and Oppenheimer is pretty blurry because, look, we can watch Murphy do do all the same things Oppenheimer did!
For some reason with "AI" we get all mystical and act like imitation = the real thing. Just because I can convince you I'm a doctor doesn't make me one
> Just because I can convince you I'm a doctor doesn't make me one
Because our society has decided you need to become certified to be a doctor, and to do that you have to be a human being. There's no reason to think an AI couldn't have the same level of skill as a human doctor.
The film comparison doesn't work for me, Cillian and Robert are real people.
I think at the end of the day it's how you view the philosophy of the human soul. For me, it follows that if brains are matter and can produce the illusion of consciousness, so can electronics.
>> Just because I can convince you I'm a doctor doesn't make me one
> Because our society has decided you need to become certified to be a doctor, and to do that you have to be a human being.
For the record, I don’t think this addresses the point. The point is that if a person can convince another that they’re a doctor, they still might not be a doctor.
Your reply, that a computer can never convince someone that they’re a doctor because it’s a computer, doesn’t address the point. Presumably the point was made by a person, who would not be categorically excluded from being a doctor.
But it’s also not a great example because “doctor” in our society isn’t just a super smart person who knows about medicine. They also need to be licensed. There’s no amount of convincing a doctor can do to a patient that gives them a license.
So maybe a better formulation of the original point would be just because I can convince you I'm a computer programmer doesn't make me one. And maybe that’s true, I’m not sure.
No, the mysticism is from those who think we need something more than the correct sort of information dynamics to have consciousness/sentience/intelligence.
Right now we don’t have any system that has the correct sort of “information dynamics.”
There is no specification available for sentience/intelligence. So people who want to believe an LLM is sentient can jump to any conclusion they want and they wouldn’t necessarily be wrong. But they’re not right either.
I think it’s highly reductionist to compare an LLM to an intelligent, sentient being and claim it’s basically the same thing. We know how model training works, how transformers work, and we know what the expected results are. Not so for consciousness, sentience, or intelligence. Scientists have been making breakthroughs in our understanding of these phenomena but it’s still incomplete.
LLM’s are useful for one thing. They’re algorithms that run on a machine to perform work for us.
It's not combining an LLM with data/memory recollection and infinite agency that will lead to leaps in cognition. Complex systems still can do that and still not have self-awareness / cognition. We will need to take some more fundamental leaps to synthesize memory, agency and prediction to operate within a single system where self-awareness resides in the entity and not in the building blocks. LLM's represent a building block towards that cognitive unit, but the real deal is still some ways away
Although i fundamentally agree with you, i will say that it is impossible to know which "tricks" will lead to a leap in cognition, even looking evolutionarily, the difference in genome between a monkey and human is very small percentage wise, yet the effects of that small difference are stark.
I'm always on the position that at some point (i make no prediction of when) "AI" will still be labeled as not-intelligent, but to external observers it won't be possible to know.
Ie we will be able to describe how a function of `add(1,2)` is most clearly definitely not intelligent. We will understand the internal makeup of these neurons better, the whole system will be deterministic and understood - just a very large `add(1,2)` .. so it's definitely not intelligent, of course.
Yet it will still produce output that is fully and truly turing-proof. If you let it run, it retains, stores, "behaves", etc. It's clearly just `add(1,2)` in a loop after all.. we will understand it so well that it's obviously just a trick of our mind that it seems real.
The conclusion in that space will not be one of the machine .. but of us. If we can't tell it apart from us.. and if we know it is most definitely not alive or even mysterious.. what does that say about us? The existential crisis will be neat.
>If we can't tell it apart from us.. and if we know it is most definitely not alive or even mysterious.. what does that say about us?
Less than people think. We know that being a biological system differentiates us. If intelligent systems tend to converge where language is concerned, so be it. It doesn't mean the AI warrants any special consideration. It's still a tool.
While life is typically associated with biological systems, intelligence doesn't necessarily need proof of life. A hive, swarm or network can be intelligent as long as it shows self-awareness and has agency.
What is then, if anything? Definitely not "being a biological system" - that seems much more boring than intelligence, and the line between "biological" and "non-biological" is somewhat arbitrary and increasingly blurry anyway.
I'm increasingly thinking the game-theoretic emergent properties of social interaction as described in papers like https://desirism.com/morality-from-the-ground-up provides a good basis for thinking about this. The distinguishing factor then becomes something like, the agent is is participant in the moral-sphere by having "enough" properties to do so. It still leaves the details of that gradient a bit undefined, but at least specifies the basis upon which to make a judgment.
> If we can't tell it apart from us.. and if we know it is most definitely not alive or even mysterious.. what does that say about us?
That concepts like "consciousness," "free will," and "qualia" are nebulous precisely because they were invented by us to describe things we don't understand yet we feel obligated to insist are special and unique characteristics of our identity.
They aren't nebulous. "Consciousness" means "an existing stream of experiences". "Free will" means "a feeling that one could've made a different choice" (the metaphysical status of the claim still unknown), and "qualia" means "specific characteristics of experiences that one can attend to". One need not "insist" on anything to believe they are unique characteristics of our identity.
Yes they are - they are AS non-nebulous as you can possibly get - totally black and white obvious. "Is there a stream of experiences?" is very straightforward, black and white. It's what you have when you're awake, and what you lose when you're in dreamless sleep. Not nebulous in any way.
"Can you pay attention to this part of your experience?" is a very black-and-white, non-nebulous question with an easy yes or no answer. Not sure what you're reading into this.
No, it is unclear because this definition insufficiently distinguishes a cluster of related, and ill-defined concepts.
Consider a video camera. It seems like it is a consuming a "stream of experiences". But most people wouldn't consider it it conscious. Why not under your definition? It works when on, and is stops when turned off. Black and white. But clearly we are missing some nuance in the details. Maybe we also require some interpretation of the experiences? I can spin a convolutional neural net that identify objects within a video: interpretation. Does that make it conscious? No? Then we need something else to define it. Maybe response to stimuli? We can program machines for that, but then also things like fire respond to environment interactions too. And so on. It really isn't black and white after all.
Dualism, or at any rate the version of it that can be described as operating on the (possibly unconscious) assumption that a world "outside" of this world exists, is the secret ingredient at work here, smuggled in along with the kinds of notions expressed by the person you're replying to. We just can't seem to fully disentangle very old deep-seated animist instincts, even sophisticated formulations of them, like the soul or consciousness, from our reasoning and descriptions of dynamic systems. I'm not sure this is a bad thing even though I recognize that it is a crutch. completely materialist interpretation of phenomena would probably lead us to the realization that we are not alive at all, and that what I call "my life" is just a kind of curling strand of causality frozen in the Parmenides block universe.
I've come to suspect that a habit of using the passive voice when discussing these phenomna contributes immensely to their ongoing currency. Steering away from this by asking questions like, "_who_ or _what_ is attending to/experiencing the stream?" may be a useful way to ground these statements in material terms.
I see dualism vs. consciousness as separate. Dualism is often used as an explanation for how (real, not compatibilist) free will can exist in an otherwise materialist world, but it'd still require the "outside" to be "magic" in the sense that it'd require a class of logic that is neither deterministic, nor random, nor just any form of combination of the two (e.g. not stochastic).
Consciousness, on the other hand, boils down to what gives rise to the sensation of self-awareness, and what that sensation actually is, and it's not a given that requires any form of dualism (or that any form of dualism would solve the question at all) or that a strict materialist world can't involve self awareness.
For what we know, self-awareness could be a property of matter itself, and the only thing different between "conscious" and "non-conscious" is getting to a level of sufficient reasoning-ability to actually be able to express it.
You keep saying stream of experience, but you don't actually have a stream in the sense of a continuous uninterrupted experience. Your mind let's you believe that though.
That's a good point. We appear to have no way (and probably can't have) to discern passage of time from a static experience of a single infinitesimally short moment, as our only experience of time is through memory that we can't know whether is a static snapshot or dynamic.
Since we don't understand its nature, we actually have no idea whatsoever whether it's a continuous stream or not. I mean obviously we sleep, but I take it you didn't mean that kind of interruption, which is beside the point. But either way, it doesn't matter, because "stream" can simply mean "the quick succession of experiences that seem to be continuous to the experiencer". And none of this really matters at all for the discussion at hand, the answer is the same either way.
Consciousness is not a continuous stream at all, the fact you fall asleep or can be given medication to turn it off is an obvious example.
Nova/PBS recently did a number of episodes on the number of tricks your brain is playing on your at anytime. Once you start seeing the shortcuts the brain takes in action the entire process is far less magical.
And which measurable, enumerable qualities of those things set them apart from not having experiences or not paying attention so that I can determine whether or not an AI, or a camera, or an animal, or another human, is "having experiences" or "paying attention"?
That wasn't the question - we don't know how to determine whether an AI or camera has it. But it's not nebulous - it's what you're experiencing right now. That's simply what we mean by the word - that's it - no nebulosity to it. Figuring out whether a computer has it is a separate question, and difficult.
Which is impossible to define in objective terms; it is the essence of subjectivity on which all other subjectivity rests. It’s not merely “we don’t know how to determine whether an AI or camera has it”, it is categorically impossible to determine if something has it. The popular narrative about which things in the natural world have it has evolved radically over time, but in no case has there been, or could there be, any empirical basis for this; its all just vibes.
It became the question when you tried to define the original terms that way, because a definition that can not be used to discriminate what it defines from what ot does not define is inherently nebulous.
And your definition is now circular and can not be used that way.
Hence the claim that the original definitions were nebulous.
> "Consciousness" means "an existing stream of experiences".
Those seem equally nebulous.
> "Free will" means "a feeling that one could've made a different choice"
That doesn't seem like the ordinary definition of the term, either in philosophy or in everyday usage. I would think most people would define "having free will" as not just feeling that one could have made a different choice, but rather it actually being physically possible for a different choice to have been made.
> "qualia" means "specific characteristics of experiences that one can attend to"
I think those are also equally nebulous.
> One need not "insist" on anything to believe they are unique characteristics of our identity.
The insistence is necessary, and invariably comes up. As soon as I say to someone "given your definitions of these concepts, I don't think they exist" they will say "well, I've started with the foregone conclusion that I cannot deny the existence of these things, so either you're plainly wrong, or we need to tweak our definitions and understanding of these things."
They aren't. It's literally happening right now as you read this - a stream of experiences is present. If that is the case, YES it is consciousness, if nothing is happening, NO it is not consciousness. Black and white - literally as non-nebulous as you can get.
Eh, that's not super concerning to me. It falls into the same pitfall as "chemical reactions aren't conscious, our brains are just lots of chemical reactions". Namely, it's the mereological fallacy: a denial of emergent properties. A single water molecule is not a liquid, a gas, or a solid. The property "phase of matter" simply does not exist until you have at least a handful of molecules.
Oh, i agree. If anything i'm apathetic to us being unique at all. However i'm speaking in the general cultural perception sense. I think a lot of humanity depends.. or at least has built a self-image around uniqueness as a foundation to our believed superiority.
We often discuss things that would challenge or overtake us. Hypothetical advanced aliens, artificial AI, etc. However i don't often see pop culture discussing the opposite - us, or even life, being challenged on it's uniqueness. Plus then that leads to all sorts of self-autonomy discussions (which is discussed in pop culture imo).
I dunno, i suspect "people" won't handle it well. Hell, there's entire religion(s) built on the foundation that we're uniquely gifted as a product of being crafted in a gods image. To question us could question god. I find these questions neat, interesting, etc - but socially i suspect it would be troubling, contentious, frightening.
We've dealt with this problem before: Historically all humans being "people" is a relatively new concept in of itself. I'm sure there's going to be growing pains there, but I think we'd figure out how to cope. Hopefully relatively quickly compared to that example.
Why not use less emotionally charged words? Goal to action mapping for instance. How far away are we from a general goal to action mapper that can outcompete a human being on a given goal.
I am an absolute layman when it comes to AI topics, but I have been wondering about this lately.
A lot of the hopes in regard to AGI in LLMs seem to focus on just making the models bigger, or feeding them even more information. On the other hand, some researchers in the AGI space seem to move in a completely different direction, trying to use neural networks to simulate different parts of a brain, moving away from language altogether.
But what I'm wondering is this: might it be a more productive avenue to consider an LLM not as an entire mind, but as a single part of an hypothetical artifical psyche, which excels at some of the overarching tasks of consciousness (such as acting within a cultural context and epistemological framework), but can delegate other aspects to more fitting subsystems? Could it be combined with other networks optimized for different tasks? Potentially, this could just be done by having inputs from other parts of this artificial psyche fed in as prompts, translated by an encoder/decoder of the likes of CLIP, i.e. a decoder that can translate ideas between different liminal spaces.
The work that I keep remembering in this context is Jaynes' Origin of Consciousness in the Breakdown of the Bicameral Mind [1], which I stumbled upon here on HN [2] [3]. Jaynes believes that consciousness is not an innate ability, but a cultural achievement - in order to become conscious, one first has to learn (through others) to conceptualize consciousness, and to think of oneself as a thinking being. He insists that consciousness is not a necessity for human agency, and that humanity has been building civilizations and using technologies long before humans discovered (or achieved) consciousness. Jaynes puts forward the idea of an ancient, non-conscious human mind based on, upon other things, "verbal hallucinations" - thoughts and memories that involuntarily enter as associations, but whose origins are interpreted by the mind to not be within itself, but somewhere outside of it, such as in a divine monologue; thought, to the pre-conscious mind, is the incessant blabbering of the Gods.
It seems that LLMs would excel in the role of this "camera" of the mind - as the source of divine monologue and divine instructions, that can then be compared with memories, interpreted and executed by more specialized subsystems. Might it be feasible to give up the plan to make AGI self-aware altogether and instead build an AGI powered by a GPT-4 ascended to divinity, just as ignorant of its own existence as ever?
According to several sources, GPT-4 is already doing this (It's called "Mixture of experts"). I agree though, it's still useful on an application level. You can build out even higher on that and have the LLM decide which is the most appropriate "hats" to wear for a given task.
In a similar position. With human brains, we know some parts are generative and some are restrictive/filtering. The filters seem to be how we align thoughts with reality. Lots of people have taken those filters out and (unfortunately for them) seem to be unable to function or (unfortunately to us) become cult leaders.
The fundamental issue is that all the talk of "self-awareness", "cognition", "souls", whatever you want to call it, is all beating around the bush to describe what we presumably each subjectively experience in our own minds but which cannot be empirically tested for in others. We generally assume that other people experience the same, but cannot empirically prove it and some people speculate that others might lack it (P-zombies, "NPCs", etc). And then when it comes to animals, a huge portion of the population refuse to believe animals can possess it. I think the majority belief is that some animals definitely have it (dolphins) and some animals probably do not (jellyfish? corals?), but there's no empirical test for it. On the fringes, some people believe only humans (or even only themselves personally) possess it, while other people believe that even inanimate rocks could possess it.
So basically we're never going to achieve some consensus about whether an LLM/AI could have it, because we can't even agree on whether dogs and cats have it. We can't even all agree on whether all humans have it.
All this assumes that "it" is a dichotomy: either something has "it" or they do not. But it seems pretty clear to me that "it" must be a continuum even in humans because it is very hard to draw a sharp line between a fertilized egg and an adult where a human comes to have "it". (This is the reason that some people conclude that fertilization must be that line, because there are very few other viable candidates.)
I agree that the line is fuzzy, not sharp. But either way, how do you precisely determine the parameters of the line or gradient experimentally? It's my personal belief that a salmon is conscious, but less so than a dog, and that a preying mantis is conscious, but less so than a salmon. But these aren't beliefs handed to me by science, they're just my guesses informed by some casual knowledge of how complex the brains of these animals are. I assume that anything with a brain can have the experience of consciousness and that this has some relationship to the complexity of the brain, but I can't confidently say much more than that. How does an octopus rank against a dog? I have no clue, their brains are structured very different so it's hard to compare the two. Octopus seem to be very smart but their subjective experience could be very different. That's not something I have insight into. Are orcas more conscious than humans? Their brains are bigger and wrinklier, so I think the answer is definitely maybe! There's a distinct possibility that in the mind of an orca the orca is more aware of itself than humans could possibly ever be, but how can we really know that for sure? I can't experience what an orca experiences, so I can't directly compare the orca mental experience to my own. Studying the behavior of orcas can reveal interesting facts, such as their use of language and complex social relationships, but as evidence of subjective consciousness these are indirect at best.
Maybe some day we'll have scientific instruments that will allow one being to 'psychically link' with others and experience the other's subjective sense of being. But until we've got something like that, I think the matter is effectively closed to empirical inquiry.
> how do you precisely determine the parameters of the line or gradient experimentally?
Well, one way is to observe similarities in the I/O behavior of these various systems, and in particular if they exhibit any evidence that they have goals and desires, and then beyond that if those goals and desires extend beyond merely that necessary to survive and reproduce. So for example, the fact that some humans seem to want to do things like read and write and make art is IMHO is evidence that they are more conscious than salmon.
It's indirect evidence at best. Some people value artistic endeavors more than others but I don't think most people would agree that artists are more human than the rest of us. These differences in behavior might evidence differences in values or opportunity.
For instance, with thumbs and a terrestrial lifestyle, humans are more capable of producing art and technology than just about any other animal. But I don't find this to be a persuasive argument for human minds being more sophisticated than orcas. Even if art is the answer, our lack of insight into orca language makes it difficult if not impossible to fairly compare them to us. Maybe Orcas have poetry so sophisticated and beautiful that it puts all human art to shame, but it's completely inaccessible to us because we don't understand their language. Or maybe they're far smarter and more aware than us and could compose such poetry, but choose not to simply because they don't value it.
I didn't mean for math and art to be an exhaustive list. Remember, my actual criterion was:
> if they exhibit ... evidence that they have goals and desires ... beyond merely that necessary to survive and reproduce
Orcas qualify because they play with their food. It's quite gruesome to watch. (And, BTW, this particular behavior is evidence that either orcas do not have a theory of mind or they lack empathy for animals that they eat. The former would be the charitable interpretation.)
> We can't even all agree on whether all humans have it.
What would it mean for humans not to have it? The debate is over the nature of consciousness, not that humans (and many other organisms) lack any sort of consciousness. So when we see color or feel pain, what is that? We are conscious of colored things and painful sensations. There's no real doubt about that, unless you're solipsist about other minds. Which good luck if you are, I guess.
I don't believe human P-zombies exist, but how do you empirically prove they don't? Nobody has been able to devise an experimental test to determine whether a person, animal or AI has a conscious experience or whether they mimic all the outward signs.
> There's no real doubt about that, unless you're solipsist about other minds. Which good luck if you are, I guess.
Exactly. It's fair to write off people who get solipsistic about other human beings because those people are fairly rare and are beyond helping anyway. But if we're talking about animals or AIs, the 'solipsist' position becomes more common and there's no real good way of debunking it. Consider the problem facing animal rights activists who wish to persuade the public that fish have feelings. What scientific test for fish having feelings can you use? Put a fish in an MRI and you can see the fish brain doing stuff, but the "fish don't have feelings" camp doesn't find that persuasive. You could try performing the mirror test on fish. Most fish would probably fail it, but what would it really mean if they didn't? The feelingless fish camp would just say the fish behave as though they have a sense of self but actually don't. How do you refute that scientifically? It seems the most effective way to persuade people that fish have feelings is to resort to sophistic tactics; appeals to emotion, bullying and browbeating, or simply just laughing at people when they say fish don't have feelings. Make them ashamed to spread their belief to others. These kind of tactics are distasteful but generally effective.
With AIs or LLMs, the problem is even worse than it is with fish. Fish at least have biological brains, which suggests they have some commonality with us. Fish have faces, so some part of the human brain is wired to empathize with them. We have a fossil record that shows we came from fish, and this relationship perhaps makes it easier to believe that fish have some sort of subjective consciousness experience that is similar to ours. In these respects, AIs are further removed from us than fish are, so it will be even harder to persuade people that AIs could have the same sort of conscious experience as we do. Even after having deep philosophical conversations with an AI in which the AI proclaims to have a conscious experience, a lot of people will say the AI doesn't actually have that and was merely trained to speak as though it does. I don't see any clear cut resolution to this sort of doubt. There's no empirical test that you can perform that gives an indisputable answer one way or the other.
The answer to what the ‘real deal’ is obvious, but fairly disturbing.
Games.
From the first clever bloke who came up with an algorithm to play a card game against himself (ie, Solitaire), to the Mechanical Turk, to Big Blue, the the ghosts in PacMan, the NPC’s in Counterstrike, etc, etc., there is a fascination with making something that can ‘beat’ us.
Humans have a need to compete.
We’re drawn to the idea of a machine that is so smart and clever that it poses a genuine challenge to our wits.
The ‘real deal’ that most have in mind isn’t some virtual assistant that can help you calendar things or write letters. No, the ‘real deal’ is always described in terms of SkyNet or sinister AI overlords, or malicious intelligent robots.
I don’t think that stuff is hyperbole.
Rather, I think that is precisely what we want… we want something that can destroy us… something, against whom, we can compete with our survival at stake.
In short, the ‘real deal’ has been staring us in the face for centuries; we make movies and write books about it… particularly in Sci-Fi. The goal IS SkyNet. The goal IS HAL. Let’s be honest… that’s what we are shooting for — the ultimate competitor that, unlike anything else on this boring planet, has the wits to take us out.
Whether it’s bored trailer-trash playing Russian Roullete, or bored Billionaires descending to the Titanic in a jerry-rigged sub, we have a very strong need to make our lives interesting by betting our lives themselves.
The purpose of AI is to make a challenger who has the ability to wipe us out, with intent.
I think we'll need more than that too, but in humans and other animals it isn't the building blocks that are self-aware -- neurons aren't any more aware than any other cell -- self-awareness is a property of a set of connected neurons, not in the neurons themselves.
I infer that "JFK" is being referred to as "the president", regardless of what stage of life he's at. Therefore the correct answer would be the length of his body as a newborn, with nothing to do with who was the president at the moment of his birth.
For example, given the headline: "Was the president strong at writing essays when JFK was a high school freshman? Read this essay to find out!", would you expect an essay from Herbert Hoover while he was in office, or an essay from JFK when he was in grade 9?
A lot of exposés of LLM weaknesses involve abusing the ambiguities of English with questions that are impossible for humans to give a satisfactory answer to, because they can be read in multiple ways.
Part of my bullishness on LLMs is that a lot of the criticism is so bad, and if they weren't the real deal there would be more low hanging fruit to attack them on. Cryptocurrency was having real problems from the first moment people realized how much data they would have to download to initialize a node.
> A lot of exposés of LLM weaknesses involve abusing the ambiguities of English with questions that are impossible for humans to give a satisfactory answer to
There's a general unstated assumption that a truly intelligent entity wouldn't make mistakes, which is sort of contradicted by, you know, the existence of humanity. I think making (certain kinds of) mistakes is a _sign_ of intelligence, rather than a sign that it's not.
Yes, you're misreading it. "the president" is not a label that applies solely to JFK, or one that applies at all to JFK in the context "when JFK was born".
If someone gave me the question "who was the president when JFK was born?", the answer is clearly not JFK, so it isn't correct to infer that JFK is the president whose height, party affiliation or essay writing skill is being asked about "when JFK was born".
If someone gave me the headline "was the president strong at writing essays when JFK was a high school freshman?" I would assume that either they were trying to trick me or that they were a non-native English speaker or computer program that didn't understand how English syntax or the concept of presidency worked. If you flip it to "was JFK strong at writing essays when the president was a high school freshman" then yes, I'd consider both to refer to JFK because "the president"can't be identified as anyone else, but I'd also consider it to be bad writing by someone cargo-culting the idea of elegant variation...
> If someone gave me the headline "was the president strong at writing essays when JFK was a high school freshman?" I would assume that either they were trying to trick me or that they were a non-native English speaker or computer program that didn't understand how English syntax or the concept of presidency worked.
I don't think it's as complicated as all that. Most English speakers wouldn't refer to a single person by two distinct proper nouns in a sentence like this, so I'd assume they were referring to two separate people. If they had meant to refer to the same person, they would have used a pronoun. I'd probably guess they were either talking about the President at the time at which JFK was a freshman (Hoover, I think) or the current President at the time the question was being asked.
On the other hand, if the question was "Was President Kennedy strong at writing essays when JFK was a high school freshman?" then yeah, I'd agree with your take. In this case, it's clear that the writer is intentionally referring to the same person but using two different proper nouns, which is a very odd phrasing.
> Most English speakers wouldn't refer to a single person by two distinct proper nouns in a sentence like this, so I'd assume they were referring to two separate people.
I agree with this (though elegant and inelegant variation is a thing). But nor would any English speaker be likely to ask about the skill of an anonymous holder of a position with reference to an ambiguous date range associated with a time in which a later holder of that position would be extensively practicing that skill, which is why I classed it the sort of sentence with no natural meaning you'd only conceive in order to be deliberately ambiguous (or because you don't understand how to write English properly).
The original sentence isn't ideally constructed, but at least the president when JFK was born is a natural and easily identified sentence object, and it would be odd to ignore that in favour of an alternative interpretation that also involved interpreting "how tall" as being more likely to refer to the [probably-not-recorded-for-posterity] length of a newborn infant (who wouldn't be idiomatically referred to as "tall" or the president in the context of his birth) than the height of a person who was president in the relevant time period
> "who was the president when JFK was born?", the answer is clearly not JFK
Yes, you can deduce that, since the question is nonsensical (trivial) if "the president" refers to JFK in that context.
> If you flip it to "was JFK strong at writing essays when the president was a high school freshman"
This is a perfect illustration of why the original sentence can be read as referring to JFK, since in speech it is common to have ill-defined references that are later clarified, and there's no formal rule that enforces that all terms must be defined before use.
> This is a perfect illustration of why the original sentence can be read as referring to JFK, since in speech it is common to have ill-defined references that are later clarified, and there's no formal rule that enforces that all terms must be defined before use.
References that are later clarified is exactly how we arrive at "the president when JFK was born" as defining Woodrow Wilson as the object of the sentence (and it makes much more sense to ask about how tall President Wilson was in the context of 1917 than to wonder how "tall" baby "president" JFK was in 1917).
Whereas "when the president was a high school freshman" can't be used to identify anybody else [unless there is a time period in which a high school freshman was the president!], even if the sentence hadn't already made it clear that JFK was the object of the sentence. But it's still an example of bad writing.
Even when there's no ambiguity, we don't do sentences like "The SpaceX founder was about 18 inches tall when Elon Musk was born" which is basically how you're implying the president question - which has a much more reasonable alternative interpretation - could be answered...
It is possible to write a truly ambiguous sentence in English; nobody is arguing otherwise. But whilst the original sentence might make you think a bit, it's not naturally interpreted as being equally/more likely to be a question about the not-tall not-president in the context of the time period instead of the person with height and presidency in the context of the time period, unless you're being obtuse or lack fluency in the language.
> "Was the president strong at writing essays when JFK was a high school freshman? Read this essay to find out!"
You write:
> "when the president was a high school freshman"
These are very different circumstances, of course.
If the original sentence had "reigning" before the word "president", then I would absolutely agree with the given interpretation.
I've noticed that Americans call ex-presidents "president", and ex-governors "governor", so that merely adds to the confusion. English is not my native language, and I'm not American, so perhaps I fall into the "lack fluency" category. Conferring with other native speakers nearby, every American immediately reads it as intended, whereas Brits and Canadians do not.
I conclude that the original sentence is poorly written, and that I misread it.
The fact JFK is being referred to as JFK in the same sentence as 'the president' is being referred to - rather than 'how tall was the president when he was born' or 'how tall was JFK when he was born' indicates they refer to different people. Therefore, 'the president' would most likely refer to whomever was the US president when JFK was born, though if the article referred to the president of some other organization before referring to JFK itt might not be the case.
I wish people would stop with the bizarre AI-latry that has gripped them en masse and realize that if its a "moment" at all, its a human moment.
The impressive feat is the ability of researchers to manipulate a generic statistical model paradigm to fit large amounts data and be reused for something apparently useful.
Whether that path has a future depends not on AI bootstraping itself but on said researchers to keep building on this paradigm.
The black box and inscrutable nature of these algorithms works against them. I.e. reaching the next level in some purposeful manner requires more understanding. Human understanding.
Something I’ve realized in the midst of all this hype is that many, many people seem to actually want to build AGI, for its own sake, and many take for granted that this is a goal. I don’t understand why. It seems like we’re racing right past using the tools we already have to solve humanity’s many very real and solvable problems, in favor of trying to invent something just because it sounds neat.
AGI sounds neat because the hype says that it will solve all human problems at once. That's obviously fake and wrong, but so is the idea of solving current human problems with current tools.
> That's obviously fake and wrong, but so is the idea of solving current human problems with current tools.
There are easily stated problems like "Reduce US car accidents by 50%" that should be solvable with AI as it stands today. This seems like a useful and achievable goal that would save money and lives. I've never understood why full self-driving was the problem most people chose to solve instead.
this is how you know the AI moment is not real. its all marketing for the likes of capitalistic shit like Tesla and self-driving as its some kind of savior for humanity instead of some marketing gimmick for some car dealer. like bro trains exists.
Solving some percentage car accidents with AI requires a law or a common agreement by car companies to put AI in cars. That's an investment of time and money that isn't going to solving some other problem. Everything is a trade off, and some problems simply come with the territory of being human.
The appeal of self-driving is usually just financial.
> Solving some percentage car accidents with AI requires a law or a common agreement by car companies to put AI in cars.
No, it doesn’t require a law or a common agreement, it just requires the absence of a law preventing it and salable benefit in a competitive market. Cf., computer vision and other AI components for lane keeping, throttle control, and automatic braking, which is “solving some percentage of car accidents with AI”.
> There are easily stated problems like “Reduce US car accidents by 50%” that should be solvable with AI as it stands today.
IF you really want to solve that, that’s an urban design problem that doesn’t take AI at all; it isn’t solved because there is insufficient desire to solve it, not because of technology. But as far as AI-use-in-cars goes…
> This seems like a useful and achievable goal that would save money and lives. I’ve never understood why full self-driving was the problem most people chose to solve instead.
In the space of auto safety technology it’s not. Driver assistance short of full self-driving is much more widely deployed, and continuously improving, by far more automakers than FSD. Full self-driving gets the media attention because its dramatic and showy.
True, I should properly say that the publicity and a lot of SV focus went to full self-driving. Thank goodness people are actually pushing the less glamorous topics.
That said, it would be great to see more public focus on solving problems like keeping bikes and pedestrians from being run over. In my town there's a lot of talk about the issue but it's not been backed up by initiatives that actually change the numbers. (Which are bad.)
FSD frees up a massive chunk of attention-hours per human in the US, attention-hours that you would presumably spend hooked into ads on various entertainment platforms that ultimately drip into Big Tech's money funnel. Incentives explain the goal.
"When you see something that is technically sweet, you go ahead and do it." Hinton referenced Oppenheimer some time before he changed direction on this and resigned from Google. It sounds like what made the difference to him was a flip from "far mode" thinking where something like human level seemed at least decades off, and thus felt like a cloudy abstraction, to near mode.
Like Feynman said, you can only really understand something if you can make it yourself. AGI may have practical uses, but it is worth creating for its own sake to get a better understanding of how our own intelligence works.
"It is too early to say that the new AI class is an inherently antihuman technological paradigm, as social media has proven itself to be.
But it is not too early to suspect that AIs will dwarf social media in their power to disrupt modern life.
If that is so, we had better learn some new and unfamiliar ways of interrogating this technology, and fast. Whatever these entities are — they’re here." -Ari Schulman
People that mistake an large language model (LLM) for anything other than a LLM make some fundamentally broken assumptions.
Are ChatGPT, Midjourney, etc. a fundamental leap in the state of the art when it comes to allowing computer systems to understand what people mean and return something useful based on it? 100%
Is ChatGPT or the like going to become self-aware, compromise other computer systems, etc? No more than your shoe is going to take over your foot.
There's far too many people worried about "AI" that don't have enough context to realize they're fearing a non-sentient tool that has zero agency, and will not for the foreseeable future.
> Is ChatGPT or the like going to become self-aware, compromise other computer systems, etc? No more than your shoe is going to take over your foot.
Agreed. It's strange though, to be honest I don't see much if any worry about self awareness, i think anyone who knows anything understands that's not the issue. The "issue" if you can call it that is how it will impact society from a labor and content perspective. How much synthetic content will be perceived as true and assumed to be correct simply because we haven't had time to adapt to the fact that the rate of synthetic content is exponential now.
> How much synthetic content will be perceived as true and assumed to be correct simply because we haven't had time to adapt to the fact that the rate of synthetic content is exponential now.
That will be a problem for sure. But people believe all sorts of easy to debunk things already and that list is ever-expanding, so I don't know if there's any cure for it.
As far as content goes, I think human beings writing out boilerplate anything is something that's not long for this world.
Labor is a trickier one to judge, but I'm not too worried about a large negative impact in the immediate future.
People thought that modern appliances would usher in a new age of leisure, and the truth was that while they removed a lot of drudgery, work expanded to fill the vacuum and the percentage of the week spent laboring didn't significantly decrease.
On a positive note, I think the kind of work people are subject to now is better because of modern appliances, so I wouldn't be surprised if the same holds true for all the labor problems that can be handled by neural net / LLMs.
What does it mean to understand language, and can you test your definition?
I say this because statements like yours are a bit like saying calculators don’t understand arithmetic. In some sense maybe they don’t. But in another sense, they actually understand arithmetic better than anyone on the planet.
So maybe ChatGPT does understand language. Maybe it understands language better than anyone. What it doesn’t have are real-world experiences.
Performance is not understanding. The difference is objectivity: a calculator does not think about arithmetic, it performs it.
A large language model is a collection of relationships between "tokens" (arbitrary groups of characters, similar to words). This is intended to exist in the same conceptual place that language does; language being defined as a collection of relationships between words and punctuation.
Personally I don't think we will succeed building a true AGI based on the current computer architecture. The computing units of the only intelligent beings we know of so far - us and maybe some animals to an extent - work very different from how a CPU works. I suspect we will need to build a computer that more closely resembles a neuron and the way it performs calculations...
That entire section is one big confused contradiction. The author argues that CYC (Doug Lenat's project to hand-code a gigantic database of logic rules encoding common sense knowledge) "can do deep reasoning" because it can answer this question correctly:
How tall was the president when JFK was born?
Unlike Google and WolframAlpha who can only find the closest association (the elevation of the town were JFK was born). Still, the author says, CYC would never "challenge humanity’s unique rational status, any more than computers that could solve equations did" because it doesn't understand natural language, only logic formulae. And yet, the author seems to be arguing that ChatGPT does understand natural language because... it can answer the same question as CYC; albeit in natural language, unlike CYC.
I think the author has confused the ability to return results in natural language with the ability to understand natural language. By that token, ELIZA (mentioned at the start of the article) must have also been able to understand natural language, even if it didn't have CYC's ability for "deep reasoning", just because it could parrot its user's input in natural language.
That doesn't make sense. Understanding is clearly something that needs to happen before utterances are formed. Just looking at the output of a system doesn't tell you anything about the internal workings of the system, that's an error that many people keep making in this entire discussion about "AI"s. Look at the sky: the sun looks like it's turning around the Earth. Well, we know it isn't. What we see on the surface is rarely enough to explain what is going on "inside" (or outside, as the case may be, for the sun).
I find the emphasis on the combination of big data, powerful computing, and advances in machine learning to be a compelling argument for the current AI momentum. The idea that these factors are creating a synergy that propels AI development forward and contributes to tangible progress in various fields is really encouraging.
230 comments
[ 4.3 ms ] story [ 263 ms ] threadThat was ~26 yeas ago. In the intervening 26 years humans have been curb-stomped so hard at chess that certain microwaves might be competitive against human grandmasters. Anyone who doesn't think that was an exciting moment for AI research has impossible standards, we're living in a tiny window of existence where humans were at the top of the intellectual food chain and are now dropping to also-ran status along with the rest of the meat-based animal kingdom.
All these moments are the real deal. The idea that somehow this particular step change is even unexpected is missing the forest for the trees. The trends that have been observed since the beginning of computers are chugging away so quickly the whole process may well happen in a single generation.
Robotics is behind in making a 'human'.
Even AlphaGo had a human moving the pieces, and nobody would say that was evidence of any 'lack' in the AI.
It's just an interfacing issue, engineering will eventually solve.
Edit> What I was getting at. Take Robots as the physical interface layer for the AI. AI advancements are currently outpacing advancements in Robotics. So potentially AI could use real humans as the 'hands and feet' to do things in the physical world, by giving directions. Something will be needed to bridge the gap until Industry can be done completely by robots. Until then, AI wont be able to manufacture itself.
https://www.nbcnews.com/id/wbna9816703
If you have ever worked in a modern warehouse, it is very evident, the humans are just following instructions from the system. Now scale that up to a manufacturing facility, it is already happening.
Everybody with an ear-piece or 'phone' type device, following instructions.
> Workplace democracy, AKA unionization, AKA the thing Amazon has pulled out every dirty trick to prevent, is the difference between centaur utopia and reverse-centaur dystopia.
-- https://pluralistic.net/2021/02/17/reverse-centaur/#reverse-...
Also, we were talking about deep blue. Perhaps trying to not go OT would be beneficial to a productive discussion.
We're building tools to help us augment our intelligence, I don't see anyone getting "curb stomped"?
If intelligence leads to more violence, then I don't think it will be as useful as we'd have hoped.
> [Hobbes] even, through sheer force of imagination, was able to outline the main psychological traits of the new type of man who would fit into such a society and its tyrannical body politic. He foresaw the necessary idolatry of power itself by this new human type, that he would be flattered at being called a power-thirsty animal, although actually society would force him to surrender all his natural forces, his virtues and his vices, and would make him the poor meek little fellow who has not even the right to rise against tyranny, and who, far from striving for power, submits to any existing government and does not stir even when his best friend falls an innocent victim to an incomprehensible raison d'etat.
-- Hannah Arendt
And that only applies to criticizing phrases like "killing it", without even demanding it to not be used -- but not to bringing out Nineteen-Eightyfour? And doublespeak? No, doublethink means holding two contradictory things to be true at the same time, doublespeak is the verbalization of that. You're thinking of wrongthink, and either way you are doing what you criticize. Instead of saying "this is horrible" you say "this is dystopian", it changes nothing.
More importantly, your reply contains nothing specifically applying to my comment. Maybe you missed the point, which is that people delude themselves to compensate their lack of agency, being restricted to partake in a rat race that will leave them empty-handed in the end -- all of which Hannah Arendt explains in the context, but I didn't want to post a wall of text. That they don't actually mean what they say, because what they mean (and do) would be to pitiful to say without such flourish, and that they aren't actually violent and powerful, but "poor meek little fellows", is precisely the point.
Over time the original meaning may fade or be forgotten, and the residual symbol in the language becomes the thing itself. Etymology is full of examples.
But that much history has not yet passed since the introduction of the term "curbstomping" and so using it is still a choice that conveys some meta-meaning about content and style.
As mentioned further up in the thread it is unnecessarily violent for conveying the meaning in the comment.
No reason to make it an issue, people just like using those metaphors.
> beat Kasparov
So much violence everywhere
I have trouble articulating it, but I can see that LLMs and chess engines don't do the slightest hint of it.
I'll be concerned when machines show the slightest hint of it, but we're playing a ball game in a sphere they can't even touch yet.
I'm genuinely continually boggled that people are not seeing this difference and feeling threatened by machine "intelligence", although they are encroaching on something thought uniquely human, and I don't think the brain is doing some kind of information processing that can't be done in any other medium, but the number of connections and kind of processing in the human brain may make it hard with current technology. The number of connections in the brain is staggering, and we certainly can't simulate anything of its scale on the scale of computer we have now, so I don't know what it will take to get any bootstrap on the kind of intelligence computers aren't doing.
But certainly it has inspired the field of AI massively and I expect many new developments which will lead is into a future that resembles sci-fi books and movies.
I am also interested in what good sci-fi will look like in the future? It looks like "the singularity" is defined by the limits of our imagination. We cannot imagine what comes after it, so this means we cannot write sci-fi on it, right?
Why do you think it's an S-shape? As you say, we're heading for a singularity!
For instance there is a relationship between information and energy, so a system processing an exponential amount of information will need an exponential amount of energy, which the universe will eventually run out of.
That's a tall order. Despite all the fancy news headlines, which I must admit gave me existential anxiety at some point, it seems that AI still can't truly make a discovery. Also I really don't know if we know little or a lot, how can we really tell ? It could be argued we actually know a lot because it is ridiculously hard to make an actual big breakthrough in physics.
[0] https://en.wikipedia.org/wiki/Long_s
After this hype cycle ends, things will probably bump along slowly for a decade or so, then suddenly you’ll look up and they’ll be everywhere, and you’ll wonder “when did that happen?”
At least that’s the way it felt for other technologies to me. Internet, Bluetooth, cell phones, etc.
The reason this is incorrect is that “AI” isn’t one capability. It’s a huge swath of capabilities under the umbrella of “automate X”. All of them have been enabled by software and hardware improvements over the last decades. The underlying technologies aren’t that new in many cases either, it’s just the scale that’s different.
After we get bored of one huge new capability, another comes along. A year ago everyone was talking about automatic image generation. Now it’s automatic text generation. There are plenty of other things we’d like to automate. So the hype train is likely to continue for quite a while.
No-one knows.
We can however make educated guesses based on:
- Extrapolating the last couple of years of progress.
- Scientific studies of LLM quality scaling in proportion to input data size, parameter count, and total training compute ops.
- How much more training data is available.
- Reasonable budgets, especially the cost-efficiency of inference, which appears to be more limiting than training, especially for long-term usage and profitability.
- Efficiencies such as 4-bit quantization, algorithmic improvements, etc...
Based on the above, there is at least a factor of 10x scaling available in many directions within a year or two, but not likely all at once. E.g.: 10x context window size at the same intelligence level? Sure! 10x inference speed? Doable! 10x cost reduction? Coming soon!
All at once? Not yet, and not for a while. Everyone is hardware constrained, and demand is pushing up prices and limiting the training scale maximums.
Maximum intelligence is much harder to predict. The current generation of AIs are trained on human-authored text as their input data. They're trained to predict that text. That means that they're "blurry JPEGs of the Internet". More training might make them "sharper JPEGs", but not necessarily smarter because the Internet didn't get smarter.
They'll better model humans, but they'll still be modelling humans, not superhumans.
That can be fixed with self-training, etc... but that will take a lot longer. I'm guessing 5-10 years, which is in line with the predictions of AI futurists that seem to know what they're talking about.
Many of our jobs will remain safe... for now.
Imagine a hypothetical scenario where some ultra-intelligent beings delivered a book with all the answers humans ever wanted, but we needed an IQ of 500 t understand it. This IMO is where we're heading.
There might be some pretty sweet low hanging fruit to attain along the way, but at some stage, we'll hit a ceiling and the only way forward would be to augment our own intelligence.
The child-parent analogy stretches to describe it. A young child has an extremely fuzzy picture about what is going on - how did this house get here, why is there food in the fridge, why is the food wrapped in this clear thing, why does Dad disappear for a few hours every day, etc. At a young enough age they're not even thinking about it.
Yet it all happens anyway and the child is provided for. Even in the best case of perfectly aligned AI, that's our future if we allow it to happen (I hope we don't...)
Actually, Wikipedia articles are already sort-of a "distillation of the Internet" into a type of average. By having many people edit them, the individual personal quirks are smoothed out, leaving a type of average-of-humans result.
This reminds me of the "average faces" project about a decade ago where they blended photos of random people together. The results were often very attractive, because it's the small flaws that make normal people unattractive. The blending of many samples smooths out the flaws, resulting in a "perfect" but ordinary face. Not some post-human face, but a perfectly human face.
That's what I see LLMs doing: as smart as us, but no smarter. Great breadth certainly, but not greater depth.
That's easy with something like transistor size. That's hard with AI, because we can't put a number on how good it is. (The model size is probably not a good number for measuring AI progress.)
So nobody knows. We're all guessing.
So far "nigerian email spam" was easy to recognize. Grammar mistakes, bad translation, very bad wording... But now every fringe group, can generate gigabytes of text and arguments that make sense! This is the end of debate, consensus and interactive communication, as we know it!
Similar moment was in Africa in 1960, after they got AK45 and other cheap weapons. It was easy to dominate Congo River Basin with a few Maxim machine guns mounted on boats, when opponents had only spears. But when every tiny village gets comparable fire power, there are no colonies!
"nigerian email spam" contains grammar mistakes, bad translation, vary bad wording... by design. To weed out those unlikely to end up sending money.
I don't know how you can make an argument that makes sense about being chosen by royalty of a country to help them in some way.
this theory is popular, but ignores the simple fact that mostly it weeds out the 90% of people whose email provider offers spam filters which are entirely uncorrelated with gullibility, as well as how low-risk, high reward email exchanges with the tiny fraction of people that bother to reply are to anonymous scammers in faraway low income countries.
I agree LLMs aren't that helpful though: the better scammers are able to generate leads even more easily through copy/pasting other people's classified ads or genuine business enquiries. And yes, having fooled a relatively smart person with someone else's prose into sincere interest in doing business with them, they then reply with poor English, unsettling disinterest in anything other than the money and repeatedly reassuring you it's safe, and a payment proposal that makes no sense (not as a filtering technique but because that's the best they can do). LLMs only solve the first bit.
Do you thing it weeds out 90% of politicians who talk total gibberish?
At end, it will enforce people who are in your social network. Some random celebrity will be ignored. Dude who lives 1 km from your location will be boosted.
Maybe LLMs free up intellectual bandwidth for some, but what will we do with that increased productivity? Mostly scams for now, but I'm sure it'll be able to find many other net-negative or wealth-extraction applications.
Am I correct that they only compute information when dealing with a prompt? If so, that seems like a fundamental flaw. An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output.
Then you can find a way to include a described video feed and method of movement into the mix.
What kind of data?
You think you can just hook up a firehose of visual or auditory data to ChatGPT and have that produce anything meaningful? That's not even the basic format that ChatGPT operates in.
Furthermore, isn't part of the point of the dataset an LLM is trained on that it has to be at least moderately structured and tagged, or you end up with garbage in the output?
This already happens. When you play around with local LLMs you'll run into the situation where the model answers your prompt. After that it will generate a new query as a response to it and then reply to that too.
And now they removed the search plugin, probably they got sued for copyright leaks from the search engine results into the generated text. Using copyrighted data in the prompt is not necessarily legal. So we have to deal with out-of-date AI that updates once every couple of years.
That's just a hypothetical scenario. Usually LLMs don't generate text used indiscriminately to train new models.
In creative tasks, the generated texts are the result of multiple explorations and corrections, filtered by the user, then posted online and commented on by other people. So there is plenty of correction and feedback mixed into the process. A child model could learn something from this feedback.
Then there is code and math, where it is possible to run tests or double check by a different method. These tasks could be iteratively improved by LLMs by incorporating the feedback.
Another field ripe with feedback signals is using LLMs in games and simulations, where you can specify a goal easily but finding a solution requires search and learning.
And a related application was RLHF, where a preference model was trained on human judgments and used to fine-tune a LLM. That preference model can be used to filter bad training data as well.
To generalize, LLMs alone can't do it, but LLMs with external systems can get feedback and improve. The garbage-in-garbage-out scenario can be avoided with a bit of external feedback.
Not "these models" if you're referring to large language models pretrained and fine-tuned with Transformers. The only study I am aware of that explored the effect you discuss [1] had to make do with very different models as proxies: Variational Autoencoders, Mixture of Gaussians and a smaller LLM with 125 million parameters that hardly qualifies as one. It is not safe to extrapolate those results to really Large-LMs trained with Transformers, as it is not safe to extrapolate any results on one neural net architecture on another.
Note the Mixtures of Gaussians aren't even neural nets at all, but a completely different statistical model.
The question of what will happen once a few generations of LLMs have trained on their ancestors' outputs is an interesting one that has not yet been clearly answered. In the long run, future developments may even mean it's not really a valid question at all because such feedback retraining can't happen in practice- who knows?
I for one remain curious to see how this goes.
______________________
[1] The Curse of Recursion: Training on Generated Data Makes Models Forget https://arxiv.org/abs/2305.17493
Also there's not a singular search rank for a site it's rank depends on the search query so I'm not even sure how you'd apply that metric.
This is one of the questions far too few people seem to be paying attention to.
"Thinking" in any way that we truly understand the term requires consciousness, and consciousness requires much more continuity than LLMs have. It would need continuity of input as well as continuity of learning in order to even be able to begin to approach something we might recognize as consciousness.
A standard LLM is a static bundle of trained data that sits, inert, on a drive, with a process waiting for discrete input. When that input arrives, it does nothing to modify the trained data—the LLM's "memory"—it simply triggers a computational process that reads both the input and the trained data and produces an output based on them. This does not resemble in any way the structure of something that could be reasonably described as a conscious mind.
While I generally favor a requirement for recurrent processing, I have low but non-zero credence for certain feedforward networks being conscious. The point of recurrence is to allow information about itself to influence its processing. But It seems plausible that feedforward constructs can represent meta-information in a way that is computed as part of constructing the output.
1) They may feel like a red herring to you, but they are a big part of what a lot of people are talking about, and coming into a discussion that is clearly talking about whether an AI is conscious or how we could decide if it was and saying "that's all pointless, you shouldn't even be having this discussion" is kind of rude.
2) I am deeply skeptical that it is possible to have anything we could reasonably deem an "AGI" without it demonstrating at least what appears to be consciousness. Certainly, the "singularity" that many people talk about as being either an inevitability or an actual goal seems impossible without an AGI that can self-analyze and self-improve, which seem, at least to me, like they probably require consciousness, at least at the level being talked about.
Don't silence people, answer them. Say why it's actually important. Don't say that by saying "when you say it's not important, you're disrespecting the people who say it's important." That kind of thinking will take us back to the middle ages.
> I am deeply skeptical that it is possible to have anything we could reasonably deem an "AGI" without it demonstrating at least what appears to be consciousness.
The meaning of the word "AGI" is not an interesting thing to talk about. Call it what you want. If there's a conventional meaning to the word AGI that you're using, explain it and explain why an algorithm that isn't constantly running fails that. If you do this using the word "consciousness," you're passing the buck. You might as well be talking about souls for all the precision that the word "consciousness" has.
> seems impossible without an AGI that can self-analyze and self-improve
Thought experiment. What if you yourself aren't an AGI that can self-analyze and improve? What if you are two not-AGIs (by your definition):
1) a learner that improves the other based on new information, and
2) a controller that produces output,
And that they switch roles between 1 and 30 times per second, based on the intensity of the input. Would that mean you weren't conscious?
"In terms of the impact such a thing would have on the world", as you put it, would it matter if that model was completely wrong, despite its great predictive power?
Did we gain something when we figured out how the planets really move?
_______________
[1] https://en.wikipedia.org/wiki/Deferent_and_epicycle
Something different is not necessarily "wrong". a plane's flight is no less "true" than a bird. It's not flying the "wrong" way.
Trying to elevate our very poor and wrong understanding of "human" to be the same as "right" or "true" is very silly. Even Biology with its set of constraints does not always solve the same problem the same way. Who are you to dub one way "right" ? Makes no sense.
Even if it's true, then at best it means that the LLMs that we've created are only a small part of what's necessary for an AGI that functions as an "artificial brain": that not only are they not conscious, they are fundamentally incapable of it, and in order to create artificial consciousness, we have now gone as far along that particular path as we can (roughly speaking), and need to be looking into how to create the other parts of the "artificial brain", and how to hook them together.
By that reasoning, we die when we sleep.
Nothing.
> Am I correct that they only compute information when dealing with a prompt?
Yes.
> If so, that seems like a fundamental flaw.
Flawed in what way? It clearly doesn't need to be like us to be useful, because it's useful and definitely not like us.
> An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output.
This might be good, but it's not clear if, or to what extent, we really do that ourselves — the differences between working/short/long term memories, between episodic and skill, even linguistically between knowledge of phonemes, words, grammar, and the connection between those and the things they represent all being impaired independently of each other by localised brain damage[0].
Then there's how much this changes with some stages of sleep, and meditation to clear your mind.
Given the number of users (what is it, 100 million?), having it always on, continuously integrating, would still be inhuman even if the architecture was a perfect mirror of the human brain.
Also, if the AI is structured to be a "thinking machine", does that make it murder to switch it off?
[0] Cognitive Psychology for Dummies, currently listening to it as an audiobook.
For many people, the question is not—and has never been—"is it useful?", but "is it conscious?", or "when will it bring the singularity?"
Yes, it is useful as it is. But it is not conscious, and if it is a step closer to the still-very-hypothetical singularity, then it is not a particularly large one, because of these very real fundamental limitations.
But engaging with people who are clearly talking about whether AI is conscious by responding as if they're talking about whether it is useful could very easily be seen as engaging in bad faith.
How can we tell? We don't have a single working definition of what "consciousness" is, rather we have at least half a dozen wildly different definitions, some of which aren't testable[0], some of which include VHS players[1], some of which exclude many humans most of the time[2].
If consciousness exists purely in working memory and not short/long term, then LLMs may have it within any given context window, even if they're effectively amnesiac between contexts.
Possibly. Like I said, we don't know what consciousness is.
> But engaging with people who are clearly talking about whether AI is conscious by responding as if they're talking about whether it is useful could very easily be seen as engaging in bad faith.
It may seem clear to you, but it didn't to me; furthermore, the singularity doesn't seem to me to require any of the many definitions of consciousness that I'm aware of, merely the capabilities for problem solving that LLMs increasingly demonstrate regardless of if they're conscious or not.
[0] what even is qualia
[1] can record sensory experiences in memory and change responses based on this
[2] rational/logical reasoning
To analogize to humans, imagine if every person who was ever born was born with all of the knowledge humanity has ever produced (though no understanding of hierarchy of evidence or any way to verify the knowledge is still correct) and the ability to produce language as an adult. They each lived long enough to produce one continuous thought, which they could write down, and then they die. New people who are born can read one such note each and add to it, up to a fixed limit of words. All people are born with the same knowledge base, the same personality, and that could never change until their God updates the DNA from which they're born out-of-band.
Could such a people be expected to produce "all technological advances that will ever happen but all at once?" Or has the definition of the singularity become significantly more modest? It's not clear to me any way in which such things could produce science and engineering on their own, as opposed to aiding in the production as tools used by higher-level beings that stay alive, form memories, and continue learning.
Depends on the size of the context window.
Humans don't, and can't, learn everything from the ground up — we seem to have started with peer learning (it predates humans), which became culture; then we got (complex) language and could share more complex discoveries about the world; then writing and the dead could share their thoughts with the not-yet born centuries later…
…and, apparently, with machines. (Imagine explaining to Plato that we'd made metal from sand and used amber to make it discuss his philosophies).
If the context window is enough to create part of something new, then yes, I'd say an LLM can likely invent (or at least contribute to inventing) ${interesting novel tech}.
That said, I don't buy the "everything all at once" meme of the technological singularity, as I think any model with infinities in it is wrong — my preferred analogy is the technological event horizon, which is more like "you can't make good predictions past this point", akin to the event horizon around singularities in relativity.
https://kitsunesoftware.wordpress.com/2022/09/20/not-a-singu...
Nothing stops you from running a loop that involves other systems such as long term memory (vector /dev storage), visual pre-processor (CNN), auto lora, and more.
That’s the fundamental flaw with most of the criticism - the tech is out only a few short months in the hands of everyone. The disruption will come from plugging it into feedback systems.
Ari Schulman is editor of The New Atlantis, as well as editor of TheNewAtlantis.com and of the New Atlantis Books series.
His writing has appeared in the Wall Street Journal, the New York Times, The Atlantic, The Hedgehog Review, Commentary, First Things, and Slate. He has previously been a research assistant in the Opinion department at the New York Times, an ontological engineer at Cycorp, and has degrees in computer science and English from the University of Texas at Austin.
I'm still not certain. I have a degree in computer science, but it had no AI focus, and even if it had, it was so long ago as to be mostly irrelevant to what is currently called AI if I had not been keeping up independently in the field.
See God in it if you want. Believe that it has feelings. Believe that it can think for itself and develop agency over it's future... one day. People believe all kinds of things about the natural universe that make no sense but it makes them feel something.
LLM's are what they are. No more, no less. Don't believe the hype, stick to the science.
This is like saying the difference between Cillian Murphy and Oppenheimer is pretty blurry because, look, we can watch Murphy do do all the same things Oppenheimer did!
For some reason with "AI" we get all mystical and act like imitation = the real thing. Just because I can convince you I'm a doctor doesn't make me one
Because our society has decided you need to become certified to be a doctor, and to do that you have to be a human being. There's no reason to think an AI couldn't have the same level of skill as a human doctor.
The film comparison doesn't work for me, Cillian and Robert are real people.
I think at the end of the day it's how you view the philosophy of the human soul. For me, it follows that if brains are matter and can produce the illusion of consciousness, so can electronics.
> Because our society has decided you need to become certified to be a doctor, and to do that you have to be a human being.
For the record, I don’t think this addresses the point. The point is that if a person can convince another that they’re a doctor, they still might not be a doctor.
Your reply, that a computer can never convince someone that they’re a doctor because it’s a computer, doesn’t address the point. Presumably the point was made by a person, who would not be categorically excluded from being a doctor.
But it’s also not a great example because “doctor” in our society isn’t just a super smart person who knows about medicine. They also need to be licensed. There’s no amount of convincing a doctor can do to a patient that gives them a license.
So maybe a better formulation of the original point would be just because I can convince you I'm a computer programmer doesn't make me one. And maybe that’s true, I’m not sure.
There is no specification available for sentience/intelligence. So people who want to believe an LLM is sentient can jump to any conclusion they want and they wouldn’t necessarily be wrong. But they’re not right either.
I think it’s highly reductionist to compare an LLM to an intelligent, sentient being and claim it’s basically the same thing. We know how model training works, how transformers work, and we know what the expected results are. Not so for consciousness, sentience, or intelligence. Scientists have been making breakthroughs in our understanding of these phenomena but it’s still incomplete.
LLM’s are useful for one thing. They’re algorithms that run on a machine to perform work for us.
Ie we will be able to describe how a function of `add(1,2)` is most clearly definitely not intelligent. We will understand the internal makeup of these neurons better, the whole system will be deterministic and understood - just a very large `add(1,2)` .. so it's definitely not intelligent, of course.
Yet it will still produce output that is fully and truly turing-proof. If you let it run, it retains, stores, "behaves", etc. It's clearly just `add(1,2)` in a loop after all.. we will understand it so well that it's obviously just a trick of our mind that it seems real.
The conclusion in that space will not be one of the machine .. but of us. If we can't tell it apart from us.. and if we know it is most definitely not alive or even mysterious.. what does that say about us? The existential crisis will be neat.
Less than people think. We know that being a biological system differentiates us. If intelligent systems tend to converge where language is concerned, so be it. It doesn't mean the AI warrants any special consideration. It's still a tool.
That concepts like "consciousness," "free will," and "qualia" are nebulous precisely because they were invented by us to describe things we don't understand yet we feel obligated to insist are special and unique characteristics of our identity.
"Can you pay attention to this part of your experience?" is a very black-and-white, non-nebulous question with an easy yes or no answer. Not sure what you're reading into this.
Consider a video camera. It seems like it is a consuming a "stream of experiences". But most people wouldn't consider it it conscious. Why not under your definition? It works when on, and is stops when turned off. Black and white. But clearly we are missing some nuance in the details. Maybe we also require some interpretation of the experiences? I can spin a convolutional neural net that identify objects within a video: interpretation. Does that make it conscious? No? Then we need something else to define it. Maybe response to stimuli? We can program machines for that, but then also things like fire respond to environment interactions too. And so on. It really isn't black and white after all.
What does "pay attention" mean?
I've come to suspect that a habit of using the passive voice when discussing these phenomna contributes immensely to their ongoing currency. Steering away from this by asking questions like, "_who_ or _what_ is attending to/experiencing the stream?" may be a useful way to ground these statements in material terms.
Consciousness, on the other hand, boils down to what gives rise to the sensation of self-awareness, and what that sensation actually is, and it's not a given that requires any form of dualism (or that any form of dualism would solve the question at all) or that a strict materialist world can't involve self awareness.
For what we know, self-awareness could be a property of matter itself, and the only thing different between "conscious" and "non-conscious" is getting to a level of sufficient reasoning-ability to actually be able to express it.
Nova/PBS recently did a number of episodes on the number of tricks your brain is playing on your at anytime. Once you start seeing the shortcuts the brain takes in action the entire process is far less magical.
Because it's still equally nebulous.
No, its not.
Its the quality of “experiencing”.
Which is impossible to define in objective terms; it is the essence of subjectivity on which all other subjectivity rests. It’s not merely “we don’t know how to determine whether an AI or camera has it”, it is categorically impossible to determine if something has it. The popular narrative about which things in the natural world have it has evolved radically over time, but in no case has there been, or could there be, any empirical basis for this; its all just vibes.
And your definition is now circular and can not be used that way.
Hence the claim that the original definitions were nebulous.
Those seem equally nebulous.
> "Free will" means "a feeling that one could've made a different choice"
That doesn't seem like the ordinary definition of the term, either in philosophy or in everyday usage. I would think most people would define "having free will" as not just feeling that one could have made a different choice, but rather it actually being physically possible for a different choice to have been made.
> "qualia" means "specific characteristics of experiences that one can attend to"
I think those are also equally nebulous.
> One need not "insist" on anything to believe they are unique characteristics of our identity.
The insistence is necessary, and invariably comes up. As soon as I say to someone "given your definitions of these concepts, I don't think they exist" they will say "well, I've started with the foregone conclusion that I cannot deny the existence of these things, so either you're plainly wrong, or we need to tweak our definitions and understanding of these things."
They aren't. It's literally happening right now as you read this - a stream of experiences is present. If that is the case, YES it is consciousness, if nothing is happening, NO it is not consciousness. Black and white - literally as non-nebulous as you can get.
We often discuss things that would challenge or overtake us. Hypothetical advanced aliens, artificial AI, etc. However i don't often see pop culture discussing the opposite - us, or even life, being challenged on it's uniqueness. Plus then that leads to all sorts of self-autonomy discussions (which is discussed in pop culture imo).
I dunno, i suspect "people" won't handle it well. Hell, there's entire religion(s) built on the foundation that we're uniquely gifted as a product of being crafted in a gods image. To question us could question god. I find these questions neat, interesting, etc - but socially i suspect it would be troubling, contentious, frightening.
A lot of the hopes in regard to AGI in LLMs seem to focus on just making the models bigger, or feeding them even more information. On the other hand, some researchers in the AGI space seem to move in a completely different direction, trying to use neural networks to simulate different parts of a brain, moving away from language altogether.
But what I'm wondering is this: might it be a more productive avenue to consider an LLM not as an entire mind, but as a single part of an hypothetical artifical psyche, which excels at some of the overarching tasks of consciousness (such as acting within a cultural context and epistemological framework), but can delegate other aspects to more fitting subsystems? Could it be combined with other networks optimized for different tasks? Potentially, this could just be done by having inputs from other parts of this artificial psyche fed in as prompts, translated by an encoder/decoder of the likes of CLIP, i.e. a decoder that can translate ideas between different liminal spaces.
The work that I keep remembering in this context is Jaynes' Origin of Consciousness in the Breakdown of the Bicameral Mind [1], which I stumbled upon here on HN [2] [3]. Jaynes believes that consciousness is not an innate ability, but a cultural achievement - in order to become conscious, one first has to learn (through others) to conceptualize consciousness, and to think of oneself as a thinking being. He insists that consciousness is not a necessity for human agency, and that humanity has been building civilizations and using technologies long before humans discovered (or achieved) consciousness. Jaynes puts forward the idea of an ancient, non-conscious human mind based on, upon other things, "verbal hallucinations" - thoughts and memories that involuntarily enter as associations, but whose origins are interpreted by the mind to not be within itself, but somewhere outside of it, such as in a divine monologue; thought, to the pre-conscious mind, is the incessant blabbering of the Gods.
It seems that LLMs would excel in the role of this "camera" of the mind - as the source of divine monologue and divine instructions, that can then be compared with memories, interpreted and executed by more specialized subsystems. Might it be feasible to give up the plan to make AGI self-aware altogether and instead build an AGI powered by a GPT-4 ascended to divinity, just as ignorant of its own existence as ever?
[1]: https://en.wikipedia.org/wiki/The_Origin_of_Consciousness_in...
[2]: https://news.ycombinator.com/item?id=27917316
[3]: https://news.ycombinator.com/item?id=27923444
https://en.wikipedia.org/wiki/Assembly_theory
https://arxiv.org/abs/2206.02279
I feel like people have this vague idea in their head of what everyone is trying to build, but don't really have it defined in a concrete way.
So basically we're never going to achieve some consensus about whether an LLM/AI could have it, because we can't even agree on whether dogs and cats have it. We can't even all agree on whether all humans have it.
Maybe some day we'll have scientific instruments that will allow one being to 'psychically link' with others and experience the other's subjective sense of being. But until we've got something like that, I think the matter is effectively closed to empirical inquiry.
Well, one way is to observe similarities in the I/O behavior of these various systems, and in particular if they exhibit any evidence that they have goals and desires, and then beyond that if those goals and desires extend beyond merely that necessary to survive and reproduce. So for example, the fact that some humans seem to want to do things like read and write and make art is IMHO is evidence that they are more conscious than salmon.
For instance, with thumbs and a terrestrial lifestyle, humans are more capable of producing art and technology than just about any other animal. But I don't find this to be a persuasive argument for human minds being more sophisticated than orcas. Even if art is the answer, our lack of insight into orca language makes it difficult if not impossible to fairly compare them to us. Maybe Orcas have poetry so sophisticated and beautiful that it puts all human art to shame, but it's completely inaccessible to us because we don't understand their language. Or maybe they're far smarter and more aware than us and could compose such poetry, but choose not to simply because they don't value it.
> if they exhibit ... evidence that they have goals and desires ... beyond merely that necessary to survive and reproduce
Orcas qualify because they play with their food. It's quite gruesome to watch. (And, BTW, this particular behavior is evidence that either orcas do not have a theory of mind or they lack empathy for animals that they eat. The former would be the charitable interpretation.)
What would it mean for humans not to have it? The debate is over the nature of consciousness, not that humans (and many other organisms) lack any sort of consciousness. So when we see color or feel pain, what is that? We are conscious of colored things and painful sensations. There's no real doubt about that, unless you're solipsist about other minds. Which good luck if you are, I guess.
I don't believe human P-zombies exist, but how do you empirically prove they don't? Nobody has been able to devise an experimental test to determine whether a person, animal or AI has a conscious experience or whether they mimic all the outward signs.
> There's no real doubt about that, unless you're solipsist about other minds. Which good luck if you are, I guess.
Exactly. It's fair to write off people who get solipsistic about other human beings because those people are fairly rare and are beyond helping anyway. But if we're talking about animals or AIs, the 'solipsist' position becomes more common and there's no real good way of debunking it. Consider the problem facing animal rights activists who wish to persuade the public that fish have feelings. What scientific test for fish having feelings can you use? Put a fish in an MRI and you can see the fish brain doing stuff, but the "fish don't have feelings" camp doesn't find that persuasive. You could try performing the mirror test on fish. Most fish would probably fail it, but what would it really mean if they didn't? The feelingless fish camp would just say the fish behave as though they have a sense of self but actually don't. How do you refute that scientifically? It seems the most effective way to persuade people that fish have feelings is to resort to sophistic tactics; appeals to emotion, bullying and browbeating, or simply just laughing at people when they say fish don't have feelings. Make them ashamed to spread their belief to others. These kind of tactics are distasteful but generally effective.
With AIs or LLMs, the problem is even worse than it is with fish. Fish at least have biological brains, which suggests they have some commonality with us. Fish have faces, so some part of the human brain is wired to empathize with them. We have a fossil record that shows we came from fish, and this relationship perhaps makes it easier to believe that fish have some sort of subjective consciousness experience that is similar to ours. In these respects, AIs are further removed from us than fish are, so it will be even harder to persuade people that AIs could have the same sort of conscious experience as we do. Even after having deep philosophical conversations with an AI in which the AI proclaims to have a conscious experience, a lot of people will say the AI doesn't actually have that and was merely trained to speak as though it does. I don't see any clear cut resolution to this sort of doubt. There's no empirical test that you can perform that gives an indisputable answer one way or the other.
Games.
From the first clever bloke who came up with an algorithm to play a card game against himself (ie, Solitaire), to the Mechanical Turk, to Big Blue, the the ghosts in PacMan, the NPC’s in Counterstrike, etc, etc., there is a fascination with making something that can ‘beat’ us.
Humans have a need to compete.
We’re drawn to the idea of a machine that is so smart and clever that it poses a genuine challenge to our wits.
The ‘real deal’ that most have in mind isn’t some virtual assistant that can help you calendar things or write letters. No, the ‘real deal’ is always described in terms of SkyNet or sinister AI overlords, or malicious intelligent robots.
I don’t think that stuff is hyperbole.
Rather, I think that is precisely what we want… we want something that can destroy us… something, against whom, we can compete with our survival at stake.
In short, the ‘real deal’ has been staring us in the face for centuries; we make movies and write books about it… particularly in Sci-Fi. The goal IS SkyNet. The goal IS HAL. Let’s be honest… that’s what we are shooting for — the ultimate competitor that, unlike anything else on this boring planet, has the wits to take us out.
Whether it’s bored trailer-trash playing Russian Roullete, or bored Billionaires descending to the Titanic in a jerry-rigged sub, we have a very strong need to make our lives interesting by betting our lives themselves.
The purpose of AI is to make a challenger who has the ability to wipe us out, with intent.
That’s the ‘real deal.’
> How tall was the president when JFK was born?
I infer that "JFK" is being referred to as "the president", regardless of what stage of life he's at. Therefore the correct answer would be the length of his body as a newborn, with nothing to do with who was the president at the moment of his birth.
For example, given the headline: "Was the president strong at writing essays when JFK was a high school freshman? Read this essay to find out!", would you expect an essay from Herbert Hoover while he was in office, or an essay from JFK when he was in grade 9?
> How tall was the president when JFK was born?
as, How tall was the "reigning" president, (which in the year 1917, Kennedys birth year, was Thomas Woodrow Wilson) on the day that JFK was born."
Part of my bullishness on LLMs is that a lot of the criticism is so bad, and if they weren't the real deal there would be more low hanging fruit to attack them on. Cryptocurrency was having real problems from the first moment people realized how much data they would have to download to initialize a node.
There's a general unstated assumption that a truly intelligent entity wouldn't make mistakes, which is sort of contradicted by, you know, the existence of humanity. I think making (certain kinds of) mistakes is a _sign_ of intelligence, rather than a sign that it's not.
If someone gave me the question "who was the president when JFK was born?", the answer is clearly not JFK, so it isn't correct to infer that JFK is the president whose height, party affiliation or essay writing skill is being asked about "when JFK was born".
If someone gave me the headline "was the president strong at writing essays when JFK was a high school freshman?" I would assume that either they were trying to trick me or that they were a non-native English speaker or computer program that didn't understand how English syntax or the concept of presidency worked. If you flip it to "was JFK strong at writing essays when the president was a high school freshman" then yes, I'd consider both to refer to JFK because "the president"can't be identified as anyone else, but I'd also consider it to be bad writing by someone cargo-culting the idea of elegant variation...
I don't think it's as complicated as all that. Most English speakers wouldn't refer to a single person by two distinct proper nouns in a sentence like this, so I'd assume they were referring to two separate people. If they had meant to refer to the same person, they would have used a pronoun. I'd probably guess they were either talking about the President at the time at which JFK was a freshman (Hoover, I think) or the current President at the time the question was being asked.
On the other hand, if the question was "Was President Kennedy strong at writing essays when JFK was a high school freshman?" then yeah, I'd agree with your take. In this case, it's clear that the writer is intentionally referring to the same person but using two different proper nouns, which is a very odd phrasing.
I agree with this (though elegant and inelegant variation is a thing). But nor would any English speaker be likely to ask about the skill of an anonymous holder of a position with reference to an ambiguous date range associated with a time in which a later holder of that position would be extensively practicing that skill, which is why I classed it the sort of sentence with no natural meaning you'd only conceive in order to be deliberately ambiguous (or because you don't understand how to write English properly).
The original sentence isn't ideally constructed, but at least the president when JFK was born is a natural and easily identified sentence object, and it would be odd to ignore that in favour of an alternative interpretation that also involved interpreting "how tall" as being more likely to refer to the [probably-not-recorded-for-posterity] length of a newborn infant (who wouldn't be idiomatically referred to as "tall" or the president in the context of his birth) than the height of a person who was president in the relevant time period
Yes, you can deduce that, since the question is nonsensical (trivial) if "the president" refers to JFK in that context.
> If you flip it to "was JFK strong at writing essays when the president was a high school freshman"
This is a perfect illustration of why the original sentence can be read as referring to JFK, since in speech it is common to have ill-defined references that are later clarified, and there's no formal rule that enforces that all terms must be defined before use.
References that are later clarified is exactly how we arrive at "the president when JFK was born" as defining Woodrow Wilson as the object of the sentence (and it makes much more sense to ask about how tall President Wilson was in the context of 1917 than to wonder how "tall" baby "president" JFK was in 1917).
Whereas "when the president was a high school freshman" can't be used to identify anybody else [unless there is a time period in which a high school freshman was the president!], even if the sentence hadn't already made it clear that JFK was the object of the sentence. But it's still an example of bad writing.
Even when there's no ambiguity, we don't do sentences like "The SpaceX founder was about 18 inches tall when Elon Musk was born" which is basically how you're implying the president question - which has a much more reasonable alternative interpretation - could be answered...
It is possible to write a truly ambiguous sentence in English; nobody is arguing otherwise. But whilst the original sentence might make you think a bit, it's not naturally interpreted as being equally/more likely to be a question about the not-tall not-president in the context of the time period instead of the person with height and presidency in the context of the time period, unless you're being obtuse or lack fluency in the language.
> "Was the president strong at writing essays when JFK was a high school freshman? Read this essay to find out!"
You write:
> "when the president was a high school freshman"
These are very different circumstances, of course.
If the original sentence had "reigning" before the word "president", then I would absolutely agree with the given interpretation.
I've noticed that Americans call ex-presidents "president", and ex-governors "governor", so that merely adds to the confusion. English is not my native language, and I'm not American, so perhaps I fall into the "lack fluency" category. Conferring with other native speakers nearby, every American immediately reads it as intended, whereas Brits and Canadians do not.
I conclude that the original sentence is poorly written, and that I misread it.
The impressive feat is the ability of researchers to manipulate a generic statistical model paradigm to fit large amounts data and be reused for something apparently useful.
Whether that path has a future depends not on AI bootstraping itself but on said researchers to keep building on this paradigm.
The black box and inscrutable nature of these algorithms works against them. I.e. reaching the next level in some purposeful manner requires more understanding. Human understanding.
Nice neologism! But wouldn't "AI-dolatry" be even better?
Asking for permission to use this going forward :-)
There are easily stated problems like "Reduce US car accidents by 50%" that should be solvable with AI as it stands today. This seems like a useful and achievable goal that would save money and lives. I've never understood why full self-driving was the problem most people chose to solve instead.
But perhaps I'm misunderstanding your point.
The appeal of self-driving is usually just financial.
No, it doesn’t require a law or a common agreement, it just requires the absence of a law preventing it and salable benefit in a competitive market. Cf., computer vision and other AI components for lane keeping, throttle control, and automatic braking, which is “solving some percentage of car accidents with AI”.
IF you really want to solve that, that’s an urban design problem that doesn’t take AI at all; it isn’t solved because there is insufficient desire to solve it, not because of technology. But as far as AI-use-in-cars goes…
> This seems like a useful and achievable goal that would save money and lives. I’ve never understood why full self-driving was the problem most people chose to solve instead.
In the space of auto safety technology it’s not. Driver assistance short of full self-driving is much more widely deployed, and continuously improving, by far more automakers than FSD. Full self-driving gets the media attention because its dramatic and showy.
That said, it would be great to see more public focus on solving problems like keeping bikes and pedestrians from being run over. In my town there's a lot of talk about the issue but it's not been backed up by initiatives that actually change the numbers. (Which are bad.)
That's not something that needs AI at all. That's a political problem, and the solutions are not ones that Americans like.
uh... how?? aside from obvious non-starters like reducing US car usage by 50% because that's not going to happen (not today anyway)
"It is too early to say that the new AI class is an inherently antihuman technological paradigm, as social media has proven itself to be.
But it is not too early to suspect that AIs will dwarf social media in their power to disrupt modern life.
If that is so, we had better learn some new and unfamiliar ways of interrogating this technology, and fast. Whatever these entities are — they’re here." -Ari Schulman
People that mistake an large language model (LLM) for anything other than a LLM make some fundamentally broken assumptions.
Are ChatGPT, Midjourney, etc. a fundamental leap in the state of the art when it comes to allowing computer systems to understand what people mean and return something useful based on it? 100%
Is ChatGPT or the like going to become self-aware, compromise other computer systems, etc? No more than your shoe is going to take over your foot.
There's far too many people worried about "AI" that don't have enough context to realize they're fearing a non-sentient tool that has zero agency, and will not for the foreseeable future.
Somebody wake me up when the panic is over.
Agreed. It's strange though, to be honest I don't see much if any worry about self awareness, i think anyone who knows anything understands that's not the issue. The "issue" if you can call it that is how it will impact society from a labor and content perspective. How much synthetic content will be perceived as true and assumed to be correct simply because we haven't had time to adapt to the fact that the rate of synthetic content is exponential now.
That will be a problem for sure. But people believe all sorts of easy to debunk things already and that list is ever-expanding, so I don't know if there's any cure for it.
As far as content goes, I think human beings writing out boilerplate anything is something that's not long for this world.
Labor is a trickier one to judge, but I'm not too worried about a large negative impact in the immediate future.
People thought that modern appliances would usher in a new age of leisure, and the truth was that while they removed a lot of drudgery, work expanded to fill the vacuum and the percentage of the week spent laboring didn't significantly decrease.
On a positive note, I think the kind of work people are subject to now is better because of modern appliances, so I wouldn't be surprised if the same holds true for all the labor problems that can be handled by neural net / LLMs.
Does language understand itself? Some of it does, and some of it doesn't.
Remember, LLMs aren't the person writing.
I say this because statements like yours are a bit like saying calculators don’t understand arithmetic. In some sense maybe they don’t. But in another sense, they actually understand arithmetic better than anyone on the planet.
So maybe ChatGPT does understand language. Maybe it understands language better than anyone. What it doesn’t have are real-world experiences.
A large language model is a collection of relationships between "tokens" (arbitrary groups of characters, similar to words). This is intended to exist in the same conceptual place that language does; language being defined as a collection of relationships between words and punctuation.
That entire section is one big confused contradiction. The author argues that CYC (Doug Lenat's project to hand-code a gigantic database of logic rules encoding common sense knowledge) "can do deep reasoning" because it can answer this question correctly:
Unlike Google and WolframAlpha who can only find the closest association (the elevation of the town were JFK was born). Still, the author says, CYC would never "challenge humanity’s unique rational status, any more than computers that could solve equations did" because it doesn't understand natural language, only logic formulae. And yet, the author seems to be arguing that ChatGPT does understand natural language because... it can answer the same question as CYC; albeit in natural language, unlike CYC.I think the author has confused the ability to return results in natural language with the ability to understand natural language. By that token, ELIZA (mentioned at the start of the article) must have also been able to understand natural language, even if it didn't have CYC's ability for "deep reasoning", just because it could parrot its user's input in natural language.
That doesn't make sense. Understanding is clearly something that needs to happen before utterances are formed. Just looking at the output of a system doesn't tell you anything about the internal workings of the system, that's an error that many people keep making in this entire discussion about "AI"s. Look at the sky: the sun looks like it's turning around the Earth. Well, we know it isn't. What we see on the surface is rarely enough to explain what is going on "inside" (or outside, as the case may be, for the sun).