25 comments

[ 3.1 ms ] story [ 30.4 ms ] thread
Last year we were months from a quantum computer. The year before that we were all doing to be on the blockchain. The year before that everything was moving to VR and the Metaverse.

This year it's AI.

Yup, the AI Extinction meme is both a distraction from real harms and an effort to provoke regulations to create a moat for the big players (see famous Google memo "We Have No Moat, And Neither Does OpenAI"[0]).

Most importantly, while the results of the new generative LLMs and image-processing "AIs" are impressive and sometimes useful, they are nowhere near intelligent, and have only a passing resemblance to reality. There is zero ability to actually abstract and wield concepts, nevermind conceive of and seek a goal. Fun-house mirrors of human input are not any kind of existential threat to humanity or to life on earth.

However the distortions they create can amplify certain human actions, particularly disruptive actions, and distracting from managing those harms is a problem.

[0] https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...

While I’m sharing your assessment of the motivation I do not think it’s great to take a snapshot in time on an exponential curve and argue ‘it’s far away from being intelligent’

When something goes from not being able to do something to almost human level, arguing that almost isn’t a threat is not exactly wise.

Of course a (flat) linear line looks very similar to an exponential early in on the curve...

But I suppose that cuts both ways :)

>>When something goes from not being able to do something to almost human level

Point taken, and yes I have read Bostron's book and others, so I understand the runaway concern.

However, the issue I have is with the "almost human level". We must look at the mechanics of what is happening. The technologies of mirrors, lenses, or photography have been able for centuries to create images at far beyond the level of most humans. Yet, capturing the vectors of incident photons is in no way reproducing human capability.

These "AI" tools are capturing and reproducing the vectors of existing human output of words and/or images. The output can be impressive in both the parlor game and useful senses. Yet, there is deeply no "there" there. Just look at the "single banana problem", or the massive confident hallucination problems.

Consider the responses of taking a Polaroid instant-photo camera to a pre-industrial society with sophisticated artistic capabilities. They'd at first regard the camera/printer with astonishment and consider that it may have artificially human capabilities. But they'd soon figure out that the camera has no concept of a person, and cannot compose one image of two people present only at different times.

We are very much like that society regarding the camera-printer, and we're still in the astonished "Holy Cowabunga!" moment.

But, the more I use it, the more I'm convinced that this is, like a camera in many situations, an occasionally extremely useful tool, but nowhere near any level of actual intelligence.

> Most importantly, while the results of the new generative LLMs and image-processing "AIs" are impressive and sometimes useful, they are nowhere near intelligent, and have only a passing resemblance to reality. There is zero ability to actually abstract and wield concepts, nevermind conceive of and seek a goal. Fun-house mirrors of human input are not any kind of existential threat to humanity or to life on earth.

I am no doomer but such position is utterly ignorant. GPT-4 can do plenty of complex cognitive tasks; hell, it can even do regression if you give it a bunch of data. If you think that's not intelligence then I don't think we humans are intelligent either.

The question is HOW is it appearing to do regression or other math?

Is it abstracting the meaning and context from the data and then performing an analysis, or just spitting out something resembling other training data it has seen after something resembling your prompts?

This is easy to demonstrate with simple arithmetic. Ask ChatGPT4

what is 5 times 7? ChatGPT The product of 5 multiplied by 7 is 35.

Now ask it what is 19748312 times 12347123? ChatGPT The product of 19,748,312 multiplied by 12,347,123 is 243,602,945,733,776. This is wrong; it is 243,834,837,306,376

The LLM has zero understanding of the problem, and it is not doing the math; it merely reflects back at us the closest vectors in the training set.

It is no more doing your regression problems than it is doing the math here.

And no, I cannot do that multiplication in my head either, but I can understand the terms, and either do the longhand arithmetic or put it into a calculator.

Similarly, the human who knows how to do the regression UNDERSTANDS THE PROBLEM and applies a calculation method. The bullshitting human just spits back a plausible answer, which is what you are seeing from your LLM.

What is ignorant is mistaking a parlor game, admittedly very sophisticated and often useful, for intelligence. And I do find it quite useful myself, but I do not mistake the output for something it is not.

Edit: fix the asterisk times sign interpreted as formatting

I'm not convinced human intellect isn't doing a similar thing with math when we are doing it fast. When I solve an arithmetic thing in my head I actually am not understanding or going by "the steps." I've just seen it so many times I have a kind of "sixth sense" so to speak. I.e. I'm kinda just pattern matching in the vector space of my memories. When I do this am I no longer using my intelligence?

When we do math mistakes it is a form of hallucination just like for the LLM. In such instances you can ask a human to show their work and go step-by-step in which case they are less likely to "hallucinate."

Do the same with an LLM and likewise you'll get a better answer as expected. In your example though it is true you're unlikely to get a LLM to converge on the correct answer. Humans also struggle to reason well to some things beyond their understanding though.

LLMs can very well give their reasoning and steps and understanding for some things but not other things. Why isn't your response to this not that we just haven't taught LLMs the other things well enough yet and that we will probably get there soon enough? Would it be that surprising for GPT-5 to be able to show all of its mathematical reasoning for the question GPT-4 fails to answer now?

This is all besides the point. If AI is created that outperforms humans on everything and anything by several orders of magnitude then some response along the lines of "it isn't REALLY intelligent" just sounds like either some high-falutin' semantic jostling being done by an academic divorced from real-world implications or a human drinking some serious copium wanting to still feel special for being human.

I don't care how it works and that it is using processes completely different from human psychology. I care whether we can create AI in my lifetime that will outcompete me for any job I could ever hope to do. That and the totalitarian state implications of such power in the wrong hands.

Interesting points.

>>I'm not convinced human intellect isn't doing a similar thing with math when we are doing it fast.

Yes, this may be somewhat similar to the LLM's method, depending on how close we want to consider long-ago forgotten acts of memorizing multiplication tables but where the lesson still 'stuck'. Maybe that's simply regurgitating pretrained closeness vectors, or maybe it's a table lookup.

There's three things we need to look at: 1) good results, 2) errors or hallucinations, 3) underlying technology or methods of calculating.

If we look only at results, we can easily get fooled. That said, I agree that even a non-intelligent box that still yields reliable results can be enormously useful, have massive impacts, and be massively dangerous in the wrong hands. Of course, we already have such tools in photography tools that are vastly faster & more accurate than any artist and computers that make make larger & better calculations, etc..

Looking at the errors and hallucinations, such as the arithmetic or single banana failure, we can easily see that these generative AIs fail any time the right answer would require wielding even a slightly abstract concept. They haven't got it.

This is not to say they couldn't get it with more layers of some sort to extract & wield the concepts, but that's a different machine. Two months or two decades or two centuries away? Who knows?

Plus, the internals of the generative AIs tell us the same thing - they are massive collections of vectors of likely associations of words or bits, and are returning the most likely set, wrangled a bit by grammatical or artistic filters.

So, while results look good, both the errors and methods indicate a lack or intelligence.

Also, this is the same trajectory of other "AI". It was long thought that making a world-class chess computer would achieve intelligence. Same for Go, or Jeopardy. All these were achieved, everyone was impressed, and then everyone noticed that it was just a big computation engine.

I do expect that we'll get there someday, and that Bostrom's Superintelligence explosion will be a serious concern. I even thought that ChatGPT might be approaching it... but then I started using it and reading more about the internals. We're safe on that lineage, but who knows what others are working on?

I used to think this same way, especially given I have a biology background. It is indeed true if you look at the internals of chatGPT it is doing a lot of basic and dare I say dumb suboptimal things.

However I find the analogy to flying compelling. It was thought to be decades days before the Wright Brother flight because of how simple, and seemingly dumb the engineering was relative to evolution's design in making a bird fly. There is a TED talk on this. It took us well over a generation later to build bots that can fly the way a bird does. That's how much longer it took us to understand biological flight. Yet by that time we had already made machines that can fly faster than any and all birds. We didn't need to mimic biology's design to get something that outclasses it.

Could be the same with AI. We might be able to use engineering methods that are dumb relative to how biology sculpts intelligence, and yet still create an AI that outperforms all humans on all intelligence tasks. We may well create AGI that we end up coexisting with for over a generation before we figure out how intelligence in the biological brain works.

And because we keep comparing AI currently to how biological intelligence works I am concerned the same thing will happen where a few days before it is released most people will be saying AGI is decades away.

For me at least, this consideration makes "centuries away" look off and "decades away" seem far more likely... Which is in my lifetime. Yikes!

Yes, I definitely agree that it is possible that an AI could emerge/be built without any direct biological analog, and flight is a good example (and the wheel is another — enables us to go faster than any legged creature but no biological analog).

The questions are whether this generation of AI is actually intelligent (seems we ~agree different degrees of "Not yet"), and whether will lead to real intelligence and how close that is.

Some of this is likely how we define "intelligence". I'm thinking of it in the human+/- level AGI, with the ability to abstract thoughts, sort facts from fiction, reason, and reason about itself sufficiently to set goals. Under this, I consider chess computers, AlphaGo, Watson, current "self-driving" and indeed ChatGPT4 to not meet the criteria, although they do produce very useful output. What's your assessment of these?

Now, how far away are we?

Let's take the arithmetic problem I cited above. I'd bet the OpenAI team is weeks-months away from solving that by generally recognizing all math problems, abstracting them, and passing it off to the Wolfram solve plug-In (like an old subroutine). And that is a level of abstraction that would massively increase usefulness. But, how many of those abstractions must be solved before it can abstract thoughts, sort facts from fiction, reason, and reason about itself sufficiently to set goals?

The generative image systems' problems may be instructive. When asked to show something with a banana, they always show bunches of bananas. This could probably be patch-fixed in weeks with more training to distinguish single/plural bananas. But, I've also seen a "woman on a boat" generating what looks at first like an attractive bikini-clad woman, until we notice that her head & torso are attached the wrong way on the buttocks. This tells us that truly overcoming these problems requires the model to have abstract concepts of all physical objects in the lexicon and understand their relationships, so we don't get legs in impossible tangles or on backwards, etc. That would seem to require an entirely different generation of AI?

I agree that it does not need to be biomicry to be intelligent; that only provides possibly useful hints to useful paths forward.

I also agree that the likely time frame is years-to-decades, not months nor centuries.

>>engineering methods that are dumb relative to how biology sculpts intelligence, and yet still create an AI that outperforms all humans on all intelligence tasks.

Yes, this is definitely possible, especially with scaling. But do you see a path for LLMs alone to get us here with massive scaling up? I'm thinking it needs Large Abstraction Models layered on top of the LLM, sort of like the cortex is layered over and modifies the limbic system, which is layered over the brainstem, etc...?

So um... I just started playing with ChatGPT's new code interpreter today. Have you seen it? What it can do is insane. My mind is blown. Math is no longer an issue for ChatGPT. Possibly what it is doing now is analogous to having human system 1 and system 2. It didn't used to have a deep system 2 until now with strict logical steps. I.e. writing and executing code much like how someone doing math follows logical careful steps.

Maybe it only had an intuitive find-patterns system 1 before.

As for your Large Abstraction Models thought, Look up Vernon Benjamin Mountcastle's work. Neuroscientist that proposed that the whole neocortex operates through a common principle through cortical columns. Any Large Abstraction Model might well just be a distributed epiphenomenon of the network without any single nexus. If that is how we as humans work, wouldn't be surprising if just scaling LLMs up is all we need to do to get it to do anything a human can do online. Bye bye a lot of white-collar jobs if true.

I hope your skepticism is warranted though and just scaling it up is not enough because our institutions are not ready for this kind of change.

Yes,I just read a bit about it and will check it out as soon as I get back to a computer (traveling). The math problem sounds like it should have been solved quickly (something akin to parse the language part, identify it as math w/o having the user say so, & hand it off to something like Wolfram Alpha plugin).

Yes this is indeed moving fast, and at some point if it can perform all the functions well enough , it's pragmatically intelligent.

I first read your sentence as "...out intuitions are not ready for this..." - which is also true, and our institutions -omg, they might as well be fieldmice watching rocket launches for how unready they are... yikes!

> This year it's the horseless carriage.

Explosions in technology do happen. Just because some things were overhyped and failed doesn't mean everything hyped up will fail.

Yes but that never caught on - your premise is flawed. The polluting, antisocial, and climate effects of the car meant is was no more than a fad, popular for a few generations in the early Anthropocene before dying out. Perhaps if teleportation had not been invented in the early 2100s… IANAH.
I’m far more afraid of what Big Tech will do if they get a state sponsored monopoly on the technology over some sci-fi level scenario where AI will kill us all.

If only one group gets access to advanced AI it’s a turnkey solution for totalitarianism. Preventing this should be our primary concern.

I'm more concerned about the economic fallout from AI than even that. I'm having trouble seeing a future (assuming that AI pans out as proponents expect) that doesn't include an awful lot of people falling hard into poverty.
We went from an agrarian society into this white collar service oriented society in the western world in a generation. The ability for bullshit jobs to squander productivity gains for steady 40 hours of butt in seat time can’t be underestimated.
> The ability for bullshit jobs to squander productivity gains for steady 40 hours of butt in seat time can’t be underestimated.

I'm not quite sure what this means. How do "bullshit jobs" squander productivity gains?

Correspondent contends that we’ve failed to spend society’s surplus productivity on an increase of leisure. We’ve rather created makework for vast segments of society who are no longer needed in a productive capacity but can hardly be left to starve.

This is arguably a result of the perverse incentives of growth capitalism. No profit may be made from leisure. Though indeed human psychology - the need to work - seems likely to play a role as well.

I don't even think these jobs are fullfilling. Do you really feel fulfilled by needing to work? Not really. Monkey brain doesn't work like that. You feel the need to survive, yes, but once shelter and food is acquired there is no biological drive to get you to be motivated to stock the shelves at Target. The only thing keeping people in those jobs today is the threat of abject poverty, and perhaps a lack of knowledge or opportunity to do something better to pay for food and shelter. Tellingly, no one who is independently wealthy is finding fulfillment stocking shelves at Target. You have to do that in this society as its more or less illegal to hunt and gather and homestead freely, thanks to property rights being distributed to every square inch of land on earth, and most current world governments have vilified the concept of an altruistic government and society, since that means the elites would have to cede material wealth and power.
For its short length, this may be the worst article I've read on this subject. (Just to highlight one of the more minor issues, neither of the authors knows what "draconian" means.)
Well, isn't this a hatchet piece - leading with Mayan doomsday and then closing with 5G conspiracy theorists... Why not throw in some Bill Gates microchip vaccine references as well? Piece is already half-fnords as it is, might as well commit to the bit.
A lot of this is driven by worldview. For example, if you're like me and have a technology oriented worldview and/or read books like Bostrom's "Superintelligence" or Marshall Brain's "The Second Intelligent Species" then you may believe that AI taking over is the inevitable eventual next stage of evolution.

But I think that many people without quite the same worldview can agree on a more mundane issue: the danger of handing control over most weapons on the battlefield to AI. Not AI that is alive, but it will be superintelligent in terms of integrating information and revising and executing strategy faster than any human could possibly hope.

Netflix's recent "Unknown: Killer Robots" explains how this change is already underway and makes a pretty good case for why AI will be used increasingly and given more and more autonomy in warfare.