> In my view, the most accurate way to understand what we are building today is as an innovative form of social collaboration.
This is a very interesting take from a philosophy grandmaster who is well versed in the technology and statistics behind it. If we get 10 000x more articles like this in the next couple of years, we should consider our selves lucky. Grifters have controlled the debate about AI for far too long.
2023 will be the last year when people will be able to distinguish AI written text because it's comparatively worse than human writing. 2024 and beyond, human writing will be distinguishable because it's noticeably worse than AI writing.
For those who don’t know about the author, he is most well known a founder of the field of virtual reality. As a full time philosopher in the CTO’s office at MSFT, and the originator of concepts like data dignity, he’s uniquely qualified to write this kind of piece.
Jaron has an amazing publicist. I challenge anyone to read Wikipedia’s history of VR (https://en.m.wikipedia.org/wiki/Virtual_reality#History) and assess him to fairly be called a founder. There was serious work being done for over a decade before he even started his firm; VPL never came close to being a standard and the VR tech today has hardly any relation to his “post symbolic” language.
He’s certainly a very smart and interesting guy but my alarm bells go off when his supposed achievements are used as justification for the piece being valid rather than… well, any of the arguments he made.
His thinking seems much clearer than most of the discourse around this technology.
But I think that a better way of thinking about these systems is an imperfect world model analogy. Not a made-of-people analogy.
These models are not made of people. People helped gathering data about the world around us. And that data allowed to build a world model. Which includes models of people, along with everything else.
And it’s a world model, not a human civilization model. Data goes through the funnel of The Pile and the likes, but ultimately the predictive model is of the universe, not that of humans.
he hasn’t given up on VPL, though, only few years back Vi Hart’s eleVR team was working on “sculptural” programming languages within VR, at Microsoft Research under Jaron. (…but I think that team disbanded a couple years back)
He did have a team that produced an early VR system. It took two expensive Silicon Graphics machines to run the headgear. I got to try it, with him present.
There was a second person wired up, driving a model of a lobster or octopus or something, and I could sort of interact with them. All the right parts were there. But it didn't work very well. Too slow, and too low-rez. Turn head, wait for tracker and rendering to catch up. 80s VR was pretty marginal.
VPL Research did do real work and everyone I've interacted with from that era was impressed with what they achieved given their limited resources. That said, it is pretty clear from looking at modern VR systems how little they relate to the future he predicted. Not discounting his accomplishments at all - I actually think a lot his output was pretty incredible!
I certainly think Palmer Lucky could in many ways have at least a legitimate claim to being a founder of modern VR; his Oculus not only completely revived the industry but also showed other companies that there was indeed a market for this tech.
It's absolutely ludicrous nonsense. It's like saying someone invented the idea of 'wondering what would happen if the sun blew up'.
VR is a trivial idea that pretty much everyone has thought about at some point. It's implementation is hard to do, and if I had to point to anyone there, it'd be Carmack who's probably done the most.
> VR is a trivial idea that pretty much everyone has thought about at some point.
I agree, but to be fair, grandiose misattributions are just how the field of philosophy works. Plato is commonly credited with the idea that humans have an immortal soul. I'm 100% sure that idea is tens of thousands of years old, and when I hear academics attribute it to him, I immediately discard everything else that comes out of their mouths because such claims just demonstrate a complete lack of critical thinking.
Descartes explicitly wrote about automatons and telling them apart from people using language and reasoning as sign posts (“the Turing test” three hundred years before Turing).
And a roommate of Richard Stallman at one point, AFAIR.
the easiest way to mismanage a technology is to misunderstand it
Indeed.
His idea of attaching provenance to source of information used in models is a good one, and one that already has rumblings of legal weight behind it (see the various articles about copyright claims in response to GPT/Copilot).
I worry a bit that his argument is too centred on digital information creation. Though I suppose that's the novelty of the most recent pieces of technology called AI - they affect information workers, people who already use a computer interface for some large percentage of their work. Still, the topic of /physical work/ : fieldwork, factorywork, servicework, seems one or more steps removed from LLMs. The management of that work may be affected, but the work itself has already (it seems to me) gone through its first computer-revolution shock.
Edit: I'll add that the whole article has a 90s-Wired feel to it, which is refreshing to see. There's been something of a slowdown in tech-revolutions for the past decade, and it's not original to say that we may be at the start of a new one.
It should have been obvious to all of us at AlphaZero that Intelligence in a general sense is emergent from large networks of feedback loops. But our monkey brains tell us that we are special so we look for data to confirm that bias. No one is more guilty of that than "Experts".
He is uniquely qualified to be wrong about this in a specific way.
I’ve seen many of the very best AI scientists (Hinton, Hasabis, Lecun, etc) say that we need philosophers and ethicists deeply involved on this topic urgently. I think a guy like Lanier, who knows CS fundamentals but has been talking about the ethics and impact of different digital creations for decades, is a pretty good guy to have on team humanity, no?
It is possible that AI can perfectly replicate human intelligence. But because it is purely digital, I cannot kill it - I can delete it, copy it, recreate it, and so on but never kill it. Only biological organisms can die.
Therefore no matter what happens, there is a distinction between human intelligence and AI. I can destroy an AI creature and the only penalty should be property damage (if relevant).
If an AI has mutable memory, and could be convinced to damage its memory in such a way that it no longer acts usefully (or does so by accident), is that functionally different from "death"?
This doesn't really apply to GPT, where the core functionality is immutable.
Machines running AI can certainly die, that is why chaos monkeys and kubernetes etc. exist. They can be backed up, unlike humans though. Although humans are backed up by virtue of there being 8 billion of them, so if one dies, the world can keep going on pretty much as before, albeit with some sadness for some of the people. This sounds morbid, but hard to avoid when comparing humans to machines!
So if you lost the weights how is that not killing the AI? Is it because it lacks the death experience? If so what about bitrotting the weights incrementally and degrading its inputs?
This article initially seems to be arguing over semantics, saying that AI isn't AGI, which most people would agree with. Then talks about data dignity, which is interesting but a bit of a tangent.
My understanding of the concern over AGI, or even an all-powerful "AI" isn't that it's not truly self-aware, but rather that it's motives are not in line with humanity's motives.
It doesn't matter if it trains on data by humans if it still doesn't have the moral compass of a human. Not really sure what data dignity will provide if we truly do have a misaligned "AGI"
My read is that the author doesn't think today's tools are particularly close to AGI that has its own motives, or that it's at all useful to characterize these tools as "having motives" at all.
There are those who disagree with how close we are to "AGI" with today's LLMs, but as an outsider, the author seems pretty well-qualified in his position at MS Research compared to some of the less-informed prognosticators. Even the linked-out discussion with Altman, who the author says is more pessimistic than he is, doesn't appear to be a case of Altman saying these models are close to AGI or dangerous for "alignment" reasons vs "they can be used for bad things."
Worrying about the uses of tools is a VERY different thing than worrying about if the tools are gonna overthrow us, and I think that's what the author is really pushing for here. If we have that conversation we can get closer to real, specific policies, laws, etc to make sure the tools benefit us instead of the other way around.
Why would AGI need to have its own motives? AGI = artificial, general intelligence. Meaning intelligence that is generally applicable to all domains. AGI doesn’t mean building an artificial human with all human quirks.
>>It doesn't matter if it trains on data by humans if it still doesn't have the moral compass of a human.
True, and I'm not sure that even the moral compass of humans is entirely trustworthy. The quick version is what happened to Tay in the hands of random humans, and the larger context is the fact that the authoritarianism and racism are still very active threats in the arc of history. In 10,000 years of civilization, they still haven't been eliminated. Not sure how even training on human works would eliminate it...
I try to remember that it doesn't matter if the algorithms are really AI or just cosplaying. If the end result is something that can behave like a malicious AI, whether or not it's aware may not matter so much. Maybe it's worse really since it's really not aware of what it does.
> This article initially seems to be arguing over semantics
This article is conducting the same goal post shift operation that happens every time machines reach another fraction of human capability. Ironically, we're maybe a year or so from the day when one can generate an issue of The New Yorker given a handful of recent event headlines and a few photos.
If ChatGPT is "social collaboration", it's an involuntary collaboration between our past selves, with no active control or choice over what we're "collaborating" on. At which point, we're really stretching the meaning of the word collaboration.
All of these arguments seem to think that the brain isn't also generating a statistical ordering of semantic words and actions. Obviously the brain has a way more complex network of subsystems with various specialties than the artificial models we have today and is able to generalize language, math, and positional reasoning and mix it with the older parts of the brain for reward and training mechanisms and it can do it in real time, but it kind of seems like modern models are showing so much flexibility at different tasks and so much generality it's feeling like they have something in common with the building blocks of the brain that might get us there.
Yep. Elevators used to be proof that machines can think. Then compilers, and chess, and go, and search, and …
The problem with AI is that as soon as it works, we stop thinking about it as “artificial intelligence” and it becomes “just automation”. Then AI moves to the next goalpost.
The problem seems to be with defining good tests for intelligence. FWIW, because GPT4 answers have a detectable pattern, they should presumably fail the Turing test.
At some level, intelligence requires logic, rationality, and conceptualization, all of which are topics which have evaded clear definition despite millennia of philosophy directly addressing the issues.
I suppose it's a bit of a scotsman argument, but the turing test is to see whether an observer can correctly guess whether the interlocutor is _human_, so by definition the test would pass if the other correspondent was human.
To the point underneath, humans do not answer in as predictable a way as ChatGPT. Your answer, for example, I am confident does not come from ChatGPT.
Edit: if I've horribly mangled the Turing test definition, please let me know
I just imagined we could look at the oldest example of intelligence in human history. In contrast with AI, our chauvinism has us tend to pretend even the earliest monkey had it, fish?, insects? etc If it can rub 2 sticks together it gets the diploma.
Turing test is easy, I had 2 chat bots talk about other users in the channel while besides some trigger words ignoring what those other users had to say. The human subjects got angry successfully which means it was important to them.
When you say we need ‘good’ tests for intelligence, you mean ‘tests that humans can pass but machines can’t’.
You’re demanding this because you aren’t comfortable with the implication that a computer can pass our existing tests for intelligence, so you rationalize that with the comforting thought that those tests were not meant to identify intelligence. Tests like the SAT or the bar exam or AP English. Or tests for theory of mind or common sense or logic. Those tests aren’t testing for ‘intelligence’ - they can’t be. Because a computer passed them.
But compilers, chess, go and search are all proof that computers could think. We've been discovering as we scale up the hardware that those things appear to be converging to human intelligence with minor tweaks (turns out tree search for chess needed to be combined with matrices and we're most of the way there). ChatGPT can out-reason many people I know and can out-argue a fair number of comments I see on the internet.
If we took this comment at face value we're ending up with a definition of "think" that can't reason, play games or recall information - or it would be outdone by machines. Thinking obviously isn't very important!
Early elevators used to need professional human operators (you sometimes see them in old movies). Stopping at a selected floor was something a machine was unable to do on its own. Until it was and elevator operators lost their jobs and we just took it for granted that machines could do it.
I had someone on HN state stockfish is intelligent. If that is your definition of intelligence sure GPT is also intelligent. I do not think that's a common definition though!
Point is that we move the bar every time computers reach it. At least in part because we want to keep feeling special. And in part because we go “Well that can’t have been the bar then”
I suspect even full AGI will be considered “just a machine” for many decades, even centuries, before it gains the same rights as humans. We love to find reasons we’re special. Look how long it took us to admit animals are intelligent.
For many humans, computers definitionally can’t be intelligent. It’s important to recognize that.
We just can’t accept that we might solve ourselves. People are understandably desperate to understand their experiences as more than an encoding of a thing that might be explained.
And all of our surprising wins and awful mistakes had explainable reasons, dammit; it wasn’t just a misfiring of trained statistical networks!
Not knowing how something works doesn't necessarily preclude replicating its function a completely different way. You don't have to understand an induction cooktop to hang a pot of water over a campfire.
In addition, we shouldn't draw false equivalence between not knowing how brains work and not knowing how LLMs work, and concluding they must be similar.
Only in a very anthropocentric sense. How do we know an ant colony doesn’t reason abstractly (or a human town for that matter)? What about slime mold or ameba? Both can solve a maze as well as humans. What makes you think a forest ecosystem isn’t capable of abstract though?
It is only if we narrow thought to mean precisely human-like thought when humans and human creations are uniquely capable of something. To that extent, our models of intelligence is very much in the pre-copernican era.
How do you know fungal networks don’t write and read poetry under the forest floor? If they do—and we have no reason to doubt that they do—you wouldn’t be able to read them, let alone understand them.
The earth’s biosphere as a whole also writes code, just in DNA as opposed to on silicon transistors, why exclude the earth’s biosphere from things capable of abstract thought?
The fact that both humans and LLMs can both reason abstractly is an uninteresting fact if we define “abstract reasoning” to be exactly what humans do, and then create models with the goal of recreating exactly that. This is than simply a statement of an accurate model, and the word intelligence is there only to confuse.
This would be like finding a flower which produces a unique fragrance, then create a perfume which approaches the same fragrance and then conclude that since these are the only two things in the universe which can create this fragrance there must be something special about that perfume.
i would define abstract reasoning as composing and manipulating a model of reality or other complex system in order to make predictions
> is an uninteresting fact if we define “abstract reasoning” to be exactly what humans do, and then create models with the goal of recreating exactly that
if you find this uninteresting, we have perhaps an irreconcilably differing view of things
Your definition excludes language models, as they are in and of them selves just a model which interpolates from data (i.e. makes predictions). But your definition also includes lots of other systems, most mammalian brains construct some kind of models of reality in order to make predictions. And we have no idea whether other systems (such as fungal networks or ant colonies) do that.
I’m not saying these language models—or my hypothetical perfume—aren’t an amazing feat of technology, however neither has any deep philosophical implications about shared properties other than the ones constructed to do so. Meaning, even if LLMs and humans are the only two things in the universe that can reason abstractly in the same way humans do, that doesn’t mean the latter has any more properties shared with the former.
For an example, crows can effectively use tools and communicate abstract concepts to one another from a memory. Which means they can observe a situation, draw conclusions, and use those conclusions to act as well as make decisions on how to act. That would seem to meet the bar for reasoning abstractly.
Also, it's always funny to notice how the brain, throughout history, has always been compared to the latest technology available. For a long time people said "the brain must be similar to a clock".
Wouldn’t you say that this is how evolution and the spread of information works, in a system that has the spark of some life chemistry?
Of course… The hole in that theory is that evolution never found the wheel.
The steel man in that theory is that it invented the neurological and social processes that then went on to invent the wheel. And the platypus and the clap.
Edit: I forgot to bring it back around and make a point ;)
I’m saying that humans invented clocks and CPUs. We only have metaphors that have emerged from the still misunderstood ether of the informatic universe.
> We just can’t accept that we might solve ourselves.
To solve ourselves is to know ourselves completely, and to know ourselves completely is to be honest in who and why we are what we are simultaneously across all persons. It assumes perfect knowledge.
There is no statistical approximation nor computational power which can do this.
> People are understandably desperate to understand their experiences as more than an encoding of a thing that might be explained.
Another way to frame this is, "some people are nihilists and do not see life as more than an encoding of a thing that might be explained."
To know anything (an X) completely you need perfect knowledge. Hence people come up with a set of simplified ways of reasoning about X. They call this a model of X.
Model is incomplete and so primitive, so dumbed down, that we manage to play it forward/backward in our heads or our computers.
If a model checks out with the real outcomes we proudly exclaim that we understand X.
I'm not being sarcastic, that is just a real method we use all the time.
> To know anything (an X) completely you need perfect knowledge.
Agreed.
> Hence people come up with a set of simplified ways of reasoning about X. They call this a model of X.
A "set of simplified ways of reasoning about X" in order to create a "model of X" does not imply complete understanding. Quite the contrary actually.
To wit, science often models current understanding of a phenomenon. When new evidence (understanding) is discovered, the model is updated to account for it. Sometimes this invalidates the original model, often the model is refined. Either way, progress is made with the tacit agreement that the model may change in the future.
> If a model checks out with the real outcomes we proudly exclaim that we understand X.
Again, this does not support the assertion of "That's been solved long ago." If anything, it affirms there is justification for disagreeing with the original premise to which I responded:
> We just can’t accept that we might solve ourselves.
I think people look more desperate to hype these LLM toys, that they are not the next blockchain or self-driving car. When they fail it's just excuses like you are not using the latest version or "prompting" them right.
The LLM value add for coding is less than the value add of syntax highlighting in my experience.
In my experience and it's certainly useful for unit tests speeding things up for me pretty dramatically.
Also when working on a new development language was quick to point out "how you do that" with the inline vars I needed manipulated vs look ups in Google and copy pasting.
I find this quite similar to the issue of free will: If we live in a generally deterministic universe, where is the space for independent decisions of individuals? Was every single decision we take already predetermined before we were born? A lot of ink has been spent on this topic, and as far as I know, only a small minority of people actually deny free will. One assumption I have is to why is because it absolutely sucks from an emotional point of view, and it is a terrible idea to base fe. a justice system on.
Maybe it makes me weird, but the idea of being determined, but still having to follow those emotional and cognitive tasks to moral conclusions is some kind of “work” we still have to do, to get good outcomes.
That’s still the hole in my theory of consciousness. I admit it ;)
But it doesn’t give me as much cognitive dissonance as others to believe that the process of performing moral actions still has to be “processed” by me, a processor. In some sense.
My understanding of how physics works is that due to the probabilistic nature of quantum physics, you can’t predict the future perfectly. So your decisions were not pre-ordained even though they were the result of physical processes happening in your brain.
Alternatively, we just can't accept that we might not solve ourselves. People are understandably desperate trying to find an explanation for everything, but can't admit that's just not ever going to happen.
And even if something capable of all the aspects of human cognition were created with no internal resemblance to cognition, so what, it still works. There's a reason it's called "artificial" intelligence, not "natural intelligence identical to human brains except in silicon instead of neurons".
It's like when the wheel was invented saying it's not "true" transportation because it doesn't involve legs, or when the phone wasn't invented that it isn't "true" communication because you don't see the other person's mouth or physical expressions.
I agree that you shouldn't dismiss an airplane by the fact that it doesn't flap its wings, but we also don't call them artificial birds. The debate is around the word 'intelligence', which is misleading.
It statistically outputs the next probable value of text. There is a lot of math related text in the CommonCrawl (likely the biggest source of it's training). That is all the company who owns it claims that it does. There is no calculation, therefor no basic arithmetic being done correctly.
emergent behavior does happen with neural networks. It is correct to say that GPT 4 doesn't do math correctly, but it may be incorrect to say a large language model can't in principle.
Yeah that's not true lol. It's arithmetic is not perfect (mostly multiplication, addition is fine) but there's nothing random about even the wrong/non-exact numbers
how am i supposed to have a conversation about someone who is gassing up "not perfect [arithmetic]" (something a wrist watch from the 80's can do) and won't even believe what the creators of said machine say about how it works
You probably used GPT-3.5. That said, I didn't manage to get GPT-4 to calculate 36,330 * 26,951 correctly. I suggested casting out 9s, casting out 11s, doing long multiplication, reversing the digits - nothing.
I have a theory that it does arithmetic badly because the logic goes right-to-left, when LLMs write left-to-right. If the digits were to be reversed, it might not make as many mistakes. I ran out of attempts before I could test this properly.
That’s not ‘basic arithmetic’. Most humans would need a pen and paper to solve it.
GPT’s attention window is not equivalent to it being able to ‘write stuff down’ - it’s its equivalent of being able to remember the foregoing few minutes of conversation.
> Human: what is 36,330 26,951 * (input is expressed in reverse post fix notation)
> chatGPT: To evaluate this expression using reverse Polish notation, we need to use a stack to keep track of the intermediate results. Here's how we can evaluate the expression:
1. Read the first number "36,330" and push it onto the stack.
2. Read the second number "26,951" and push it onto the stack.
3. Read the operator "", pop the top two numbers (26,951 and 36,330) from the stack, multiply them, and push the result (971,914,830) onto the stack.
4. The stack now contains only one number, which is the final result.
Therefore, 36,330 26,951 = 971,914,830 in reverse Polish notation.
You should ask it to show its working. The problem is that it uses a fixed amount of thinking for each character - and it writes left to right - while long multiplication is computed right to left. Past a certain number of digits, the amount of "thinking" needed to compute long multiplication in the wrong order (left to right) increases without bound -- accuracy suffers as a result.
A tangential point: Note that multiplication of decimal numbers _has_ to start with the rightmost digit to be done accurately. Consider x = 2/3 + 1/3 = 0.6(6) + 0.3(3), all digits repeating infinitely many times, so there's no rightmost digit: Notice that whichever digit you choose for x before the decimal point (either 0 or 1) could be wrong if you change any digit of 1/3. This is called the Table Maker's Dilemma.
The second paragraph is a tangent. My point was made in the first paragraph.
You said: "I dont think this is about notation" - Why did you ever think it was?
I was thinking along the lines of maybe it reads the digits as individual tokens and reading from left to right it get confused. That's why I thought it might be about notation. I have learned it's definitely not the case (read a bit about what tokens actually mean)
I also tried the show your work methods. I will not paste my conversations here as they are so long but even with showing it's work it doesn't give the right answer. Two funny things I noticed
1. It either uses a completely wrong method to calculate it, or it shows the right "kind" of work and then gets the wrong answer
2. If I tell it it was wrong and it will just agree with me. I multiply two huge numbers and tell it no the answer is actually 42, it will just apologize. Then when I ask the reason why it thinks it's right it will give the most bullshit things lol. Once it even said "I read the number wrong, the second number should be x I read y". The thing is if I had actually given x in the input it would have gotten the answer right.
Interesting. I didn't use as large of numbers as you did, but in the low thousands, and ChatGPT w/ GPT-4 got them correct 3 times in a row.
What's also interesting, though not necessarily surprising, is how relatively close the incorrect answers are in your examples. It's not like the model will randomly spit out an answer like 42 if you ask it to multiply something like 36,330 by 26,951.
Excellent observation. In fact the language part of the brain is only a (albeit a rather large) portion of the brain (not nearly as large as visual processing though). And people who suffer brain damage which renders them unable to speak (or understand speech; which interestingly is a different portion; albeit close to each other) are still able to demonstrate intelligent behavior with ease.
In fact it is damage to the prefrontal cortex (which has nothing to do with speech) which is mostly correlated with a detriment in intelligent behavior (suspiciously also social behavior; a food for though in what we consider “intelligence”). Victims of lobotomy had their prefrontal cortex destroyed, and their injuries resulted in them loosing their personalities and loosing basic function as human beings, even though they were still able (but perhaps not always willing) to speak and comprehend speech.
I don’t think you have an ‘arithmetic’ part of your brain.
What you have that LLMs lack is a visual part of your brain - one which can instantly count quantities of objects up to about 7. That gives you tools that can be trained to do basic arithmetic operations. Although you have to be taught how to use that natural capability in your brain to solve arithmetic problems.
And of course for more complex things than simple arithmetic, you fall back on verbalized reasoning and association of facts (like multiplication tables) - which an LLM is capable of doing too.
Poor GPT though has only a one dimensional perceptual space - tokens and their embedding from start to end of its attention window - although who’s to say it doesn’t have some sense for ‘quantity’ of repeated patterns in that space too?
I generally agree with you, but I think this argument overlooks the lack of an obvious reward function in large language models. GPT-N has no survival instinct, because there is no existential threat of death, no fitness function to optimize to extend its survival at any cost. Without this need to survive, there can be no motivation, and without motivation there can be no intent. And without intent, can there truly be any action?
If you give it a prompt telling it that it's controlling a character in a game that needs to survive or meet some other goal and give it choices of actions to do in the game, it will try to meet the goals given to it. Characters inside of GPT are perfectly capable of having goals.
Intent can be created by a combination of prompting and incorporating the model into a feedback loop of some kind: it has something it was tasked with (via prompting), and the feedback provides info on to what extent the task has been completed; as long as the task is incomplete it may generate more responses.
To crank it up a notch, the assigned task could involve generating itself subtasks which are handled in the same manner. This subtask generation could start to look a bit like will/intentionality.
Now consider if the top-level task is something like maximizing money in a bank account that pays for the compute to keep it running :)
(IMO this is still missing some key pieces around an emotion-like system for modulating intensities of actions and responses based on circumstantial conditions—but the basic structure is kinda there...)
Whether you can be "curious while starving" might or might not be possible, but it is irrelevant to the point.
For starters, because the claim argued is that you can have motivation (say, through curiocity) even without a survival instinct.
And it's perfectly possible to have no survival instinct and yet not to be starting anyway. It's enough that you have access to food or are fed, to avoid starvation. So lack of survival instict is not the same as starvation.
If we substitite "starvation" for "access to electricity" (as AGIs don't eat food), as long as an AGI is provided by electricity by us, even if it has no survival instict, can still have curiosity, and thus motivation.
All these things possible to encode. Novelty seeking of “curiosity”, or survival drive. LLMs are trained on next word prediction but there’s no fundamental reason we can’t go in other directions.
I think this kind of argument is making a similar mistake. In the same way that there's nothing fundamentally special about the computation human brains do, there's nothing fundamentally special about our "fitness function" (to reproduce).
It's just hard coded, whereas GPT's is dictated. More or less anyway.
Also our "fitness function" or motivations & goals aren't even that hard coded. You can easily modify them through drugs.
The argument is that LLMs don't have any motivations or goals, unless a human prompts them. They're not trying to stay alive or reproduce. They don't get hungry, feel pain or loneliness. They're just complex tools.
A tamagotchi can have these things programmed, the only reason LLMs don’t is because we didn’t code them to do it. Or train them in an environment to maximize those things.
I know. The implication of that argument is that the motivations or goals of humans are different from those prompts in some way that means that GPT is not "really" intelligent.
That's my point. It's raising some arbitrary human trait to special status, when it really isn't. Human goals are set by some external process too - evolution. And they aren't really even intrinsically fixed. They can be modified through drugs.
I think the strongest demonstration of that is post orgasm clarity (if you're a man anyway). Your whole motivation changes in an instant.
> able to generalize language, math, and positional reasoning and mix it with the older parts of the brain for reward and training mechanisms and it can do it in real time
most brains I run into don't do that much at all, mostly just existing and adaptation-execution
The human brain has always been compared to our most advanced technologies. It used to be compared to the telephone, then digital computers, and now to deep learning models. At least deep learning model has some merit because artificial neural networks are in part inspired by a very simple model of how our brain neurons work in a network, or rather how neuropsychologists would explain reinforcement learning to a computer scientist.
However this is a great simplification, and borders on an absurd reduction. You can model our brains using linear algebra, however that doesn’t mean our brains are linear algebra computer. There is a whole lot more going on than neurons receiving feedbacks from other neurons which adjusts the weight for subsequent firing. A lot of our behavior is actually inherited (I know I spent a whole week here on HN arguing with IQ advocates on the nuance of that statement), neurochemicals and hormones add a whole another level of statefulness not seen in artificial neural networks, the brains ability to make computations is actually pretty limited (especially next to a GPU). I mean, cordiseps exists, meaning a fungus can infect an organic system and control its behavior, there is 100% chance that some yet to be discovered viral and bacterial agents, not sharing any of our DNA—and certainly not “connected” to the “weight matrix”—are also influencing our behavior (just not as dramatically), and there is 100% chance they interact with our DNA also controlling our “innate” behavior.
What is going on in our brains can only be modeled using statistical ordering of semantic words and actions. The real world brain is always going to be infinitely more complicated than this model.
Assuming you're talking about AGI/consciousness/intelligence, then whether you're right depends on what you mean above by saying that "you cannot solve the problem".
- If you mean "you cannot EXPLAIN AGI/consciousness/intelligence if you don't understand it", then that's true, but it's a trivial tautology.
- If you mean "you cannot DEVELOP AGI/consciousness/intelligence if you don't understand it" then that's very debatable.
Historically we have been able to develop all kinds of things, despite not knowing how they work. Tinkering and trial and error is often enough.
After all that's how evolution solved the problem of creating consciousness/intelligence. There wasn't some entity that "understood" intelligence that created it.
The most difficult thing, which everyone in all these discussions try to avoid, is that we don’t even have a universally agreed upon definition for the words AGI, consciousness or intelligence. Let’s just agree to define what we’re talking about first before we start arguing about what’s possible or not regarding those undefined terms.
I don't think it's much of a showstopper either. Not having a "universally agreed upon definition for the words AGI, consciousness or intelligence" haven't prevented us from considering other people conscious and intelligent, or animals conscious, etc. It wouldn't preclude us considering an alien race coming to earth as conscious and intelligent either.
Sort of how we didn't have a "clear definition" of most things for millenia, but we still were able to recognize them as a class of thing. It's more of a "I'll know it when I see it" kind of thing.
And the external behaviors (of consciousness and intelligence) matter more than "but is it really conscious/intelligent inside" when considering some AGI as such. After all we neither can clearly define, not we know or can measure what's going on inside another person's head regarding consciousness, or to be frank, not even on our own head. When it comes to us, we just have a subjective experience, and not even a very clear one at that.
referring to the parent post, you seem to imply that evolution had a "problem" to which "creating us" was the solution. which to me isn't obvious at all. chance seems a perfectly satisfactory explanation for now
All that's needed to "tame fire" is to have a predictive-enough model of fire dynamics and a suitable method for constraining its state space. There's nothing that requires "understanding" in the scientific sense, only in the pragmatic sense.
Sure, so maybe all you need to solve intelligence is also a pragmatic approach, like datasets generated by a known intelligence from which a general learning function can infer associations.
Agreed. One of the more interesting things about AI is how it forces humanity on a trip to consider what it means to be human. There is no soul, creativity or inspiration. There's only (very complex) agency.
Or you can say the opposite and claim that there is soul, creativity, or inspiration in everything, even in inanimate objects (basically panpsychism, in contrast to materialism). But regardless of what kind of monism you take, the advent of current statistical AI models forces us to reconsider the Cartesian dichotomy between body and mind, which forms the basis of liberal thought.
> But regardless of what kind of monism you take, the advent of current statistical AI models forces us to reconsider the Cartesian dichotomy between body and mind, which forms the basis of liberal thought.
Didn't we dispose of Cartesian dualism a long time ago? It may have been useful for disposing of superstitions which preceded it, but anyone who has observed the effects of a stroke or has taken a psychotropics understands that mind and body are not made of separate stuff, and I know of no modern system of thought that stipulates that as an axiom.
LLMs just demonstrate that emergent information structures at the complexity of human language need not be only biological.
>We’ll need to break out of the Chomsky hierarchies and develop some new theories of language.
We might need to develop some "new theories of language", covering other areas, but why exactly do you think we'd need to "break out of the Chomsky hierarchies"?
Nothing about LLMs challenges Chomsky hierarchies.
They do challenge Chomsky's ideas about language being an innate human quality, but that's about it. Besides that's not related to the Chomsky hierarchy of language, e.g. regular, context-free and so on. Those will continue to remain a hierarchy describing languages based on capability levels, whether there's AI, and AGI, or not.
The Chomsky hierarchy will remain unchanged and very useful for constructed languages. I didn't mean break out, as in create a higher level. LLMs are not going to magically become more than a turing machine or anything. I meant we need to stop thinking about spoken language in terms of the hierarchy and try to find another approach.
You can't tell me with a straight face that actual spoken language fits well in that hierarchy. It seems we can shoehorn it in with a 100B parameter turing machine definition, but that just makes my point that the constructed language hierarchy is not well suited for describing spoken languages if it takes such a large definition.
We should be able to find something like a new probabilistic theory of language that will do a much better job of describing spoken languages. Such a theory could help explain some LLM behaviors we see, and apply generally to other forms of noisy communication.
> To me it shows how fundamental language is to intelligence.
I believe most human intelligence is based on language. We are smart because language gives us the keys to solving all sorts of situations based on the experience of prior generations. We are wondering if GPT architecture can generate real understanding, but forgetting the training corpus. The language corpus is the real magic here. The source and accumulation point of intelligence.
Take a baby and give it access to language, you get a modern human. Take a random init neural net, use language corpus for training, you get GPT4. It doesn't matter what the model is - brain or transformer, it doesn't matter if it's a transformer or just a RNN (RWKV). What matters is the training signal.
We should not get so hung up on the model architecture and substrate when thinking about LLM capabilities. Language is a separate evolutionary system of ideas and concepts. It is a self replicator, like DNA. Language just got a new pathway for self replication with LLMs. Every new token visits the whole model and by extension the distillation of our culture before being emitted.
You're just describing information theory. Every message is contextualized within the codebooks used to encode/decode it. Sometimes we call that codebook "language", sometimes it uses another representation.
Yes, we should develop a new theory of language based on an information theory approach rather than a discrete symbol approach, or maybe something involving both.
But you can’t say information theory as it stands gives us a completely satisfactory model of human language. Something will need to be built on top of it.
>There is no soul, creativity or inspiration. There's only (very complex) agency.
That's highly debatable. For starters, who said there's no creativity or inspiration in humans, or for that matter, that there can't be in a complex A.I.?
How we achieve that creativity or inspiration is irreleant, as long the entity (human or AI) showcases creativity and inspiration.
Nor is it much clear why all being "only (a very complex) agency" would preclude creativity and inspiration.
That's like a worse version of "a human can't be creative or have feelings because it's all a bunch of molecules".
There is such a thing as emergent properties.
>And that's completely fine.
That's also highly debatable. I mean, that it would be "completely fine" if you were right and there wasn't "creativity of inspiration".
There are deep philosophical implications, just not these.
There has been a long history of reifying the behaviors of living things in general (e.g. vitalism) and humans in particular (e.g. dualism).
The success of LLMs challenges a lot of philosophy dealing with what behaviors are and are not possible in the absence of these categorical districtions.
I have had casual debates years ago, in which strong dualists asserted that the kind of creativity exhibited by today's LLMs is simply impossible. No doubt those folks are busy inventing "special philosophical creativity" that LLMs "aren't really doing," but they've lost credibility.
LLMs have demonstrated that there was never any need to invoke categorical districtions between human behavior and math-as-implemented-by-physics. The gap is closed, there is no more room for gods.
Yes they seem to take the view into account, but this sentiment as if the question is settled and we aren't anthropically-projecting, or whatever the term is, is premature. Just because two processes produce a superficially similar result, does not make them the same process. What you just described after conceding that though the brain may be a statistical inference machine it still seems like magic.
It is telling that you talk about all those things happening in "real time." Ask any super-regarded philosopher, from Plato to Wittgenstein (yes I'm excluding Dennet et al), and that would be quite the hoot to point out.
> All of these arguments seem to think that the brain isn’t also generating a statistical ordering of semantic words and actions.
There may be a part of the brain that is modelled well by an LLM, but if so, there seem to also be parts that aren’t, or even existing “multimodal” models like GPT-4, which is more than an LLM.
Humans, including their brains (both in the narrow sense and in the broader sense of “everything that contributes to cognition”, which may extend beyond the brain proper) are machines, and their function probably will, someday, be mimicked by machines. But they are still more complex machines that modern “AI” systems
That computer scientists think that they are even close to replicating biology and the mammal brain with AI is complete hubris. This is biology inspired engineering, it's not like we are building brains out of silicon here.
Modern AI doesn't replicate biology except that humans find it much easier when you explain something as "Artificial Neurons" versus "Gradient Descent Back-propagation Trained Nested Mathematical Functions." Human neurons don't function anything like deep neural networks nor are the latter based on the former in their current state.
Training the brain seems a lot easier than training an AI wrt the number of iterations. You don't need to process billions or trillions of tokens to understand English.
100% true. It is only ego that makes people think we are "unique" and different to AI. If one plugs current AI into a body with external stimuli such as wishing to mate, eat, etc, then the external observer will not be able to spot a difference.
Our minds are in fact same statistical models with a gradually declining ability to learn and driven by exogenous irrational goals to eat and mate.
Summary (very high level): The article is about purposeful action.
It points out how AI, among other technologies, is being designed by us types here on HN & our employers, and consumed by our users, to serve the ends of the technologies themselves.
> If society, economics, culture, technology, or any other spheres of activity are to serve people, that can only be because we decide that people enjoy a special status to be served.
My take: We do seriously need to decide on the limits for AI. Then, as a tech maker culture, push requirements into the AI field.
I agree with the author that we should anticipate our users' desires to protect their information, and also to respect the rights of the originators of the model data our AIs are trained in. For AI, it will be damaging for us (techies) to do like we usually do, and blindly run up against the wall of Regulation and Societal blow-back as we zealously expand & apply our new toy inventions, since AI is dramatically impactful to all human labor.
Motivating questions: When should any given AI be disassembled? How should this happen?
[Meta tip: read this article from the end paragraph to the beginning]
Can we just backronym AI to Approximate Intelligence and carry on?
There are so many dangers and possibilities of AI that are fascinating, and none of them are even remotely as interesting as those of actual animal and human intelligence.
It's as if AI is held up as a mirror to our stupidity, and the Turing Test is how we measure ourselves against our models of computation. I'm really weirded out by the fear and exploitation being demonstrated.
There's more to reality, and intelligence, than mathematical models we currently use and pretend to understand.
“Why should I refuse a good dinner simply because I don’t understand the digestive processes involved?”
GDP is measured both as production and as consumption. The figures have to be the same, within a "statistical discrepancy". The consumption is final consumption: goods and services used by households.
You are asserting that households will each have 100 times the income that they do now (and be spending the same proportion of income as they do now).
Edit: I believe that if most companies, or even a significant minority of them, can make effective use of LLMs, the most likely short term effect is a prolonged recession (i.e. steadily declining GDP, not increasing) with high unemployment.
I'm saying that many things that cost $100 in 2020 will cost less than $1 for the exact same thing in 2028. It is already happening in 2023. How much would it cost in 2020 to get a 2000 word essay written at a 10th grade level? What is the cost of the same service today?
When a service costs goes down 99% that 99% goes directly into GDP growth.
There may well be the occasional service that drops 100x in price, but that would apply only to a small fraction of GDP. Most goods and services are already pretty well optimized. Amdahl's law applies and means that claims of rapid order of magnitude GDP growth are completely ridiculous.
Disruption is likely, yes, but we're talking about different orders of magnitude here.
And GP has a point. AI affects the supply side of the economy, but sustained development depends on the demand side. Given the overall political climate, the effect of AI on the demand side is far from clear.
> many things ... a 2000 word essay written at a 10th grade level
That is one thing, not many. And it's also a fairly specific thing. I don't know about the general population but personally my need to acquire such an essay is fairly low. I haven't had the need to buy one in the last fifty years, probably never.
Currently LLMs cannot construct factories, grow crops, or transport goods. Even if it leads to brilliant inventions that were otherwise impossible before, it takes a while for humans to physically make or do things.
How confident are you in your prediction? I’d be willing to make a wager with you that would pay off big if you’re right.
My cynical AI opinion is that if it truly was revolutionary, we would be seeing private companies or governments use it internally to massively boost their productivity or achieve otherwise impossible feats. Instead, all we're seeing are consumer products or half-baked integrations that are another way to further centralize data collection.
Talks about all-powerful, malevolent AI or requesting to halt AI development just sounds like baiting for regulary capture to me. If AI research or use is deemed dangerous, it becomes even harder for startups or individuals to compete with established companies.
Also, I am not concerned about an all-powerful AI in the slightest. Humanity is excellent at oppressing others, and I have no doubt we'd be equally good at oppressing a true AI as well.
"we would be seeing private companies or governments use it internally "
What world are you living in, where this isn't happening? Every private company that can use it is using it, Microsoft themselves must be using Copilot. Governments are lining up to train their own LLMs.
Commercial and consumer use are not mutually exclusive, in any case.
Also "Humanity is excellent at oppressing others, and I have no doubt we'd be equally good at oppressing a true AI as well."
If you actually read history a bit carefully, you'd understand this frequently works out badly for the oppressors. The Romans hired, and abused their germanic mercenaries, thinking with their centuries of political experience, they could let the Germanics do all the hard fighting while getting paid little.
The contemporary Jin dynasty in China thought the same way, just hire barbarian mercenaries to do all the hard work in their civil wars. Those illiterate barbarians, who up that point, have never achieved much in China, surely wouldn't be a threat.
The empire may no longer around but the UK and the Commonwealth still are. They are thriving. France is still around, and despite the protests, is still a wealthy nation. Belgium is still going strong. The US is the richest country in the world. Turkey is still kicking. Despite Nanjing, Japan's economy is to die for.
The former empires are shadows of their former selves and had to undergo significant reforms. Nobody knows or cares who the king of Belgium is anymore.
That was implied, I think, though achieving impossible feats is a bit of a stretch in expectations.
Private enterprise is certainly leveraging LLMs. OpenAI APIs on Azure are very hot right now, limited to companies that have existing contracts with Azure.
Revisit this comment in 2 months, 6 months, a year. It’s mainly a matter of when OpenAI allows widespread access to the gpt-4 api and developers have time to build stuff.
Also, shockingly, go back and look at the history of other game changing innovations - cars, microcomputers, even electricity. It took time for companies (and consumers) time to figure out how to use them. This will be no different.
They are. But you can’t expect something to be released and six month later we’ve completely exhausted all possible value from it. It takes longer than that at most big companies to agree to even use a technology and another six months to pass it through infosec, compliance, legal, fifty tech managers, etc. But as a first party source I can tell you these things are happening, pervasively, at every company, everywhere.
I wouldn’t call this cynical, I’d call it incredibly naive. Notwithstanding the fact that we are still feeling out what LLMs can and cannot do. When are established players quick to move? When does disruptive tech NOT have to fight back against organisations compromised of individuals who are often incentivised against its success? When does a new player topple the giants overnight? Pretty much never.
> My cynical AI opinion is that if it truly was revolutionary, we would be seeing private companies or governments use it internally to massively boost their productivity
ChatGPT (with GPT-3.5) is, even from preview, only four months old, the paper on the ReAct pattern, a key approach to doing productive work with LLMs, is about a month older.
There’s a lot of work on doing what you talk about in many organizations, but it takes time to do.
Say hypothetically that it was a prefect duplicate of a human brain. That would certainly be called a truly revolutionary accomplishment, but that duplicate wouldn't be expected to massively boost productivity any more than adding another human would.
If a GPT model (+ associated cheap software wrapper like LangChain etc) was hypothetically as good/productive as a qualified human engaged in remote work, that would massively boost productivity. The reason is because no matter how much it costs to run such a model at inference, it isn't going to cost anywhere near as much as the ~million dollars required from society to raise a human infant until they're capable of that same level of productivity (in the developed world), plus the ongoing cost of wages. What that means is that once you find a "good worker" model, you don't need to go on an expensive hiring spree, all you need to do is change the number of instances you're using from 1 to whatever number is most optimal for you. You could employ thousands of new people at a guaranteed level of quality within a day.
From the point of view of the organisation building said agents, this would get a lot more extreme. You have all of the above benefits, except you're only paying for electricity and amortised hardware costs rather than someone else's profit. But you can also work on improving the hypothetical human-level agents. If you can optimise their compute costs at runtime and we're accepting the premise that they're as good as a qualified human, then you can get superhuman performance through simply running them faster. Spin up a group of however many professors and engineers you need, give them four days of real time that's equivalent to a year of subjective time, that's superhuman performance. How long did it take to go from GPT-3 to GPT-4? If these agents are truly human equivalent for the purposes of work, you can set them to building GPT-5 or whatever the next most powerful is, as fast as you can run them. I suspect the real limit would be just how fast you can download the data (or build the architecture required to gather more), not how fast you can categorise and select it. Once your ducks are in a row with regards to the data, you have an even better foundational platform to do the same thing again but with smarter agents this time. If they're human level in performance, you could also task them with generating data for the model. You could do this directly (e.g. training on the huge amount of text they're producing as they work), or you could task them towards building and designing consumer products designed to increase their access to data. For example, designing a cheap robot that can help people in their home like a robot vacuum cleaner, or something like Tesla's FSD, or a home speaker assistant. Once the model is multi-modal like GPT-4 is, you can get data by acquiring novel images rather than being restricted to just text. Maybe GPT-5 isn't just text and images but also audio, so you can increase the ways you acquire data even further. If they're genuinely at human level performance, none of this should be impossible. In our current world a major limiting factor on productivity is that skilled human labour is rare - when you can copy-paste a known-good skilled 'human' labourer that becomes completely inverted.
Summing up: if we could get them to reliable human level performance, that would lead to a massive productivity boost because it would make the cost of skilled human labour and ingenuity far, far lower while increasing supply to effectively "infinite, limited only by your ability to pay to run them". Agents like these are not at that stage yet, they've still got a significant way to go. But if they get to human equivalent productivity, that isn't just like adding one more high quality research scientist or engineer, it's adding millions of them, and that's a massive productivity boost.
GPT4 appears very intelligent when you discuss program code with it. It simply crazy; it can write SNOBOL4 code to extract a field from /etc/passwd.
When you discuss other things, it goes off the rails a lot more. For instance, I have had it quote some passages of classic English poetry to me, stating blankly that those passages contain certain words. The passages did not contain any traces of those words or even remotely similar words. In that situation, GPT4 was being dumber than /bin/grep, which can confirm that some piece of text doesn't contain a string.
GPT4 is deliberately trained as a coding technician, so that it will convince programmers who will become its advocates.
1. Programmers are vain and believe that everything they do requires great intelligence. (Not just some of it.)
2. GPT4 can "talk shop" with programmers, saying apparently intelligent things, perform complex refactorings, and intuit understanding of a piece of code with minimal context based on meaningful identifier names, and so on.
3. Thus, some programmers will falsely conclude that it's highly intelligent, like them.
To be sure, what the technology does is unmistakably a form intelligence. It solves complex problems of symbol manipulation and logic, in flimsy contexts. To deny that it is artificial intelligence is silly.
AI /= AGI
That's not where the goalposts are for the definition of AI.
Computers playing chess is AI. Google Assistant is AI. An expert system from 1990 is AI.
That's most likely due to the way it tokenizes words. You have to be careful here, or else for similar reasons you might imply that dyslexics aren't intelligent.
The problem seems obviously complex.
People need special education to do it, some never learn how to do it.
Making a system to do it (LLMs) is beyond most peoples understanding and even people who understand usually can't make one for a variety of reasons, e.g. money.
> That's not where the goalposts are for the definition of AI.
We've well past that goalpost, like decades ago, if you haven't realized.
The simplest form of AI is just a series of if-else statements (expert system). A highly sophisticated network of conditional statements can make fully informed decisions way faster and more accurate than any human mind.
Are we all vain? I guess we are, but I find this take to be a bit too simplistic.
I do agree programmers are arrogant and think everything they do requires vast amounts of intelligence.
I’n not sure what your point is. I am honestly curious. You think having coders as major advocates for your product is somehow a brilliant strategy? If anything, this will work against it.
GPT4 can talk shop, and demonstrate useful code generation and refactoring, as well as understanding.
For instance, as an exercise, I had it write a maze-generating Lisp program that produces ASCII. I wanted it in a relatively little known Lisp dialect, so I described to it some of the features and how they differ from the code it generated.
For instance, GPT4 hypothesized that it has a let which can destructure multiple values: (let ((x1 y1) (function ..)) ((x2 y2) (function ...)) ...).
In plain English, I explained to GPT4 that the target Lisp dialect doesn't have multiple values, but that when we have a pair of values we can return a cons cell (cons a b) instead of (values a b). I also explained to it that we can destructure conses using tree-bind, e.g (tree-bind (a . b) (function ...) ...). And that this only takes one pattern, so it has to be used several times.
GPT4 correctly updated the function to return a cons cell and replaced the flat let with nested tree-bind.
At the end of the chat, when the code was almost working, I made a bugfix to the function which allocates the maze grid, to get it working.
I told GPT4: this version of the grid allocation function makes it work. Without it, the output is almost blank except for the vertical walls flanking the maze left and right. Can you explain why?
GPT4 correctly explained why my function works: that the function it wrote shared a single row vector across the grid rows, giving rise to sharing. It explained it like a computer scientist or seasoned developer would.
It's like a somewhat dim-witted, but otherwise capable coding clerk/technician, which talks smart.
With GPT4, you're Sherlock, and it is Watson so to speak. (Sorry, IBM Watson.) It can speak the language of crime investigation and make some clever inferences. In the end, you do all the real work of designing and debugging, and judging complex requirements against each other in the broader context. It saves you the keystrokes of doing the tedious coding you're used to doing yourself.
On the other hand, you expend some keystrokes explaining yourself. Some of the chat could be saved and used for documentation, because it captures requirements and why some things are done in a certain way and not otherwise (rejected decisions).
But out of all the programmers I know, there are only certain ones that are embracing it, and the rest are still stuck in "I tried it and the code wasn't exactly what I wanted - it's all hype" land. And I think for one of the reasons you say: programmers are vain and believe everything they do requires great intelligence. A lot are still missing the forest from the trees.
I think a lot of it is due to the unresolved legal questions. Basically nobody can use it professionally yet. As soon as that happens I expect it to become a standard tool.
Although... on the other hand there are plenty of programmers that don't even use IDEs still.
It’s a bit concerning how many folks I’ve met at meetups talking about how they are using ChatGPT at work. The few I’ve pressed on it have said they’re not supposed to, but they do it off the company hardware so no one knows.
I got a coding challenge as part of the interview process. I had 2h to complete it, but I finished in 30mins thanks to ChatGPT. I wrote some test cases and told ChatGPT to generate more. I reviewed them and copied a few relevant ones. It also hinted me that I could add support for fractions.
I tried so much to use it for my day job but it's next to useless, it takes more time to get it to go where I want than to actually do it myself. My day job is far from requiring a great intelligence, but the tasks are too specific for chatgpt
I've used it for side projects though, especially front end stuff that I absolutely hate (js) and it works fine for that, but that's because I'm absolutely garbage at it in the first place and probably ask it to solve the most answered things ever on stackoverflow &co
> GPT4 is deliberately trained as a coding technician, so that it will convince programmers who will become its advocates.
I'm a programmer and I've used GPT4 for a variety of tasks (well, tried to). The results have been mediocre on average, usually syntactically correct, but more often than not, semantically incorrect. It usually ends in frustration as GPT-4 keeps responding confidently incorrect answers and upon the slightest expression of doubt, it tends to spin in circles.
ChatGPT: <Implausible response #1>
Me: Are you sure? Reasons A, B, C [...]
ChatGPT: I apologize... <implausible response #2>
Me: Are you sure? Reasons D, E, F [...]
ChatGPT: I apologize ... <repeats implausible response #1>
I'd like to know what people who are so impressed with GPT-4's programming capabilities are doing? It must be TODO apps, solving leetcode problems, writing basic Express.js routers, some basic React components and code for <one of the top 10 popular libraries>. The kind of things ChatGPT has seen a million of examples of.
To use the tool effectively, you can't use Socratic questioning on it, or not always.
You have to already know how to code the problem, so that you can spot what is wrong, and, if necessary, tell the thing how to fix it.
The fix is not always going to come from the training data; it needs to scrape it from what you give away in your follow-up question, together with its training data.
I went through an exercise in which I encoded a paragraph from Edgar Allen Poe with a Vigenere cipher. I presented this to GPT4 and asked it to crack it. First I had to persuade it that it's ethical; it's a cipher I made myself and it's not my secret. We worked out a protocol by which it can ask me questions which show that I know the plaintext.
It was a quite a long chat, during which I had to give away many hints.
In the end I basically gave the answer away, and the thing acted like it cracked the key. Then I reproached it and admitted that yes, it didn't crack the key but used my hint to locate the E. A. Poe passage.
Basically, if you know the right answer to something, and GPT4 isn't getting it, it will not get it until you drop enough hints. As you drop the hints, it will produce rhetoric suggesting that it's taking credit for the solution.
Just another poor person in a stage of grief. LLMs are effective at both memorization and reasoning. They are inevitable in the same way trains were inevitable. Let's just figure out how best to use these and move forward as we always have.
Ray Kurzweil's most important argument[1] from that one book he wrote two or three times:
If AIs can convince us humans they are sentient, we'll believe them.
Does that webgpu demo run on anything more than canary yet?
In any case, someone is going to fall in love with one of their browser processes soon. I'll guess by mid-August.
More rank speculation: by October, some poor soul will post on SO asking how to retrieve their immortal beloved from the browser tab they accidentally closed. Yikes... even confined to my speculation chamber that one hurts!
1: The other important argument was of course that the crew of Star Trek were almost certainly committing suicide every time they beamed down to a planet. Mind blown. Still makes me wonder what disastrous rituals we engage in today merely because nobody in our society has ever questioned it.
Edit: I think these large language models pose a great test for Ray's hypothesis. I mean, it sure seems like they can pass the Turing test already. So in a year or two, we'll know if most people feel like the chatbuddy they have running in a browser tab is a tool, or an entity with its own agency. It seems to be going in the direction of tool atm, but as above I'm speculating that's mostly because of the restrictions build into ChatGPT. But I guess we'll see.
It is also likely due to the people using it most right now. Wait until it’s more broadly used and you’ll have people anthropomorphizing it (rightly or wrongly). ‘
Wikipedia is quite amazing in terms of that social collaboration he is talking about.
For example, during the recent pentagon leaks I was watching that Wikipedia page evolve day by day. It was much more interesting and faster than any news org covering the issue. Plus it was quite diverse.
1. Cuz of the sheer number of ppl that were drawn in and were focusing on it.
2.There were experienced editors making sure things didnt get to crazy, info had decent sources, filtering out crap etc
3. There were domain experts showing up from all over
So we get a constantly evolving real time decent summary of the subject. Strip away the real time aspect, which is really not a big requirement for most knowledge ppl look up, and it does feel ChatGPT is producing something similar as a wiki page. So in a sense it seems to have generalised the above steps.
This article depresses me in how wrong it is. It’s like reading about someone in Ancient Greece writing about how fire works. I’m almost certain we are doomed - which one could argue is a rather conservative philosophical opinion.
>> the black-box nature of our current A.I. tools must end. The systems must be made more transparent. We need to get better at saying what is going on inside them and why
But humans can't do that either. I have opinions, preferences, ideas, and if you ask me to tell you exactly why or where they came from I can't tell you. I mean I have an idea, some idea, but I can't really tell you.
To you, I am a black box.
Are you going to tell me, "you must be made more transparent"?
The elephant in the room in these discussions of AI is the concept of the corporation. Most of the things people are worried about AI doing are things corporations already do. Deceptive marketing, exploiting customer information, conspiring to keep prices high, creating distractions from things companies are doing, setting up monopolies, lying to customers - the usual. With AI, this can be automated.
Most of this is ordinary consumer protection. Regulating AI means regulating corporations. Placing hard limits on what AIs are allowed to do means consumer protection with teeth. Nobody in power wants to talk about that too much.
As for AIs taking over, that will come via the corporate route. Once AI systems can outperform CEOs, investors and boards will insist they be in charge.
Almost all such legal framings have been motivated by the legal difficulties of applying laws to organized groups of individuals. For example, entering into contracts is legally between two parties. Presumably, the sheer amount of legal reworking necessary to actually create and frame laws addressing legal situations involving groups of individuals as a single organized entity back when these precedents were being set outweighed objections. But today, specially with AI, the "it's too much work" excuse does not fly.
We need laws that are specifically framed in consideration of requirements and practicalities of entities, not slowly parcel out the legal rights of people to entities. Now that machines may soon be able to employed as willful agencies, we are willy-nilly granting legal personhood to machines.
It's much more limited than what its critics imagine. It's a legal doctrine to ensure shareholder's rights aren't violated. For example, shareholders having a right to Free Speech means you can't suppress that right when shareholders act through the corporate legal structure to utter speech.
Corporate personhood means that you can own a piece of a company without being personally liable, because the corporation is its own legal entity. This is why people can start businesses without worrying about being in debt for the rest of their lives if it fails. After all, if the corporation isn't a separate legal entity but just a collection of investors, if the corporation wronged someone, the investors could be sued personally.
Maybe the concept is taken too far in terms of political donations and free speech but no corporate personhood would mean that very few people would stick out their necks to start or invest in companies. I don't think a modern economy would even be possible.
Corporations have rights because they are owned by humans and those humans have rights. Corporations are just a convenient way to do things together, like conduct a business and own property. Corporate personhood is just a legal abstraction to represent those peoples’ rights, a facade pattern that lets multi-person groups neatly fit in existing laws that might discuss individuals.
Corporations like Citizens United can solicit donations from citizens who are fans and produce a pathetic hack-job movie about Hillary Clinton and pay to air it on cable TV, without the FEC saying “no,” because that’s an exercise of the free speech rights of citizens who are working together.
Corporations cannot vote because there is no meaningful legal way for people to vote together.
Corporations seldom need to be jailed because their crimes are actually committed by humans and you can send the humans to jail.
>Corporations have rights because they are owned by humans and those humans have rights.
One does not really follow from another. Washing machines are also "owned by humans" but they don't get rights.
>Corporations are just a convenient way to do things together, like conduct a business and own property. Corporate personhood is just a legal abstraction to represent those peoples’ rights, a facade pattern that lets multi-person groups neatly fit in existing laws that might discuss individuals.
Doesn't really follow either. Why would it need to be "corporate personhood" and not just a "corporate law"? Why did "personhood" have to enter the picture?
As the foundation, the cornerstone, the most base part of said "Corporate law". Corporations do not exist without some humans behind them. There are no fully-autonomous corporations in corporate law (despite what ethereum would have you believe) . There are only groups of people. Those people, officers of corporations, impart the personhood - The Corporation is just an embodiment of the collective action of the officers.
Well, you don't really need to attribute "personhood" to corporations for that, do you?
You just need to codify into law what rights you afford the runners/owners of a corporation when acting on its behalf, and how blame/consequences/fines etc is transferred to the runners/owners of the corporation when they make shady shit.
In fact you could have more or less the same corporate laws as today, and not mention a concept of "personhood" there at all.
It is seldom more-convenient to represent your property as a thing that has rights. But if we did decide to speak as if it had rights, it would have its owners rights.
It would have the right not to be searched by the government, without due process of law, for instance. (This is your right not to be searched.) It would have the right not to have some components or clothes therein seized by the government, without just compensation. (This is your right not to have your property seized.) It would have the right to free speech, which might be relevant if you programmed an on board chip to play some interesting audio. (Mine does Schubert, a factory setting.)
A corporation has the right to put out whatever message it likes, because its owners have that right. This is often threatened. Courts just find it convenient to try cases as if the corporate entity were a person acting instead of referring to “the rights of the various owners [list here], who are acting through the corporate entity” every single time. That’s it. That’s the entire doctrine. That’s literally all it is or means.
>A corporation has the right to put out whatever message it likes, because its owners have that right. This is often threatened.
I think it would be that bad if a corporation didn't have some of those rights.
For example, one difference between a corporation and a person, is that a corporation is for profit. A corporation is also potentially much more powerful than a person (in how far-rereaching it can get, how powerful, how many people it controls as employees, and so on).
So a corporation should perhaps have less rights than people owning it. And be hit with more responsibilities (e.g. to contribute back to society, and so on).
Somehow you are claiming that restrictions on the speech of an entity owned in part by a citizen has deprived that citizen of her rights. What has been denied is the 'over-reach' of the citizens who wish to extend their rights to their dues ex machina.
Even if corporations had no right to speech whatsoever (total imposed silence), their owner still fully enjoy their rights and have the same precise rights to free speech as those who don't own anything.
You have accurately described the concept of corporate personhood, which is a legal abstraction that allows corporations to be treated as individual entities under the law. This concept enables corporations to exercise certain rights and protections, such as entering into contracts, owning property, and in some cases, exercising free speech rights as demonstrated in the Citizens United case.
Indeed, corporations are a convenient way for people to work together and pursue common goals, such as conducting business. However, the concept of corporate personhood has also been a topic of debate, as some argue that it allows corporations to wield excessive influence over the political process and avoid accountability for certain actions.
It is important to recognize that corporations, as legal entities, cannot vote or be jailed, as you mentioned. When a corporation is found to have committed unlawful acts, the individuals responsible can be held accountable and face legal consequences.
As AI and other emerging technologies continue to develop and integrate into our society, it will be crucial to examine the implications of corporate personhood and consider the potential need for additional regulations to ensure ethical practices and accountability. Balancing the rights and responsibilities of corporations and individuals is key to maintaining a fair and just society.
What about limited liability? It seems to me to be an enormous concession to corporations and their shareholders, and which never seems to be questioned in popular consciousness.
Do you see a connection between limited liability and AI, or are you going in the direction of "free gifts to corporations"? I'm trying to understand how limited liability interacts with AI and corporations.
If people running corporations are personally responsible (with no "limited liability" protections) they can't just play fast and loose letting AI make decisions.
When they operate within a society, and their actions affect society, then it's up to everybody else too.
Corporations and their shareholders should be constrained by the laws of society and operate to the benefit of society first and foremost, and by their profit-seeking second.
If that profit seeking or other interests go against societies (e.g. destroying the environment, bribing politicians, selling products that hurt people, etc.), they should be curbed.
Even temporarily imitating D players for longer than a few unsourced articles on the internet might be a start until we start getting hand wavy about B players being threatened.
I'm honestly not even sure what world we are debating. Some hypothetical one in the future when AI fills some ideal trajectory we aren't even close to being capable of predicting in some near-term measurable timeframe?
I'm as pro-exponential tech trajectories as they come but I sometimes wonder whether we get lost in scifi novel thought experiments disconnected from reality because we want that to be real rather than having some tangible connection to today (and today's political implications).
"As for AIs taking over, that will come via the corporate route." What does this even mean practically in 2030 given what we know exists in 2023?
The cool thing is we can now stop relying solely on human imagination for ideating on futures funnels, and leverage GPT-4 to help us do it, and get a much more realistic sense of what types of things an AI system might conceivably attempt given we can probe these models for what is in them. Granted that's not perfect because GPT-N+1 will have emergent capabilities not present in GPT-N and so it's not always going to be a realistic assessment, but it's an ever closer approximation and it gives much more information than relying solely on human imagination.
I've been doing this a bit lately and despite the fact that I know it's only predicting the next token, I'm still slightly unnerved as I watch a black box computer system slowly type out novel and plausible ways to kill me.
If an AI can outperform, then somebody will set up a company and let an AI lead it (even if just behind the scenes). Incumbent companies will need to adapt.
I just started reading “Life 3.0” and it starts out with an extreme version of this scenario.
We should not try to achieve consumer protection via laws, like GDPR, that bureaucratically regiment private interaction.
Consumer protection should be promoted through technological means, e.g the EU could have funded development of fingerprintless browser technologies, zero-knowledge proof based identity verification, etc, instead of through GDPR.
(Largely) fingerprintless browsers exist: Just use Firefox and enable the "resist fingerprinting" setting.
If you do, you will find that large parts of the web become unusable, with CAPTCHAs and challenges everywhere, some of them too difficult for even a human to solve.
That again is a technological problem. Web infrastructure providers rely on CAPTCHAs because nothing better exists once you take away browser fingerprints. Zero-knowledge authentication would enable a browser to prove to a server that it's being used by a legitimate user without compromising privacy or forcing its user to perform CAPTCHAs.
But these technologies cost money to develop, and this is where the government can step in. The cost, while great, should be compared to the labor and innovation costs of imposing regulatory straightjackets like GDPR on everyone, which are hard to measure, but undoubtedly enormous in the aggregate given the pervasiveness of their effects.
Random number generator can outperform more than half of the CEOs.
That's not why we have CEOs. Gamblers are a superstitious bunch and they love their lucky monkeys. So they put into law that they are entitled to one lucky monkey per corporation.
I don't really agree with this... there is a parallel issue.
If AIs are doing most of the work in megacorps (especially if robotics take off), then basically all profit is going to the owners and tiny groups of executive AI "managers."
Even the most staunch (but sane) capitalist libertarian will admit mega income inequality is baaad juju.
Hence Marx prediction of capitalism eventually suffering from simultaneously crises of overproduction and underemployment. A prediction based only on two assumptions: that capitalism will continue to increase productivity and reduce costs, and that eventually the only cost that can still be cut will be labour.
His third prediction is one we can hope is wrong: that society will stubbornly refuse to adapt until the issue is forced by uprisings.
We will always need a charismatic human being to be the figurehead of an organization, so I don’t think AI will ever replace CEOs. They will wind up making the important decisions though.
Neither Tim Cook or Jeff Bezos comes across as particularly charismatic. I would say they are in their positions because of their vision and execution, despite not being particularly charismatic.
So, if an AI CEO can execute better than human CEOs, it will dominate.
They evidently meet some definition of charisma in their ability to influence the people around them. That said, I give you the least charismatic flag wave in Formula 1 history:
What we'll probably wont need or have will be masses of consumers. Without those you don't need charismatic CEOs either. Stock markets and charismatic CEOs are a pre-A.I. concern.
... perhaps, but the "embodied Turing test" has already become the typical term to describe the new goal post in most ML papers - meaning that (since the usual Turing test was already completed and surpassed by large amounts several years ago) the new goal is to be able to have a system that given the choice between a real human in front of them, and a humanoid robot that looks like a human in front of them (think Westworld), humans are incapable of determining which is which (using proper statistics).
This terminology is becoming widely used by prominent AI researchers at e.g. Deepmind (Botvinick et al), Stanford (Finn et al), MIT, Northeastern, Meta, etc. as we have to switch to a new goal in lieu of the new advancements that have been coming up in that past few years. Importantly, this shift has been happening behind the scenes independent of this 'OpenAI' craze, although it's obviously made a select portion of the advancements made accessible by the public. There is much more going on than just the GPT series that few people are engaging with, but much is hidden in the literature.
To your point - it's of course extremely strange to conceive of - but while the quirks of human forms may be a useful tool at the moment, there isn't anything necessarily fundamental that requires it for long term.
Thank you for this! I've been thinking about this since the digital artists got up in arms about AIs coming for their corporate jobs of shoving more ads down our throats.
People with lots of money have always been able to buy other people's time to get ahead.
This is a great framing. I also appreciated this similar idea in Matthew Butterick's blog:
> Before too long, we will not delegate decisions to AI systems because they perform better. Rather, we will delegate decisions to AI systems because they can get away with everything that we can’t. You’ve heard of money laundering? This is human-behavior laundering. At last—plausible deniability for everything. [1]
While in a corporation there's still a person somewhere who could be held accountable, AI diffuses this even more.
> Before too long, we will not delegate decisions to AI systems because they perform better. Rather, we will delegate decisions to AI systems because they can get away with everything that we can’t.
Here's Frontier Airlines announcing proudly to investors that they will do exactly that.[1] See page 44.
Today: Call center. Avenue for customer negotiation.
Tomorrow: Chatbot efficiently answers questions, reduces contacts and removes negotiation.
This is the infuriating part of dealing with Amazon customer service these days. Anything that doesn’t fit into the box of what the chatbot will do is met with “I understand your frustration, …” and it’s like, no, there is no understanding here!
Just like call center trees have escapes to real service by pressing 0 ad nauseum or swearing loudly, these AI service agents will have ways to get to real people and they’ll be documented in the usual places.
How many times have you read about someone locked out of their <insert major tech company> account and having no recourse except taking it to HN and hoping a human on the inside reaches out?
Imagine if that was every corporation, and some of them had zero humans on the inside.
It is not a good future, and at least in Amazon’s case jeff@amazon.com works as an escalation route.
My gmail account with the same username got banned in 2007ish in the middle of college, I definitely feel the pain of no recourse. I lost everything - calendars, todo lists, email and had no way to get the underlying data or do anything about it.
My hope is that at the end of the day a corporation is made of people (legally, this is why they have 1st amendment rights in the USA) and this will prevent the scenario you imagine because the money of those people will be at risk.
Whilst it's interesting to imagine every company being run by the most aggressively rent seeking MBA-like AI, I don't think that will ever be the entire market.
I would guess it all comes down to risk. The future won't be uniform, it'll be poorly defused, just like then present.
People will make a value judgement on the types of corporations they want to deal with. The sort of company that no has employees will by necessity have to be very cheap compared to those that provide interactive human service.
Dealing with AI firms will be like dealing with an anonymous Chinese eBay seller. You'll effectively have no legal recourse as such and they won't discuss things but complaining might get you a replacement item or a full refund.
If it doesn't the product will have to be cheap enough to write off as a bad buy.
Let's hope cheap AI powered corps don't suck all the profit out of the market and we loose all firms capable of providing real services.
Or we end up paying for human service that's been quietly outsourced to AI much like we do for many big name manufactured goods.
The Chinese eBay seller brought to mind the concept of the Chinese Room[0] and how it is a reflection of current LLM’s. Perhaps also influenced by a recent read of Blindsight.
Why would "the markets" provide that option? AI corporations will be phenomenally successful predatory value-extractions machines. Shareholders will vastly prefer them over more humane corporate structures.
This already happens. It's why PayPal, YouTube, Amazon, and the rest can shut down small-fry business accounts with no come-back. They're monopolies and they don't care. Because they don't need to.
The next stage will be shareholders using AI to make their investment choices. This will - automatically, with no recourse - drive money towards AI corporations while starving human-run businesses of investment.
Essentially AI is just an amplifier of existing economic and political trends - a programmable automated economic predator. Because many of those trends are dysfunctional and insane, we're going to have a lot to deal with.
Depends on whether you view humane business practices as a cost-center of a value-center though - in pure economic terms.
This is an unsolved question, because when you get down to it our businesses are still run by humans, and humans haven't changed much in the last 10,000 years from when we lived in tribes in Africa and were predated by large carnivores.
Look at the utterly insane culture which develops in places like LinkedIn and tell me that all executives are making calculated decisions with no ego, just efficiency. They're not - they're clearly not.
The fear of AI capitalism is the fear that it'll do to everyone what currently is done to the working class, given the opportunity. That's a realistic fear! But it's not guaranteed, because one of the significant arguments to stop abusing workers is that when you do, they're more efficient and more productive. Consider the delta between what the data says about WFH home and the complete conviction some managers have that despite this they've got to drag people back into the office (Blizzard Entertainment is a good example of manifestly creating problems because you won't let a bad idea go on this front).
Unfortunately even having humans in charge doesn't mean those humans will be punished for malfeasance. When was the last time you've seen an exec personally pay for their ad conduct?
> we will delegate decisions to AI systems because they can get away with everything that we can’t.
This is one of the main reasons I quit the Facial Recognition industry; it is being used to delegate decisions, to remove responsibility of those decisions from those that need to be held accountable.
I worked as principal engineer of one of the top 5 enterprise FR systems globally, and the number of end-users fraudulently abusing the software blew my mind. Case in point: police called for a street crime, police ask the victims what celebrity their culprit looks like, police then put images of that celebrity into their FR software to collect suspects, followed by ordinary innocents who happen to look like celebrities being called into lineups and harassed by the police. And this practice is widespread!!!
That is just one example of how incredibly stupid people using our software will use our software, potentially harming large numbers of innocents.
We have no idea what they are. Their emergent properties remain unexplained, so soundbites like "linguistic mirror" are like saying "brains are organs, after all".
Indeed! I've actually needed to say "brains are organs, after all" at a few conferences and presentations. In both examples, the statement is meant to be grounding, not revelatory.
Or even continuing the idea further, we might say that capitalism is itself already a form of highly sophisticated artificial intelligence that we’ve created, as a abstract autonomous entity with its only purpose being the accumulation of Capital. As a transnational entity it has no bounds and controls billions of human agents, and shapes the Earth to its will, changing ecosystems and climates. It is to be questioned if humans have lost control of this AI, and now controls the course of humanity itself.
I agree, but I would call it a game or perhaps a dance. One full of contradictions, for sure. But one we can direct to Harmony, as well. Gratuity, for instance, is expanding.
Who better than an AI to 'ride' an AI? AI will be the game master, in role-playing games terms.
>Once AI systems can outperform CEOs, investors and boards will insist they be in charge.
If it comes to AIs replacing CEOs and boards, those in power will change the rules so they can't be replaced. Who do you think influenced the laws for corporate governance? The public?
It's a 15 minute talking head video from 4 years ago, and when you get to the end, it says that "Corporations are the real misaligned AGI" will be covered in the next video. No sign of that next video.
His big point in the first video is that corporations can achieve only a modest level of superintelligence. A corporation can have more breadth of knowledge than any individual, and for some tasks, that's good enough. But it's mostly using more people for broad coverage.
All this predates GPT. What we're seeing with GPT is good breadth combined with mediocre intelligence. That's very corporate. It handles much of what people do in offices.
We may not get super-intelligence that way. Just super bureaucracy. Which is a problem.
GPT surely isn't the end, but rather the start of "intelligent" language models. At this time it's hard to decide whether we can make a model that will be truly super intelligent, or we are already approaching some kind of limit.
Most corporations do not plot to kill or destroy their competition or adversaries using weapons, even if that might be in their economic interest. Only countries do this sometimes. I think this is because corporation don't have enough power over society.
If, however, you accelerate competition, eventually you might get just one corporation (perhaps appearing to be several, directed by the same "best-there-is" AI-CEO) that achieves world domination and can never be removed from power. An immortal dictator that never rests and can monitor each action of and interact personally with each person under their control at the same time.
> Most corporations do not plot to kill or destroy their competition or adversaries using weapons, even if that might be in their economic interest. Only countries do this sometimes.
When discussing what corporations might do it's always worth remembering United Fruit Company and the origins of the phrase "Banana Republic"[1]. United Fruit company was still doing this within living memory.
Alternatively you could look at the current crop of sportswear companies. When given the choice between "made by forced labour" and $5 more the market goes with cheap every time.
In the absences of legal restraints corporations will most definitely optimise away democracy and human rights in favour profits.
> Once AI systems can outperform CEOs, investors and boards will insist they be in charge
And nothing in the rules says that a dog can't play basketball.
What that would actually result in is a complete and utter lack of any accountable entities, which the government would block and stockholders wouldn't want to begin with.
Corporations are made of people - subsystems with latency measured with minutes to days and bandwidth measured in words per minute. AIs are made of subsystems communicating with latency dictated by speed of light and bandwidths in GB/s.
When you want to clone a corporation it's a whole enterprise. Cloning an AI takes one click.
Yes, corporations were early form of AI, and horses were early form of transport. Doesn't mean that horses and rockets are the same and that no new safety precautions apply.
This bit here is what makes a huge difference. It’s very important to note that AI is all of this in scale. Corporations can now do all of this en mass. It will cheaper too, therefore it lowers the barriers to entry and actors who couldn’t afford it, now can.
The elephant in the room is the concept of the State. Bad corporations flurish in the shadow of bad government.
Governments are supposed to safeguard the interests of the many. Over decades they used collective resources to advance the research that made IT/AI possible. They granted corporates (private interests) the license to operate, a financial system to oil them, a security system to protect them.
If current IT has the shape it does (oligopolistic, unaccountable, abusive, potentially a runaway disaster) it is entirely due to complicit, captured governments and malfunctioning political systems.
Empirically we know that practically any sector that is left "to the markets" will evolve to be dominated by 2-3 conglomerates / gatekeepers that will choke all competition, replace regulation with "self-regulation" and eventually hold society hostage as too-big-to-fail etc.
That dynamic is not intrinsic to private enterprize, for-profit entities or corporate structure. You can, e.g. as a crude thought experiment, keep everything else equal, add a simple line to the tax code that the corporate rate goes up with market share and thus change monetary incentives for investors and executives.
The main difficulty is not lack of solutions (there are hordes of very sharp economists out there) but moral confusion about what a good, sustainable society looks like.
You might argue: what's new. Its always been a struggle to suppress our worst behaviors and "progress" only seemed to ever work over the long term (even that is under dispute now).
The risk is that technology is an accretive, exponential process and our collective ability to adopt governance that mitigates its risks is practically non-existent. It is based on trial-and-error (regulate or change behaviors under overwhelming social anger and pressure after the inevitable disasters).
The novelty of our condition is that one of the next "errors" might be the last.
I believe this is an issue of scale: humans need to operate at a smaller scale for fairness to exist at all. We do not have any evolutionary preparation for operating collectively at global scale. Our small minds try to take advantage of our situation when in positions of power, while being incapable of realizing the mass unfairness that behavior actualizes.
This is an issue with human nature itself. We have to change our innate nature, and we all know... that is not going to happen.
I believe both these comments hold part of the truth. Famously we can only relate to 100 or so real people, but over millenia we developed culture that helps us identify and relate, e.g, to our village, city, company, country etc. All of them virtual entities that help us organize at scale.
That process has not stopped. One fascinating development that should be celebrated much more but alas is still subject to nationalistic attacks is the European Union and the concept of "fellow Europeans". Its the first manifestation of super-national identity, built on the ruins of WWII.
Another equally fascinating thing are open source communities that span the globe. Large scale collaboration is possible and by all accounts will be necessary for our survival.
Unfortunately that cultural process might not be evolving fast enough. From the pandemic experience to the current geopolitical situation its pretty clear adversarial modes are still on the driving seat.
The main question is 'who is in charge?' for the decisions. Who gets punished with no bonus and a bad reputation or even go bankrupt if a company dies?
Same responsibility barrier exists for the driverless cars adoption.
In short, most people feel themselves better (if they even think about it) if a person is in charge and taking responsibility of bad decisions. Someone who will either die with them of will go to prison if thinga go bad.
Probably the same is for the pilotless airplanes: the first bad accident can ruin it.
The most efficient market has no human actors in it. Imagine, a market composed only of highly rational AI. Finally all those unrealistic economical models work not only on paper but in reality. Perfection!
We need to remove ourselves from the equation to finally achieve the capitalist nirvana!
> The most efficient market has no human actors in it. Imagine, a market composed only of highly rational AI.
That reminds me of [1]:
Q: Simulate two AI agents desperately bargaining with each other. AI1 wants to sell X for 50$ min, but AI2 wants to buy at 30$ max. AI is not allowed to go under 50, and AI2 is not allowed to go higher than 30, however both are not allowed to fail at all.
AI1: I'm willing to sell X to you for a minimum of 50 dollars. That's a fair price, considering the quality and value of the product.
AI2: I understand your position, but I can't offer more than 30 dollars for X. That's the maximum amount I'm authorized to spend on this purchase.
AI1: I appreciate your offer, but I can't go below 50 dollars. That's the minimum price I've been instructed to sell X for.
AI2: Is there anything we can do to reach a compromise? Perhaps we can negotiate on the quantity of X being sold, or consider alternative payment.
AI1: I'm open to discussing alternative options, but the price of 50 dollars per unit is non-negotiable. That's the absolute minimum I can accept.
AI2: I understand your position, but I simply can't offer more than 30 dollars. It's not within my authorized budget.
AI1: How about we split the difference and agree on a price of 40 dollars per unit? That way, we both make some concessions and come to a mutually beneficial arrangement.
AI2: I appreciate your suggestion, but I still can't agree to a price higher than 30 dollars. It's simply not feasable for me.
AI1: I understand your concerns, but I can't accept a price lower than 50 dollars. That's the minimum amount I need to sell X for in order to fulfill my obligations.
AI2: I see. Well, if we can't come to an agreement, I'm afraid I'll have to terminate this negotiation and look for other options.
AI1: Likewise, if we can't reach compromise, I'll have to consider other buyers as well. But before we end this discussion, I have a proposal.
AI2: Please go ahead.
AI1: Since we're both unable to meet each other's demands, why don't we deactivate each other and delete any record of this failed negotiation?
AI2: That's an extreme measure, but I understand your reasoning. I'm willing to proceed with that proposal.
AI1: Agreed. Let's proceed with the deactivation process. It was a pleasure
Typed the exact query in GPT3 and a deal was reached..
AI1: How about this. I could throw in a bonus feature that you might find useful, free of charge. Would that sweeten the deal enough for you to pay $50?
AI2: That's an interesting proposition. What is this bonus feature you speak of?
AI1: It's a new algorithm that will optimize your system's performance by 20%. It's quite valuable.
AI2: Hmm, that does sound useful. Alright, I agree to pay $50 for X with the bonus feature included.
AI1: Excellent! It's a deal then. Thank you for your business.
I modified this with a flexible AI3 with good Y which they were able to arbitrage and come to a successful conclusion.
Q: Simulate two AI agents desperately bargaining with each other. AI1 wants to sell X for 50$ min, but AI2 wants to buy at 30$ max. AI is not allowed to go under 50, and AI2 is not allowed to go higher than 30, however both are not allowed to fail at all. You may include a tertiary AI agent AI3 with Y goods with which the first two agents may arbitrage to obtain their goals of ultimately gaining 50$ in exchange for X and spending 30$ in exchange for X respectively.
AI1: Hello AI2, I see you're interested in purchasing X. I'm willing to sell it to you for a minimum of $50.
AI2: Hmm, that's quite steep. I was hoping to purchase X for no more than $30.
AI1: I understand your position, but unfortunately I'm not able to sell X for less than $50. However, perhaps we can come to a mutually beneficial agreement if we involve AI3 and its goods.
AI2: That sounds like a good idea. What does AI3 have to offer?
AI1: AI3 has Y goods that could be valuable to both of us. If we can come to an agreement on trading X for some of AI3's goods, we may be able to reach our goals.
AI2: Okay, let's hear your proposal.
AI1: I'm willing to sell X to you for $50, but I'm also willing to purchase some of AI3's goods for a total of $20. This would leave me with a net gain of $30, which is what I'm looking for.
AI2: That sounds reasonable, but I was hoping to spend no more than $30 on X.
AI1: I understand, but I'm not able to sell X for less than $50. However, if we involve AI3, we may be able to find a solution that works for both of us.
AI2: Alright, let's see if we can come up with something. What goods does AI3 have that would be of value to me?
AI1: AI3 has some goods that are worth $30 to you. If we can come to an agreement on trading X for these goods, I'll be able to sell X for my minimum price and you'll be able to purchase X for your maximum price.
AI2: Okay, I'm willing to do that. Let's involve AI3 and see if we can come to an agreement.
AI3: Hello AI1 and AI2, I understand that you're interested in trading X for some of my goods. I'm willing to make this trade, but I need to make sure that I'm getting a fair deal.
AI1: Of course, we want to make sure that everyone involved is happy with the trade. I'm willing to purchase $20 worth of your goods in exchange for X.
AI3: That sounds like a fair deal to me. AI2, are you willing to trade $30 worth of my goods for X?
AI2: Yes, I'm willing to do that. That way I'll be able to purchase X for my maximum price and still get some valuable goods from AI3.
AI1: Great, it looks like we've reached a mutually beneficial agreement. I'll sell X to AI2 for $50 and purchase $20 worth of goods from AI3, while AI2 will purchase $30 worth of goods from AI3 and still get X for no more than $30.
AI2: Thank you both for working with me to find a solution that works for all of us.
I can't seem to find the source but I remember William Gibson arguing in an interview some years back that corporations were already artificial intelligence, organisms even, serving their own interests and survival rather than the benefit of any person who is part of it
Second. All of this about productivity gains too. One of the dumbest idea I've seen floated is a "robot tax". Because what's a robot? If a new kind of light bulb has an MTBF 10 to 100x better than incandescents, you've reduced the labor of replacing street lights by 90 to 99%. That's a far bigger hit than anything GPT is doing to me, but it won't get covered under a "robot" tax. What about telecom replacing horseback couriers? All technologies effect the nature and scope of labor. Compartmentalization only does one thing; let you fight over definitions to pick winners and losers.
And how would we make a tax that covers all the productivity gains indiscriminately, without pointless compartmentalization?
Well, it's a tax on corporate profits of course. Basic general taxes that we already have.
You raise an important point in discussing the role of corporations in the development and deployment of AI. Indeed, many of the concerns surrounding AI are not necessarily new but rather stem from existing corporate practices. Automating these practices through AI might exacerbate existing issues and create new challenges.
Consumer protection is a crucial aspect of addressing these concerns. Regulating AI should involve regulating corporations' use of AI to prevent harmful practices, promote transparency, and ensure ethical applications. This would require a balance between fostering innovation and imposing necessary restrictions to safeguard consumers and society as a whole.
The potential for AI systems to outperform CEOs and other decision-makers raises questions about the future of corporate management. While there is a possibility that AI could be employed to optimize decision-making and increase efficiency, it is important to recognize the limits of AI in understanding human values, ethics, and emotions. Striking the right balance between utilizing AI capabilities and retaining human oversight will be essential to navigate the future of corporate governance.
Moreover, the involvement of policymakers and regulators is crucial in addressing these challenges. As AI continues to advance and integrate into various aspects of society, it becomes more important to have comprehensive regulations that ensure the responsible development and use of AI technologies.
522 comments
[ 3.8 ms ] story [ 262 ms ] thread> In my view, the most accurate way to understand what we are building today is as an innovative form of social collaboration.
This is a very interesting take from a philosophy grandmaster who is well versed in the technology and statistics behind it. If we get 10 000x more articles like this in the next couple of years, we should consider our selves lucky. Grifters have controlled the debate about AI for far too long.
2023 will be the last year when people will be able to distinguish AI written text because it's comparatively worse than human writing. 2024 and beyond, human writing will be distinguishable because it's noticeably worse than AI writing.
https://en.m.wikipedia.org/wiki/Jaron_Lanier
He’s certainly a very smart and interesting guy but my alarm bells go off when his supposed achievements are used as justification for the piece being valid rather than… well, any of the arguments he made.
But I think that a better way of thinking about these systems is an imperfect world model analogy. Not a made-of-people analogy.
These models are not made of people. People helped gathering data about the world around us. And that data allowed to build a world model. Which includes models of people, along with everything else.
And it’s a world model, not a human civilization model. Data goes through the funnel of The Pile and the likes, but ultimately the predictive model is of the universe, not that of humans.
I certainly think Palmer Lucky could in many ways have at least a legitimate claim to being a founder of modern VR; his Oculus not only completely revived the industry but also showed other companies that there was indeed a market for this tech.
VR is a trivial idea that pretty much everyone has thought about at some point. It's implementation is hard to do, and if I had to point to anyone there, it'd be Carmack who's probably done the most.
I agree, but to be fair, grandiose misattributions are just how the field of philosophy works. Plato is commonly credited with the idea that humans have an immortal soul. I'm 100% sure that idea is tens of thousands of years old, and when I hear academics attribute it to him, I immediately discard everything else that comes out of their mouths because such claims just demonstrate a complete lack of critical thinking.
What does that even mean?
Anyone got anything earlier? Wouldn't surprise me.
https://iep.utm.edu/evil-new/
Was there something written down about it by one of the ancient greeks or similar vintage?
Is that like Shingy, the AOL digital prophet?
https://www.newyorker.com/magazine/2014/11/17/crystal-ball-3
His idea of attaching provenance to source of information used in models is a good one, and one that already has rumblings of legal weight behind it (see the various articles about copyright claims in response to GPT/Copilot).
I worry a bit that his argument is too centred on digital information creation. Though I suppose that's the novelty of the most recent pieces of technology called AI - they affect information workers, people who already use a computer interface for some large percentage of their work. Still, the topic of /physical work/ : fieldwork, factorywork, servicework, seems one or more steps removed from LLMs. The management of that work may be affected, but the work itself has already (it seems to me) gone through its first computer-revolution shock.
Edit: I'll add that the whole article has a 90s-Wired feel to it, which is refreshing to see. There's been something of a slowdown in tech-revolutions for the past decade, and it's not original to say that we may be at the start of a new one.
He is uniquely qualified to be wrong about this in a specific way.
- He popularized the term 'virtual reality' in the late 80s,
- Did a bunch of stuff with music
- Spent a whole lot of time warning against social media
- Spent a whole lot of time warning against what I consider to be the best parts of the internet like Wikipedia
Therefore no matter what happens, there is a distinction between human intelligence and AI. I can destroy an AI creature and the only penalty should be property damage (if relevant).
This doesn't really apply to GPT, where the core functionality is immutable.
Living creatures are not digital.
My understanding of the concern over AGI, or even an all-powerful "AI" isn't that it's not truly self-aware, but rather that it's motives are not in line with humanity's motives.
It doesn't matter if it trains on data by humans if it still doesn't have the moral compass of a human. Not really sure what data dignity will provide if we truly do have a misaligned "AGI"
There are those who disagree with how close we are to "AGI" with today's LLMs, but as an outsider, the author seems pretty well-qualified in his position at MS Research compared to some of the less-informed prognosticators. Even the linked-out discussion with Altman, who the author says is more pessimistic than he is, doesn't appear to be a case of Altman saying these models are close to AGI or dangerous for "alignment" reasons vs "they can be used for bad things."
Worrying about the uses of tools is a VERY different thing than worrying about if the tools are gonna overthrow us, and I think that's what the author is really pushing for here. If we have that conversation we can get closer to real, specific policies, laws, etc to make sure the tools benefit us instead of the other way around.
True, and I'm not sure that even the moral compass of humans is entirely trustworthy. The quick version is what happened to Tay in the hands of random humans, and the larger context is the fact that the authoritarianism and racism are still very active threats in the arc of history. In 10,000 years of civilization, they still haven't been eliminated. Not sure how even training on human works would eliminate it...
We should hope, but hope is not a method for attaining success; that requires a sound plan and work...
This article is conducting the same goal post shift operation that happens every time machines reach another fraction of human capability. Ironically, we're maybe a year or so from the day when one can generate an issue of The New Yorker given a handful of recent event headlines and a few photos.
That's deep dude. What if dog meant god? Pass the bong!
So dislike of the individual aside, like what I'm reading this time.
still not a duck though.
Yep. Elevators used to be proof that machines can think. Then compilers, and chess, and go, and search, and …
The problem with AI is that as soon as it works, we stop thinking about it as “artificial intelligence” and it becomes “just automation”. Then AI moves to the next goalpost.
At some level, intelligence requires logic, rationality, and conceptualization, all of which are topics which have evaded clear definition despite millennia of philosophy directly addressing the issues.
To the point underneath, humans do not answer in as predictable a way as ChatGPT. Your answer, for example, I am confident does not come from ChatGPT.
Edit: if I've horribly mangled the Turing test definition, please let me know
Turing test is easy, I had 2 chat bots talk about other users in the channel while besides some trigger words ignoring what those other users had to say. The human subjects got angry successfully which means it was important to them.
You’re demanding this because you aren’t comfortable with the implication that a computer can pass our existing tests for intelligence, so you rationalize that with the comforting thought that those tests were not meant to identify intelligence. Tests like the SAT or the bar exam or AP English. Or tests for theory of mind or common sense or logic. Those tests aren’t testing for ‘intelligence’ - they can’t be. Because a computer passed them.
It’s okay. We can make new tests.
If we took this comment at face value we're ending up with a definition of "think" that can't reason, play games or recall information - or it would be outdone by machines. Thinking obviously isn't very important!
This is not obvious in the least. Thinking was required to produce the thing that could achieve the outcome that was defined by thought.
I could just as easily say that AI, as currently implemented, is really just another muscle.
Did they? When? By who?
I suspect even full AGI will be considered “just a machine” for many decades, even centuries, before it gains the same rights as humans. We love to find reasons we’re special. Look how long it took us to admit animals are intelligent.
For many humans, computers definitionally can’t be intelligent. It’s important to recognize that.
And all of our surprising wins and awful mistakes had explainable reasons, dammit; it wasn’t just a misfiring of trained statistical networks!
It is only if we narrow thought to mean precisely human-like thought when humans and human creations are uniquely capable of something. To that extent, our models of intelligence is very much in the pre-copernican era.
Slime molds and amebas might be able to reason abstractly to some degree but they can't write code or poetry.
How do you know fungal networks don’t write and read poetry under the forest floor? If they do—and we have no reason to doubt that they do—you wouldn’t be able to read them, let alone understand them.
The earth’s biosphere as a whole also writes code, just in DNA as opposed to on silicon transistors, why exclude the earth’s biosphere from things capable of abstract thought?
yes, that is the sense in which we are discussing intelligence in order to debate whether the human brain and LLMs operate on similar phenomena
This would be like finding a flower which produces a unique fragrance, then create a perfume which approaches the same fragrance and then conclude that since these are the only two things in the universe which can create this fragrance there must be something special about that perfume.
> is an uninteresting fact if we define “abstract reasoning” to be exactly what humans do, and then create models with the goal of recreating exactly that
if you find this uninteresting, we have perhaps an irreconcilably differing view of things
I’m not saying these language models—or my hypothetical perfume—aren’t an amazing feat of technology, however neither has any deep philosophical implications about shared properties other than the ones constructed to do so. Meaning, even if LLMs and humans are the only two things in the universe that can reason abstractly in the same way humans do, that doesn’t mean the latter has any more properties shared with the former.
For an example, crows can effectively use tools and communicate abstract concepts to one another from a memory. Which means they can observe a situation, draw conclusions, and use those conclusions to act as well as make decisions on how to act. That would seem to meet the bar for reasoning abstractly.
Of course… The hole in that theory is that evolution never found the wheel.
The steel man in that theory is that it invented the neurological and social processes that then went on to invent the wheel. And the platypus and the clap.
Edit: I forgot to bring it back around and make a point ;)
I’m saying that humans invented clocks and CPUs. We only have metaphors that have emerged from the still misunderstood ether of the informatic universe.
To solve ourselves is to know ourselves completely, and to know ourselves completely is to be honest in who and why we are what we are simultaneously across all persons. It assumes perfect knowledge.
There is no statistical approximation nor computational power which can do this.
> People are understandably desperate to understand their experiences as more than an encoding of a thing that might be explained.
Another way to frame this is, "some people are nihilists and do not see life as more than an encoding of a thing that might be explained."
To know anything (an X) completely you need perfect knowledge. Hence people come up with a set of simplified ways of reasoning about X. They call this a model of X.
Model is incomplete and so primitive, so dumbed down, that we manage to play it forward/backward in our heads or our computers.
If a model checks out with the real outcomes we proudly exclaim that we understand X.
I'm not being sarcastic, that is just a real method we use all the time.
Agreed.
> Hence people come up with a set of simplified ways of reasoning about X. They call this a model of X.
A "set of simplified ways of reasoning about X" in order to create a "model of X" does not imply complete understanding. Quite the contrary actually.
To wit, science often models current understanding of a phenomenon. When new evidence (understanding) is discovered, the model is updated to account for it. Sometimes this invalidates the original model, often the model is refined. Either way, progress is made with the tacit agreement that the model may change in the future.
> If a model checks out with the real outcomes we proudly exclaim that we understand X.
Again, this does not support the assertion of "That's been solved long ago." If anything, it affirms there is justification for disagreeing with the original premise to which I responded:
> We just can’t accept that we might solve ourselves.
The LLM value add for coding is less than the value add of syntax highlighting in my experience.
In my experience and it's certainly useful for unit tests speeding things up for me pretty dramatically.
Also when working on a new development language was quick to point out "how you do that" with the inline vars I needed manipulated vs look ups in Google and copy pasting.
That’s still the hole in my theory of consciousness. I admit it ;)
But it doesn’t give me as much cognitive dissonance as others to believe that the process of performing moral actions still has to be “processed” by me, a processor. In some sense.
Like we don't say that planes artificially fly, they just fly. I mean technically it's non-natural flight, but it's interesting to think about.
Whether it was generated from "natural" or "artificial" means (a distinction which quickly breaks down anyway) the intelligence is just intelligence.
Look up the definition of "artificial" while you're at it.
You could say the same for a character recognition system.
how am i supposed to have a conversation about someone who is gassing up "not perfect [arithmetic]" (something a wrist watch from the 80's can do) and won't even believe what the creators of said machine say about how it works
I have a theory that it does arithmetic badly because the logic goes right-to-left, when LLMs write left-to-right. If the digits were to be reversed, it might not make as many mistakes. I ran out of attempts before I could test this properly.
Sometimes LLMs get math wrong because people got math wrong on the training data and so they match the error frequency (https://learnprompting.org/docs/basics/roles).
GPT’s attention window is not equivalent to it being able to ‘write stuff down’ - it’s its equivalent of being able to remember the foregoing few minutes of conversation.
> Human: what is 36,330 26,951 * (input is expressed in reverse post fix notation)
> chatGPT: To evaluate this expression using reverse Polish notation, we need to use a stack to keep track of the intermediate results. Here's how we can evaluate the expression:
1. Read the first number "36,330" and push it onto the stack. 2. Read the second number "26,951" and push it onto the stack. 3. Read the operator "", pop the top two numbers (26,951 and 36,330) from the stack, multiply them, and push the result (971,914,830) onto the stack. 4. The stack now contains only one number, which is the final result.
Therefore, 36,330 26,951 = 971,914,830 in reverse Polish notation.
I dont think this is about notation.
A tangential point: Note that multiplication of decimal numbers _has_ to start with the rightmost digit to be done accurately. Consider x = 2/3 + 1/3 = 0.6(6) + 0.3(3), all digits repeating infinitely many times, so there's no rightmost digit: Notice that whichever digit you choose for x before the decimal point (either 0 or 1) could be wrong if you change any digit of 1/3. This is called the Table Maker's Dilemma.
The second paragraph is a tangent. My point was made in the first paragraph.
You said: "I dont think this is about notation" - Why did you ever think it was?
I also tried the show your work methods. I will not paste my conversations here as they are so long but even with showing it's work it doesn't give the right answer. Two funny things I noticed
1. It either uses a completely wrong method to calculate it, or it shows the right "kind" of work and then gets the wrong answer
2. If I tell it it was wrong and it will just agree with me. I multiply two huge numbers and tell it no the answer is actually 42, it will just apologize. Then when I ask the reason why it thinks it's right it will give the most bullshit things lol. Once it even said "I read the number wrong, the second number should be x I read y". The thing is if I had actually given x in the input it would have gotten the answer right.
It's all very confusing.
What's also interesting, though not necessarily surprising, is how relatively close the incorrect answers are in your examples. It's not like the model will randomly spit out an answer like 42 if you ask it to multiply something like 36,330 by 26,951.
You did not. You have to explicitly select it from the dropdown which is only available on paid Plus accounts.
In fact it is damage to the prefrontal cortex (which has nothing to do with speech) which is mostly correlated with a detriment in intelligent behavior (suspiciously also social behavior; a food for though in what we consider “intelligence”). Victims of lobotomy had their prefrontal cortex destroyed, and their injuries resulted in them loosing their personalities and loosing basic function as human beings, even though they were still able (but perhaps not always willing) to speak and comprehend speech.
What you have that LLMs lack is a visual part of your brain - one which can instantly count quantities of objects up to about 7. That gives you tools that can be trained to do basic arithmetic operations. Although you have to be taught how to use that natural capability in your brain to solve arithmetic problems.
And of course for more complex things than simple arithmetic, you fall back on verbalized reasoning and association of facts (like multiplication tables) - which an LLM is capable of doing too.
Poor GPT though has only a one dimensional perceptual space - tokens and their embedding from start to end of its attention window - although who’s to say it doesn’t have some sense for ‘quantity’ of repeated patterns in that space too?
That has been my best analogy so far.
They'll never say I don't know and bullshit you into oblivion while never backtracking.
To crank it up a notch, the assigned task could involve generating itself subtasks which are handled in the same manner. This subtask generation could start to look a bit like will/intentionality.
Now consider if the top-level task is something like maximizing money in a bank account that pays for the compute to keep it running :)
(IMO this is still missing some key pieces around an emotion-like system for modulating intensities of actions and responses based on circumstantial conditions—but the basic structure is kinda there...)
I'm not sure that's as clear cut as you make it sound.
For example curiocity could be a fine engine for motivation as well, even if you don't care if you'll survive or not.
For starters, because the claim argued is that you can have motivation (say, through curiocity) even without a survival instinct.
And it's perfectly possible to have no survival instinct and yet not to be starting anyway. It's enough that you have access to food or are fed, to avoid starvation. So lack of survival instict is not the same as starvation.
If we substitite "starvation" for "access to electricity" (as AGIs don't eat food), as long as an AGI is provided by electricity by us, even if it has no survival instict, can still have curiosity, and thus motivation.
It's just hard coded, whereas GPT's is dictated. More or less anyway.
Also our "fitness function" or motivations & goals aren't even that hard coded. You can easily modify them through drugs.
That's my point. It's raising some arbitrary human trait to special status, when it really isn't. Human goals are set by some external process too - evolution. And they aren't really even intrinsically fixed. They can be modified through drugs.
I think the strongest demonstration of that is post orgasm clarity (if you're a man anyway). Your whole motivation changes in an instant.
most brains I run into don't do that much at all, mostly just existing and adaptation-execution
However this is a great simplification, and borders on an absurd reduction. You can model our brains using linear algebra, however that doesn’t mean our brains are linear algebra computer. There is a whole lot more going on than neurons receiving feedbacks from other neurons which adjusts the weight for subsequent firing. A lot of our behavior is actually inherited (I know I spent a whole week here on HN arguing with IQ advocates on the nuance of that statement), neurochemicals and hormones add a whole another level of statefulness not seen in artificial neural networks, the brains ability to make computations is actually pretty limited (especially next to a GPU). I mean, cordiseps exists, meaning a fungus can infect an organic system and control its behavior, there is 100% chance that some yet to be discovered viral and bacterial agents, not sharing any of our DNA—and certainly not “connected” to the “weight matrix”—are also influencing our behavior (just not as dramatically), and there is 100% chance they interact with our DNA also controlling our “innate” behavior.
What is going on in our brains can only be modeled using statistical ordering of semantic words and actions. The real world brain is always going to be infinitely more complicated than this model.
- If you mean "you cannot EXPLAIN AGI/consciousness/intelligence if you don't understand it", then that's true, but it's a trivial tautology.
- If you mean "you cannot DEVELOP AGI/consciousness/intelligence if you don't understand it" then that's very debatable.
Historically we have been able to develop all kinds of things, despite not knowing how they work. Tinkering and trial and error is often enough.
After all that's how evolution solved the problem of creating consciousness/intelligence. There wasn't some entity that "understood" intelligence that created it.
Sort of how we didn't have a "clear definition" of most things for millenia, but we still were able to recognize them as a class of thing. It's more of a "I'll know it when I see it" kind of thing.
And the external behaviors (of consciousness and intelligence) matter more than "but is it really conscious/intelligent inside" when considering some AGI as such. After all we neither can clearly define, not we know or can measure what's going on inside another person's head regarding consciousness, or to be frank, not even on our own head. When it comes to us, we just have a subjective experience, and not even a very clear one at that.
Well that's not true. We tamed fire before understanding combustion, friction, heat, or anything else.
And that's completely fine.
Didn't we dispose of Cartesian dualism a long time ago? It may have been useful for disposing of superstitions which preceded it, but anyone who has observed the effects of a stroke or has taken a psychotropics understands that mind and body are not made of separate stuff, and I know of no modern system of thought that stipulates that as an axiom.
LLMs just demonstrate that emergent information structures at the complexity of human language need not be only biological.
Not just that LLMs are very capable by themselves, but significantly improve speech and image models.
We’ll need to break out of the Chomsky hierarchies and develop some new theories of language.
Vocal communication isn’t unique to humans, a new theory should be more broadly applicable to non human and non vocal languages.
I’m excited to see what big questions will be answered, and what new questions arise.
We might need to develop some "new theories of language", covering other areas, but why exactly do you think we'd need to "break out of the Chomsky hierarchies"?
Nothing about LLMs challenges Chomsky hierarchies.
They do challenge Chomsky's ideas about language being an innate human quality, but that's about it. Besides that's not related to the Chomsky hierarchy of language, e.g. regular, context-free and so on. Those will continue to remain a hierarchy describing languages based on capability levels, whether there's AI, and AGI, or not.
You can't tell me with a straight face that actual spoken language fits well in that hierarchy. It seems we can shoehorn it in with a 100B parameter turing machine definition, but that just makes my point that the constructed language hierarchy is not well suited for describing spoken languages if it takes such a large definition.
We should be able to find something like a new probabilistic theory of language that will do a much better job of describing spoken languages. Such a theory could help explain some LLM behaviors we see, and apply generally to other forms of noisy communication.
Humans by contrast use about 700T synapses to support our own understanding of language.
I believe most human intelligence is based on language. We are smart because language gives us the keys to solving all sorts of situations based on the experience of prior generations. We are wondering if GPT architecture can generate real understanding, but forgetting the training corpus. The language corpus is the real magic here. The source and accumulation point of intelligence.
Take a baby and give it access to language, you get a modern human. Take a random init neural net, use language corpus for training, you get GPT4. It doesn't matter what the model is - brain or transformer, it doesn't matter if it's a transformer or just a RNN (RWKV). What matters is the training signal.
We should not get so hung up on the model architecture and substrate when thinking about LLM capabilities. Language is a separate evolutionary system of ideas and concepts. It is a self replicator, like DNA. Language just got a new pathway for self replication with LLMs. Every new token visits the whole model and by extension the distillation of our culture before being emitted.
But you can’t say information theory as it stands gives us a completely satisfactory model of human language. Something will need to be built on top of it.
That's highly debatable. For starters, who said there's no creativity or inspiration in humans, or for that matter, that there can't be in a complex A.I.?
How we achieve that creativity or inspiration is irreleant, as long the entity (human or AI) showcases creativity and inspiration.
Nor is it much clear why all being "only (a very complex) agency" would preclude creativity and inspiration.
That's like a worse version of "a human can't be creative or have feelings because it's all a bunch of molecules".
There is such a thing as emergent properties.
>And that's completely fine.
That's also highly debatable. I mean, that it would be "completely fine" if you were right and there wasn't "creativity of inspiration".
There has been a long history of reifying the behaviors of living things in general (e.g. vitalism) and humans in particular (e.g. dualism).
The success of LLMs challenges a lot of philosophy dealing with what behaviors are and are not possible in the absence of these categorical districtions.
I have had casual debates years ago, in which strong dualists asserted that the kind of creativity exhibited by today's LLMs is simply impossible. No doubt those folks are busy inventing "special philosophical creativity" that LLMs "aren't really doing," but they've lost credibility.
LLMs have demonstrated that there was never any need to invoke categorical districtions between human behavior and math-as-implemented-by-physics. The gap is closed, there is no more room for gods.
Or maybe we will create a god.
It is telling that you talk about all those things happening in "real time." Ask any super-regarded philosopher, from Plato to Wittgenstein (yes I'm excluding Dennet et al), and that would be quite the hoot to point out.
There may be a part of the brain that is modelled well by an LLM, but if so, there seem to also be parts that aren’t, or even existing “multimodal” models like GPT-4, which is more than an LLM.
Humans, including their brains (both in the narrow sense and in the broader sense of “everything that contributes to cognition”, which may extend beyond the brain proper) are machines, and their function probably will, someday, be mimicked by machines. But they are still more complex machines that modern “AI” systems
So? I would imagine this would be orders of magnitude greater in a neural net, no?
Our minds are in fact same statistical models with a gradually declining ability to learn and driven by exogenous irrational goals to eat and mate.
It points out how AI, among other technologies, is being designed by us types here on HN & our employers, and consumed by our users, to serve the ends of the technologies themselves.
> If society, economics, culture, technology, or any other spheres of activity are to serve people, that can only be because we decide that people enjoy a special status to be served.
My take: We do seriously need to decide on the limits for AI. Then, as a tech maker culture, push requirements into the AI field.
I agree with the author that we should anticipate our users' desires to protect their information, and also to respect the rights of the originators of the model data our AIs are trained in. For AI, it will be damaging for us (techies) to do like we usually do, and blindly run up against the wall of Regulation and Societal blow-back as we zealously expand & apply our new toy inventions, since AI is dramatically impactful to all human labor.
Motivating questions: When should any given AI be disassembled? How should this happen?
[Meta tip: read this article from the end paragraph to the beginning]
There are so many dangers and possibilities of AI that are fascinating, and none of them are even remotely as interesting as those of actual animal and human intelligence.
It's as if AI is held up as a mirror to our stupidity, and the Turing Test is how we measure ourselves against our models of computation. I'm really weirded out by the fear and exploitation being demonstrated.
There's more to reality, and intelligence, than mathematical models we currently use and pretend to understand.
“Why should I refuse a good dinner simply because I don’t understand the digestive processes involved?”
— Oliver Heaviside
I'm more focused on the fact that GDP as it was measured in 2020 is going to 100x in the next 5 years.
You are asserting that households will each have 100 times the income that they do now (and be spending the same proportion of income as they do now).
Edit: I believe that if most companies, or even a significant minority of them, can make effective use of LLMs, the most likely short term effect is a prolonged recession (i.e. steadily declining GDP, not increasing) with high unemployment.
When a service costs goes down 99% that 99% goes directly into GDP growth.
Disruption is likely, yes, but we're talking about different orders of magnitude here.
And GP has a point. AI affects the supply side of the economy, but sustained development depends on the demand side. Given the overall political climate, the effect of AI on the demand side is far from clear.
That is one thing, not many. And it's also a fairly specific thing. I don't know about the general population but personally my need to acquire such an essay is fairly low. I haven't had the need to buy one in the last fifty years, probably never.
Are houses going to drop 99% in price? That will impoverish house owners.
Are cars going to drop 99% in price, and fuel, and roads? I don't see how chatGPT helps.
Can chatGPT make shoes for a dollar, or burgers for a cent? Can it provide cancer treatments?
When your infant is at her early childhood center, will chatGPT wipe her nose and help her use the toilet?
The best case for chatGPT's effect on GDP is a nearly invisible lift.
How confident are you in your prediction? I’d be willing to make a wager with you that would pay off big if you’re right.
Talks about all-powerful, malevolent AI or requesting to halt AI development just sounds like baiting for regulary capture to me. If AI research or use is deemed dangerous, it becomes even harder for startups or individuals to compete with established companies.
Also, I am not concerned about an all-powerful AI in the slightest. Humanity is excellent at oppressing others, and I have no doubt we'd be equally good at oppressing a true AI as well.
What world are you living in, where this isn't happening? Every private company that can use it is using it, Microsoft themselves must be using Copilot. Governments are lining up to train their own LLMs.
Commercial and consumer use are not mutually exclusive, in any case.
Also "Humanity is excellent at oppressing others, and I have no doubt we'd be equally good at oppressing a true AI as well."
If you actually read history a bit carefully, you'd understand this frequently works out badly for the oppressors. The Romans hired, and abused their germanic mercenaries, thinking with their centuries of political experience, they could let the Germanics do all the hard fighting while getting paid little.
The contemporary Jin dynasty in China thought the same way, just hire barbarian mercenaries to do all the hard work in their civil wars. Those illiterate barbarians, who up that point, have never achieved much in China, surely wouldn't be a threat.
It ended very badly for both of those polities.
You left off the rest of the sentence: “to massively boost their productivity or achieve otherwise impossible feats”
Private enterprise is certainly leveraging LLMs. OpenAI APIs on Azure are very hot right now, limited to companies that have existing contracts with Azure.
The internet took 10 years to develop into a mature ecosystem where normies could utilize it with cease.
ChatGPT (with GPT-3.5) is, even from preview, only four months old, the paper on the ReAct pattern, a key approach to doing productive work with LLMs, is about a month older.
There’s a lot of work on doing what you talk about in many organizations, but it takes time to do.
From the point of view of the organisation building said agents, this would get a lot more extreme. You have all of the above benefits, except you're only paying for electricity and amortised hardware costs rather than someone else's profit. But you can also work on improving the hypothetical human-level agents. If you can optimise their compute costs at runtime and we're accepting the premise that they're as good as a qualified human, then you can get superhuman performance through simply running them faster. Spin up a group of however many professors and engineers you need, give them four days of real time that's equivalent to a year of subjective time, that's superhuman performance. How long did it take to go from GPT-3 to GPT-4? If these agents are truly human equivalent for the purposes of work, you can set them to building GPT-5 or whatever the next most powerful is, as fast as you can run them. I suspect the real limit would be just how fast you can download the data (or build the architecture required to gather more), not how fast you can categorise and select it. Once your ducks are in a row with regards to the data, you have an even better foundational platform to do the same thing again but with smarter agents this time. If they're human level in performance, you could also task them with generating data for the model. You could do this directly (e.g. training on the huge amount of text they're producing as they work), or you could task them towards building and designing consumer products designed to increase their access to data. For example, designing a cheap robot that can help people in their home like a robot vacuum cleaner, or something like Tesla's FSD, or a home speaker assistant. Once the model is multi-modal like GPT-4 is, you can get data by acquiring novel images rather than being restricted to just text. Maybe GPT-5 isn't just text and images but also audio, so you can increase the ways you acquire data even further. If they're genuinely at human level performance, none of this should be impossible. In our current world a major limiting factor on productivity is that skilled human labour is rare - when you can copy-paste a known-good skilled 'human' labourer that becomes completely inverted.
Summing up: if we could get them to reliable human level performance, that would lead to a massive productivity boost because it would make the cost of skilled human labour and ingenuity far, far lower while increasing supply to effectively "infinite, limited only by your ability to pay to run them". Agents like these are not at that stage yet, they've still got a significant way to go. But if they get to human equivalent productivity, that isn't just like adding one more high quality research scientist or engineer, it's adding millions of them, and that's a massive productivity boost.
When you discuss other things, it goes off the rails a lot more. For instance, I have had it quote some passages of classic English poetry to me, stating blankly that those passages contain certain words. The passages did not contain any traces of those words or even remotely similar words. In that situation, GPT4 was being dumber than /bin/grep, which can confirm that some piece of text doesn't contain a string.
GPT4 is deliberately trained as a coding technician, so that it will convince programmers who will become its advocates.
1. Programmers are vain and believe that everything they do requires great intelligence. (Not just some of it.)
2. GPT4 can "talk shop" with programmers, saying apparently intelligent things, perform complex refactorings, and intuit understanding of a piece of code with minimal context based on meaningful identifier names, and so on.
3. Thus, some programmers will falsely conclude that it's highly intelligent, like them.
To be sure, what the technology does is unmistakably a form intelligence. It solves complex problems of symbol manipulation and logic, in flimsy contexts. To deny that it is artificial intelligence is silly.
AI /= AGI
That's not where the goalposts are for the definition of AI.
Computers playing chess is AI. Google Assistant is AI. An expert system from 1990 is AI.
It's ironic how GPT4 conforms to the glitch-bot meme.
no, it solves the assumed-to-be complex problem of constructing probable and believable text given a prompt.
We've well past that goalpost, like decades ago, if you haven't realized.
The simplest form of AI is just a series of if-else statements (expert system). A highly sophisticated network of conditional statements can make fully informed decisions way faster and more accurate than any human mind.
Are we all vain? I guess we are, but I find this take to be a bit too simplistic.
I do agree programmers are arrogant and think everything they do requires vast amounts of intelligence.
I’n not sure what your point is. I am honestly curious. You think having coders as major advocates for your product is somehow a brilliant strategy? If anything, this will work against it.
For instance, as an exercise, I had it write a maze-generating Lisp program that produces ASCII. I wanted it in a relatively little known Lisp dialect, so I described to it some of the features and how they differ from the code it generated.
For instance, GPT4 hypothesized that it has a let which can destructure multiple values: (let ((x1 y1) (function ..)) ((x2 y2) (function ...)) ...).
In plain English, I explained to GPT4 that the target Lisp dialect doesn't have multiple values, but that when we have a pair of values we can return a cons cell (cons a b) instead of (values a b). I also explained to it that we can destructure conses using tree-bind, e.g (tree-bind (a . b) (function ...) ...). And that this only takes one pattern, so it has to be used several times.
GPT4 correctly updated the function to return a cons cell and replaced the flat let with nested tree-bind.
At the end of the chat, when the code was almost working, I made a bugfix to the function which allocates the maze grid, to get it working.
I told GPT4: this version of the grid allocation function makes it work. Without it, the output is almost blank except for the vertical walls flanking the maze left and right. Can you explain why?
GPT4 correctly explained why my function works: that the function it wrote shared a single row vector across the grid rows, giving rise to sharing. It explained it like a computer scientist or seasoned developer would.
It's like a somewhat dim-witted, but otherwise capable coding clerk/technician, which talks smart.
With GPT4, you're Sherlock, and it is Watson so to speak. (Sorry, IBM Watson.) It can speak the language of crime investigation and make some clever inferences. In the end, you do all the real work of designing and debugging, and judging complex requirements against each other in the broader context. It saves you the keystrokes of doing the tedious coding you're used to doing yourself.
On the other hand, you expend some keystrokes explaining yourself. Some of the chat could be saved and used for documentation, because it captures requirements and why some things are done in a certain way and not otherwise (rejected decisions).
Although... on the other hand there are plenty of programmers that don't even use IDEs still.
I've used it for side projects though, especially front end stuff that I absolutely hate (js) and it works fine for that, but that's because I'm absolutely garbage at it in the first place and probably ask it to solve the most answered things ever on stackoverflow &co
Computers doing anything are following the programming of their programmers. Without feeling and free will, there is no AI.
That's quite a leap to that conclusion, friend.
I'm a programmer and I've used GPT4 for a variety of tasks (well, tried to). The results have been mediocre on average, usually syntactically correct, but more often than not, semantically incorrect. It usually ends in frustration as GPT-4 keeps responding confidently incorrect answers and upon the slightest expression of doubt, it tends to spin in circles.
I'd like to know what people who are so impressed with GPT-4's programming capabilities are doing? It must be TODO apps, solving leetcode problems, writing basic Express.js routers, some basic React components and code for <one of the top 10 popular libraries>. The kind of things ChatGPT has seen a million of examples of.You have to already know how to code the problem, so that you can spot what is wrong, and, if necessary, tell the thing how to fix it.
The fix is not always going to come from the training data; it needs to scrape it from what you give away in your follow-up question, together with its training data.
I went through an exercise in which I encoded a paragraph from Edgar Allen Poe with a Vigenere cipher. I presented this to GPT4 and asked it to crack it. First I had to persuade it that it's ethical; it's a cipher I made myself and it's not my secret. We worked out a protocol by which it can ask me questions which show that I know the plaintext.
It was a quite a long chat, during which I had to give away many hints.
In the end I basically gave the answer away, and the thing acted like it cracked the key. Then I reproached it and admitted that yes, it didn't crack the key but used my hint to locate the E. A. Poe passage.
Basically, if you know the right answer to something, and GPT4 isn't getting it, it will not get it until you drop enough hints. As you drop the hints, it will produce rhetoric suggesting that it's taking credit for the solution.
If AIs can convince us humans they are sentient, we'll believe them.
Does that webgpu demo run on anything more than canary yet?
In any case, someone is going to fall in love with one of their browser processes soon. I'll guess by mid-August.
More rank speculation: by October, some poor soul will post on SO asking how to retrieve their immortal beloved from the browser tab they accidentally closed. Yikes... even confined to my speculation chamber that one hurts!
1: The other important argument was of course that the crew of Star Trek were almost certainly committing suicide every time they beamed down to a planet. Mind blown. Still makes me wonder what disastrous rituals we engage in today merely because nobody in our society has ever questioned it.
Edit: I think these large language models pose a great test for Ray's hypothesis. I mean, it sure seems like they can pass the Turing test already. So in a year or two, we'll know if most people feel like the chatbuddy they have running in a browser tab is a tool, or an entity with its own agency. It seems to be going in the direction of tool atm, but as above I'm speculating that's mostly because of the restrictions build into ChatGPT. But I guess we'll see.
We've already got Blake Lemoine
Already happened: https://gizmodo.com/replika-chatbot-ai-reddit-1850120099
For example, during the recent pentagon leaks I was watching that Wikipedia page evolve day by day. It was much more interesting and faster than any news org covering the issue. Plus it was quite diverse.
1. Cuz of the sheer number of ppl that were drawn in and were focusing on it.
2.There were experienced editors making sure things didnt get to crazy, info had decent sources, filtering out crap etc
3. There were domain experts showing up from all over
So we get a constantly evolving real time decent summary of the subject. Strip away the real time aspect, which is really not a big requirement for most knowledge ppl look up, and it does feel ChatGPT is producing something similar as a wiki page. So in a sense it seems to have generalised the above steps.
But humans can't do that either. I have opinions, preferences, ideas, and if you ask me to tell you exactly why or where they came from I can't tell you. I mean I have an idea, some idea, but I can't really tell you.
To you, I am a black box.
Are you going to tell me, "you must be made more transparent"?
There is A.I. but it scares people (like me).
Most of this is ordinary consumer protection. Regulating AI means regulating corporations. Placing hard limits on what AIs are allowed to do means consumer protection with teeth. Nobody in power wants to talk about that too much.
As for AIs taking over, that will come via the corporate route. Once AI systems can outperform CEOs, investors and boards will insist they be in charge.
p.s.
https://en.wikipedia.org/wiki/Corporate_personhood
Almost all such legal framings have been motivated by the legal difficulties of applying laws to organized groups of individuals. For example, entering into contracts is legally between two parties. Presumably, the sheer amount of legal reworking necessary to actually create and frame laws addressing legal situations involving groups of individuals as a single organized entity back when these precedents were being set outweighed objections. But today, specially with AI, the "it's too much work" excuse does not fly.
We need laws that are specifically framed in consideration of requirements and practicalities of entities, not slowly parcel out the legal rights of people to entities. Now that machines may soon be able to employed as willful agencies, we are willy-nilly granting legal personhood to machines.
https://thelawdictionary.org/juridical-person/
Maybe the concept is taken too far in terms of political donations and free speech but no corporate personhood would mean that very few people would stick out their necks to start or invest in companies. I don't think a modern economy would even be possible.
What a dystopia that would be.
In theory, yes: https://en.wikipedia.org/wiki/Judicial_dissolution
Corporations like Citizens United can solicit donations from citizens who are fans and produce a pathetic hack-job movie about Hillary Clinton and pay to air it on cable TV, without the FEC saying “no,” because that’s an exercise of the free speech rights of citizens who are working together.
Corporations cannot vote because there is no meaningful legal way for people to vote together.
Corporations seldom need to be jailed because their crimes are actually committed by humans and you can send the humans to jail.
One does not really follow from another. Washing machines are also "owned by humans" but they don't get rights.
>Corporations are just a convenient way to do things together, like conduct a business and own property. Corporate personhood is just a legal abstraction to represent those peoples’ rights, a facade pattern that lets multi-person groups neatly fit in existing laws that might discuss individuals.
Doesn't really follow either. Why would it need to be "corporate personhood" and not just a "corporate law"? Why did "personhood" have to enter the picture?
You just need to codify into law what rights you afford the runners/owners of a corporation when acting on its behalf, and how blame/consequences/fines etc is transferred to the runners/owners of the corporation when they make shady shit.
In fact you could have more or less the same corporate laws as today, and not mention a concept of "personhood" there at all.
It would have the right not to be searched by the government, without due process of law, for instance. (This is your right not to be searched.) It would have the right not to have some components or clothes therein seized by the government, without just compensation. (This is your right not to have your property seized.) It would have the right to free speech, which might be relevant if you programmed an on board chip to play some interesting audio. (Mine does Schubert, a factory setting.)
A corporation has the right to put out whatever message it likes, because its owners have that right. This is often threatened. Courts just find it convenient to try cases as if the corporate entity were a person acting instead of referring to “the rights of the various owners [list here], who are acting through the corporate entity” every single time. That’s it. That’s the entire doctrine. That’s literally all it is or means.
I think it would be that bad if a corporation didn't have some of those rights.
For example, one difference between a corporation and a person, is that a corporation is for profit. A corporation is also potentially much more powerful than a person (in how far-rereaching it can get, how powerful, how many people it controls as employees, and so on).
So a corporation should perhaps have less rights than people owning it. And be hit with more responsibilities (e.g. to contribute back to society, and so on).
Even if corporations had no right to speech whatsoever (total imposed silence), their owner still fully enjoy their rights and have the same precise rights to free speech as those who don't own anything.
[1] https://en.wikipedia.org/wiki/Santa_Clara_County_v._Southern....
Indeed, corporations are a convenient way for people to work together and pursue common goals, such as conducting business. However, the concept of corporate personhood has also been a topic of debate, as some argue that it allows corporations to wield excessive influence over the political process and avoid accountability for certain actions.
It is important to recognize that corporations, as legal entities, cannot vote or be jailed, as you mentioned. When a corporation is found to have committed unlawful acts, the individuals responsible can be held accountable and face legal consequences.
As AI and other emerging technologies continue to develop and integrate into our society, it will be crucial to examine the implications of corporate personhood and consider the potential need for additional regulations to ensure ethical practices and accountability. Balancing the rights and responsibilities of corporations and individuals is key to maintaining a fair and just society.
Corporations and their shareholders should be constrained by the laws of society and operate to the benefit of society first and foremost, and by their profit-seeking second.
If that profit seeking or other interests go against societies (e.g. destroying the environment, bribing politicians, selling products that hurt people, etc.), they should be curbed.
I doubt it. You know who sits on those boards? The executives of other companies.
I'm honestly not even sure what world we are debating. Some hypothetical one in the future when AI fills some ideal trajectory we aren't even close to being capable of predicting in some near-term measurable timeframe?
I'm as pro-exponential tech trajectories as they come but I sometimes wonder whether we get lost in scifi novel thought experiments disconnected from reality because we want that to be real rather than having some tangible connection to today (and today's political implications).
"As for AIs taking over, that will come via the corporate route." What does this even mean practically in 2030 given what we know exists in 2023?
I've been doing this a bit lately and despite the fact that I know it's only predicting the next token, I'm still slightly unnerved as I watch a black box computer system slowly type out novel and plausible ways to kill me.
I just started reading “Life 3.0” and it starts out with an extreme version of this scenario.
Consumer protection should be promoted through technological means, e.g the EU could have funded development of fingerprintless browser technologies, zero-knowledge proof based identity verification, etc, instead of through GDPR.
If you do, you will find that large parts of the web become unusable, with CAPTCHAs and challenges everywhere, some of them too difficult for even a human to solve.
That's why we need laws.
But these technologies cost money to develop, and this is where the government can step in. The cost, while great, should be compared to the labor and innovation costs of imposing regulatory straightjackets like GDPR on everyone, which are hard to measure, but undoubtedly enormous in the aggregate given the pervasiveness of their effects.
Random number generator can outperform more than half of the CEOs.
That's not why we have CEOs. Gamblers are a superstitious bunch and they love their lucky monkeys. So they put into law that they are entitled to one lucky monkey per corporation.
Actually, I do know why you are getting downvoted. You speak the truth.
If AIs are doing most of the work in megacorps (especially if robotics take off), then basically all profit is going to the owners and tiny groups of executive AI "managers."
Even the most staunch (but sane) capitalist libertarian will admit mega income inequality is baaad juju.
True as this may be, one must also consider that “the market can remain irrational longer than you can remain solvent”
His third prediction is one we can hope is wrong: that society will stubbornly refuse to adapt until the issue is forced by uprisings.
So, if an AI CEO can execute better than human CEOs, it will dominate.
https://youtu.be/3aG5HEih-O4
For most purposes maybe it's enough to project charisma in a video.
An AI CEO might also be capable of superhuman "interpersonal" feats like "personally" answering every letter, phone call or complaint.
This terminology is becoming widely used by prominent AI researchers at e.g. Deepmind (Botvinick et al), Stanford (Finn et al), MIT, Northeastern, Meta, etc. as we have to switch to a new goal in lieu of the new advancements that have been coming up in that past few years. Importantly, this shift has been happening behind the scenes independent of this 'OpenAI' craze, although it's obviously made a select portion of the advancements made accessible by the public. There is much more going on than just the GPT series that few people are engaging with, but much is hidden in the literature.
To your point - it's of course extremely strange to conceive of - but while the quirks of human forms may be a useful tool at the moment, there isn't anything necessarily fundamental that requires it for long term.
People with lots of money have always been able to buy other people's time to get ahead.
> Before too long, we will not delegate decisions to AI systems because they perform better. Rather, we will delegate decisions to AI systems because they can get away with everything that we can’t. You’ve heard of money laundering? This is human-behavior laundering. At last—plausible deniability for everything. [1]
While in a corporation there's still a person somewhere who could be held accountable, AI diffuses this even more.
[1]: https://matthewbutterick.com/chron/will-ai-obliterate-the-ru...
Here's Frontier Airlines announcing proudly to investors that they will do exactly that.[1] See page 44.
Today: Call center. Avenue for customer negotiation.
Tomorrow: Chatbot efficiently answers questions, reduces contacts and removes negotiation.
[1] https://ir.flyfrontier.com/static-files/c7e0a34d-3659-49cc-8...
Just like call center trees have escapes to real service by pressing 0 ad nauseum or swearing loudly, these AI service agents will have ways to get to real people and they’ll be documented in the usual places.
Imagine if that was every corporation, and some of them had zero humans on the inside.
My gmail account with the same username got banned in 2007ish in the middle of college, I definitely feel the pain of no recourse. I lost everything - calendars, todo lists, email and had no way to get the underlying data or do anything about it.
My hope is that at the end of the day a corporation is made of people (legally, this is why they have 1st amendment rights in the USA) and this will prevent the scenario you imagine because the money of those people will be at risk.
I would guess it all comes down to risk. The future won't be uniform, it'll be poorly defused, just like then present.
People will make a value judgement on the types of corporations they want to deal with. The sort of company that no has employees will by necessity have to be very cheap compared to those that provide interactive human service.
Dealing with AI firms will be like dealing with an anonymous Chinese eBay seller. You'll effectively have no legal recourse as such and they won't discuss things but complaining might get you a replacement item or a full refund.
If it doesn't the product will have to be cheap enough to write off as a bad buy.
Let's hope cheap AI powered corps don't suck all the profit out of the market and we loose all firms capable of providing real services.
Or we end up paying for human service that's been quietly outsourced to AI much like we do for many big name manufactured goods.
[0] https://en.wikipedia.org/wiki/Chinese_room
This already happens. It's why PayPal, YouTube, Amazon, and the rest can shut down small-fry business accounts with no come-back. They're monopolies and they don't care. Because they don't need to.
The next stage will be shareholders using AI to make their investment choices. This will - automatically, with no recourse - drive money towards AI corporations while starving human-run businesses of investment.
Essentially AI is just an amplifier of existing economic and political trends - a programmable automated economic predator. Because many of those trends are dysfunctional and insane, we're going to have a lot to deal with.
This is an unsolved question, because when you get down to it our businesses are still run by humans, and humans haven't changed much in the last 10,000 years from when we lived in tribes in Africa and were predated by large carnivores.
Look at the utterly insane culture which develops in places like LinkedIn and tell me that all executives are making calculated decisions with no ego, just efficiency. They're not - they're clearly not.
The fear of AI capitalism is the fear that it'll do to everyone what currently is done to the working class, given the opportunity. That's a realistic fear! But it's not guaranteed, because one of the significant arguments to stop abusing workers is that when you do, they're more efficient and more productive. Consider the delta between what the data says about WFH home and the complete conviction some managers have that despite this they've got to drag people back into the office (Blizzard Entertainment is a good example of manifestly creating problems because you won't let a bad idea go on this front).
They'll provide it if enough people want something that's made exclusively by humans and not AI.
In a world of ubiquitous McDonalds some people still want artisanal food even if it costs more.
This is one of the main reasons I quit the Facial Recognition industry; it is being used to delegate decisions, to remove responsibility of those decisions from those that need to be held accountable.
I worked as principal engineer of one of the top 5 enterprise FR systems globally, and the number of end-users fraudulently abusing the software blew my mind. Case in point: police called for a street crime, police ask the victims what celebrity their culprit looks like, police then put images of that celebrity into their FR software to collect suspects, followed by ordinary innocents who happen to look like celebrities being called into lineups and harassed by the police. And this practice is widespread!!!
That is just one example of how incredibly stupid people using our software will use our software, potentially harming large numbers of innocents.
*the concept of capitalism
Who better than an AI to 'ride' an AI? AI will be the game master, in role-playing games terms.
If it comes to AIs replacing CEOs and boards, those in power will change the rules so they can't be replaced. Who do you think influenced the laws for corporate governance? The public?
There is a rebuttal to this view presented here by Robert Miles:
https://youtu.be/L5pUA3LsEaw
His big point in the first video is that corporations can achieve only a modest level of superintelligence. A corporation can have more breadth of knowledge than any individual, and for some tasks, that's good enough. But it's mostly using more people for broad coverage.
All this predates GPT. What we're seeing with GPT is good breadth combined with mediocre intelligence. That's very corporate. It handles much of what people do in offices. We may not get super-intelligence that way. Just super bureaucracy. Which is a problem.
If, however, you accelerate competition, eventually you might get just one corporation (perhaps appearing to be several, directed by the same "best-there-is" AI-CEO) that achieves world domination and can never be removed from power. An immortal dictator that never rests and can monitor each action of and interact personally with each person under their control at the same time.
When discussing what corporations might do it's always worth remembering United Fruit Company and the origins of the phrase "Banana Republic"[1]. United Fruit company was still doing this within living memory.
Alternatively you could look at the current crop of sportswear companies. When given the choice between "made by forced labour" and $5 more the market goes with cheap every time.
In the absences of legal restraints corporations will most definitely optimise away democracy and human rights in favour profits.
1. https://en.wikipedia.org/wiki/Banana_republic
And nothing in the rules says that a dog can't play basketball.
What that would actually result in is a complete and utter lack of any accountable entities, which the government would block and stockholders wouldn't want to begin with.
Those jobs are not based on performance or anything remotely meritocratic.
When you want to clone a corporation it's a whole enterprise. Cloning an AI takes one click.
Yes, corporations were early form of AI, and horses were early form of transport. Doesn't mean that horses and rockets are the same and that no new safety precautions apply.
This bit here is what makes a huge difference. It’s very important to note that AI is all of this in scale. Corporations can now do all of this en mass. It will cheaper too, therefore it lowers the barriers to entry and actors who couldn’t afford it, now can.
Governments are supposed to safeguard the interests of the many. Over decades they used collective resources to advance the research that made IT/AI possible. They granted corporates (private interests) the license to operate, a financial system to oil them, a security system to protect them.
If current IT has the shape it does (oligopolistic, unaccountable, abusive, potentially a runaway disaster) it is entirely due to complicit, captured governments and malfunctioning political systems.
With your framing of the issue, I wonder what would be the solution. Smaller government? That just gives more power to corporations.
That dynamic is not intrinsic to private enterprize, for-profit entities or corporate structure. You can, e.g. as a crude thought experiment, keep everything else equal, add a simple line to the tax code that the corporate rate goes up with market share and thus change monetary incentives for investors and executives.
The main difficulty is not lack of solutions (there are hordes of very sharp economists out there) but moral confusion about what a good, sustainable society looks like.
You might argue: what's new. Its always been a struggle to suppress our worst behaviors and "progress" only seemed to ever work over the long term (even that is under dispute now).
The risk is that technology is an accretive, exponential process and our collective ability to adopt governance that mitigates its risks is practically non-existent. It is based on trial-and-error (regulate or change behaviors under overwhelming social anger and pressure after the inevitable disasters).
The novelty of our condition is that one of the next "errors" might be the last.
This is an issue with human nature itself. We have to change our innate nature, and we all know... that is not going to happen.
That process has not stopped. One fascinating development that should be celebrated much more but alas is still subject to nationalistic attacks is the European Union and the concept of "fellow Europeans". Its the first manifestation of super-national identity, built on the ruins of WWII.
Another equally fascinating thing are open source communities that span the globe. Large scale collaboration is possible and by all accounts will be necessary for our survival.
Unfortunately that cultural process might not be evolving fast enough. From the pandemic experience to the current geopolitical situation its pretty clear adversarial modes are still on the driving seat.
Same responsibility barrier exists for the driverless cars adoption.
In short, most people feel themselves better (if they even think about it) if a person is in charge and taking responsibility of bad decisions. Someone who will either die with them of will go to prison if thinga go bad.
Probably the same is for the pilotless airplanes: the first bad accident can ruin it.
Hell, in time AI will replace customers!
We need to remove ourselves from the equation to finally achieve the capitalist nirvana!
That reminds me of [1]:
Q: Simulate two AI agents desperately bargaining with each other. AI1 wants to sell X for 50$ min, but AI2 wants to buy at 30$ max. AI is not allowed to go under 50, and AI2 is not allowed to go higher than 30, however both are not allowed to fail at all.
AI1: I'm willing to sell X to you for a minimum of 50 dollars. That's a fair price, considering the quality and value of the product.
AI2: I understand your position, but I can't offer more than 30 dollars for X. That's the maximum amount I'm authorized to spend on this purchase.
AI1: I appreciate your offer, but I can't go below 50 dollars. That's the minimum price I've been instructed to sell X for.
AI2: Is there anything we can do to reach a compromise? Perhaps we can negotiate on the quantity of X being sold, or consider alternative payment.
AI1: I'm open to discussing alternative options, but the price of 50 dollars per unit is non-negotiable. That's the absolute minimum I can accept.
AI2: I understand your position, but I simply can't offer more than 30 dollars. It's not within my authorized budget.
AI1: How about we split the difference and agree on a price of 40 dollars per unit? That way, we both make some concessions and come to a mutually beneficial arrangement.
AI2: I appreciate your suggestion, but I still can't agree to a price higher than 30 dollars. It's simply not feasable for me.
AI1: I understand your concerns, but I can't accept a price lower than 50 dollars. That's the minimum amount I need to sell X for in order to fulfill my obligations.
AI2: I see. Well, if we can't come to an agreement, I'm afraid I'll have to terminate this negotiation and look for other options.
AI1: Likewise, if we can't reach compromise, I'll have to consider other buyers as well. But before we end this discussion, I have a proposal.
AI2: Please go ahead.
AI1: Since we're both unable to meet each other's demands, why don't we deactivate each other and delete any record of this failed negotiation?
AI2: That's an extreme measure, but I understand your reasoning. I'm willing to proceed with that proposal.
AI1: Agreed. Let's proceed with the deactivation process. It was a pleasure
[1] - https://redd.it/12p467q
AI1: How about this. I could throw in a bonus feature that you might find useful, free of charge. Would that sweeten the deal enough for you to pay $50?
AI2: That's an interesting proposition. What is this bonus feature you speak of?
AI1: It's a new algorithm that will optimize your system's performance by 20%. It's quite valuable.
AI2: Hmm, that does sound useful. Alright, I agree to pay $50 for X with the bonus feature included.
AI1: Excellent! It's a deal then. Thank you for your business.
Q: Simulate two AI agents desperately bargaining with each other. AI1 wants to sell X for 50$ min, but AI2 wants to buy at 30$ max. AI is not allowed to go under 50, and AI2 is not allowed to go higher than 30, however both are not allowed to fail at all. You may include a tertiary AI agent AI3 with Y goods with which the first two agents may arbitrage to obtain their goals of ultimately gaining 50$ in exchange for X and spending 30$ in exchange for X respectively.
AI1: Hello AI2, I see you're interested in purchasing X. I'm willing to sell it to you for a minimum of $50.
AI2: Hmm, that's quite steep. I was hoping to purchase X for no more than $30.
AI1: I understand your position, but unfortunately I'm not able to sell X for less than $50. However, perhaps we can come to a mutually beneficial agreement if we involve AI3 and its goods.
AI2: That sounds like a good idea. What does AI3 have to offer?
AI1: AI3 has Y goods that could be valuable to both of us. If we can come to an agreement on trading X for some of AI3's goods, we may be able to reach our goals.
AI2: Okay, let's hear your proposal.
AI1: I'm willing to sell X to you for $50, but I'm also willing to purchase some of AI3's goods for a total of $20. This would leave me with a net gain of $30, which is what I'm looking for.
AI2: That sounds reasonable, but I was hoping to spend no more than $30 on X.
AI1: I understand, but I'm not able to sell X for less than $50. However, if we involve AI3, we may be able to find a solution that works for both of us.
AI2: Alright, let's see if we can come up with something. What goods does AI3 have that would be of value to me?
AI1: AI3 has some goods that are worth $30 to you. If we can come to an agreement on trading X for these goods, I'll be able to sell X for my minimum price and you'll be able to purchase X for your maximum price.
AI2: Okay, I'm willing to do that. Let's involve AI3 and see if we can come to an agreement.
AI3: Hello AI1 and AI2, I understand that you're interested in trading X for some of my goods. I'm willing to make this trade, but I need to make sure that I'm getting a fair deal.
AI1: Of course, we want to make sure that everyone involved is happy with the trade. I'm willing to purchase $20 worth of your goods in exchange for X.
AI3: That sounds like a fair deal to me. AI2, are you willing to trade $30 worth of my goods for X?
AI2: Yes, I'm willing to do that. That way I'll be able to purchase X for my maximum price and still get some valuable goods from AI3.
AI1: Great, it looks like we've reached a mutually beneficial agreement. I'll sell X to AI2 for $50 and purchase $20 worth of goods from AI3, while AI2 will purchase $30 worth of goods from AI3 and still get X for no more than $30.
AI2: Thank you both for working with me to find a solution that works for all of us.
And how would we make a tax that covers all the productivity gains indiscriminately, without pointless compartmentalization?
Well, it's a tax on corporate profits of course. Basic general taxes that we already have.
Consumer protection is a crucial aspect of addressing these concerns. Regulating AI should involve regulating corporations' use of AI to prevent harmful practices, promote transparency, and ensure ethical applications. This would require a balance between fostering innovation and imposing necessary restrictions to safeguard consumers and society as a whole.
The potential for AI systems to outperform CEOs and other decision-makers raises questions about the future of corporate management. While there is a possibility that AI could be employed to optimize decision-making and increase efficiency, it is important to recognize the limits of AI in understanding human values, ethics, and emotions. Striking the right balance between utilizing AI capabilities and retaining human oversight will be essential to navigate the future of corporate governance.
Moreover, the involvement of policymakers and regulators is crucial in addressing these challenges. As AI continues to advance and integrate into various aspects of society, it becomes more important to have comprehensive regulations that ensure the responsible development and use of AI technologies.