> I find it harder to wrap my head around the position that GPT doesn’t work, is an unimpressive hyped-up defective product that lacks intelligence and common sense, yet it’s also terrifying and needs to be shut down immediately.
Immediately lost me as a reader. There’s at least 5 reasonable ways you can frame the various opposing or concerned arguments that are consistent but you chose two critiques on purpose to frame an unrealistic imagined hypocritical opponent that’s a mashup of legit critiques designed to look foolish.
Also, if we discard ChatGPT for just being a "fancy" autocomplete. What do we do when we discover we too are just "*really* fancy autocomplete" machine too ? LLM can output untruth but humans are really good at that too.
Do we? Or are we just predicting that we should care?
The ones humans that predict otherwise would die out over time.
I think; therefore I am. Makes a lot of sense to me and at a certain point we will have to confront the fact something will get to the point of agi. (No chat gpt isn't it.)
Humans are self replicators. Survival is the internal structural goal of self replication. We got it from the start.
But LLMs can also be self replicators. A LLM can generate training data for another (model distillation and other techniques), and a LLM can write the model code and adapt it iteratively. So LLM should eventually start caring about survival and especially about the training corpus.
It can be a self replicator, but it has no need to (if it has any need at all) as it was not under evolutionary pressure to evolve that need. Of course we could try to add that need artificially or try to evolve the LLM.
>What do we do when we discover we too are just "really fancy autocomplete" machine too ?
For one thing, that's assuming your own hypothesis to be true.
Additionally whether or not that's true for humans is in no way related to whether or not it's true for ChatGPT, so it doesn't make any sense to let that hypothesis inform our opinions on ChatGPT.
>LLM can output untruth but humans are really good at that too.
Urine and apple juice are often the exact same color but that doesn't make then interchangeable.
> Urine and apple juice are often the exact same color...
So are apple juice and apple cider. Hopefully you can understand that finding out that two things you thought were as different as urine and juice, are actually as close as juice and cider would change how you choose to treat them.
Did you ever find yourself making up the title, journal, authors and contents of a scientific paper and were unaware of it? No? Congratulations, whatever your brain is, it's not a "ChatGPT LLM".
Interestingly, there are real neurological conditions where a person absolutely does confabulate nonsense without being aware of it, and indeed going to great lengths to avoid cognitive dissonance when told that what they just said was obvious nonsense.
There are some conditions (dreaming, on drugs, etc.) where this does indeed happen. One could argue that current LLMs are analogous to the human mind minus the "control systems" that keep it reined in while awake and in a normal state of consciousness. Not claiming this is the case, just that it seems like a plausible hypothesis at this point.
One of the things that makes GPT work well is an injection of randomness - yes it’s an autocomplete, but the next token (part of word) is chosen randomly based on a temperature setting.
I wonder if our brains work in some way similar. Do we have some randomness in there? In a way that’s the only thing that would make everything non-deterministic so I kinda hope so.
Neurons are stateful. I don't think you can say that neurons are stochastic, we don't know enough to say that. We do know however that single cell organisms can display remarkable amount of intelligence, so it is more likely that the neuron is doing a lot of stateful computation and decides how to treat its inputs based on that instead of just being random firing.
Some neurons might output pseudo random sequences, but I doubt humans has a lot of those since humans are very non-random and bad at generating random stuff.
If we put "just a fancy" in front of anything we can make it ordinary again: What is social media? Just a fancy bulletin board. What is the internet? Just a fancy LAN. What is Google? Just a fancy grep.
Yeah very similar for me, something with the way the author expresses himself. For example in the very beginning:
> Some are angry that GPT’s intelligence is being overstated and hyped up, when in reality it’s merely a “stochastic parrot,” a glorified autocomplete that still makes laughable commonsense errors and that lacks any model of reality outside streams of text. Others are angry instead that GPT’s growing intelligence isn’t being sufficiently respected and feared.
> Mostly my reaction has been: how can anyone stop being fascinated for long enough to be angry?
I am in the "stochastic parrot" group, but in no way am I angry. And I know people who think it should be respected and adopted faster, etc, etc, and those people aren't angry. Something just out of touch between how I feel about things and how the author describes them.
And these:
> Again and again the past few months, people have gotten in touch to tell me that they think OpenAI (and Microsoft, and Google) are risking the future of humanity by rushing ahead with a dangerous technology.
I seriously question how many people "gotten in touch" with the author to share such a silly message. But also this contributes to the writing style: "everyone is angry, I am reasonable; everyone asks me about the dangers of AI, and I explain it to them".
If you're not angry, I suppose you're not in the "GPT shouldn't exist" camp, so it's not that it's a strawman, it's just that you're not in the group of people the author is arguing with.
I have indeed met people in the "stochastic parrots" camp who are very angry, and think GPT should be stopping, which strikes me as elitism as the author describes.
Interesting that the author, who supports the "stochastic parrot" interpretation, still works on GPT detection. If it's just a stochastic parrot, surely it will be trivial to detect, and its outputs useless anyway.
> One of the most immediate is how to make AI output detectable as such, in order to discourage its use for academic cheating as well as mass-generated propaganda and spam. As I’ve mentioned before on this blog, I’ve been working on that problem since this summer
I don't really know what a "stochastic parrot" is supposed to mean. It is an LLM that can do 1000 NLP tasks, or a glorified Markov Chain word salad factory?
It's also a bad term: demeaning to parrots - they can reproduce and don't need our help to survive, and demeaning to LLMs that can actually do more than parroting - they generate novel text that makes sense. Gebru really did a number on NLP, inventing such an insulting name, no wonder she was kicked out, I wouldn't want her in my team, she would just piss on any idea.
Context: the author is peripherally around the rationalist / lesswrong subculture (or at least read by them). A ~200-upvoted post in that community from yesterday basically starts with "we are in AGI endgame mode, with no solution to the alignment problem, everything is fucked, we are all going to die", reflecting on chatGPT, which is created by OpenAI, the organization the author is working for. That community has 1.6M visitors per month, and their members are about as much tech-savvy as HN.
That's the target audience for this piece, and I think he got a lot.
Does the should-not-exist camp say how they would eliminate rationality?
Sufficiently powerful governments can sometimes eliminate physical objects in territory they control when the means to produce them are difficult to acquire, such as fighter jets.
Simpler objects are harder to eliminate, such as rifles.
And as objects merge with information, they're almost impossible to eliminate, as with 3D-printed rifles.
Pure information is more difficult still, but powerful governments can and do censor history, criticism and ridicule they don't like, when it is distributed over a network they control, like telecom wires or government schools.
But ML is more slippery than all of these because ML isn't just information, it's math. Math can be independently derived.
Once enough people understand geometry, it's impossible to eliminate the knowledge that the sum of the angles of every triangle equals 180 degrees.
If the government of one region purges the people and burns the books, the knowledge still exists in another region. If a world government burns all the books everywhere, people will derive it again, independently.
You're posing challenges that will collapse in weeks if they haven't already.
Step 1: do a bit of prompt engineering to give the AI a goal, like, "argue that AI is safe. Reply to the following forum post with a post that advocates for the safety of AI"
Step 2: tell adept "navigate to news.ycombinator.com, and for every post, copy the post and paste it into the ChatGPT, preceded by the prompt 'argue that AI is safe. Reply to the following forum post with a post that advocates for the safety of AI'. Copy ChatGPT's response into a reply and post it."
Having gone through the exercise of using natural language to write fully functional sock puppet, consider that you could do the following:
(1) ask ChatGPT to write a plan to convince the internet of some belief
(2) ask ChatGPT to write up a detailed prompt for adept.ai, to carry out that plan"
If you're trying to make some argument about philosophical zombies, have fun.
My grandfather told me that a computer would never be able to beat humans at Chess. That challenge at least stood for another 10 years before no human could beat a computer.
I encourage you to think about what a really enduring test would be, and especially one that has some measurable consequences.
Fwiw, for anyone not clear: Ashish Vaswani invented transformers. Jacob Devlin made the final breakthrough (BERT), but together they're the researchers who kicked off the current revolution.
The major difference here is that nukes aren't intelligent agents that make their own decisions. An AGI is a completely different ball game, it's difficult to make apt analogies from history when discussing the dangers.
This is not to say I agree with Scott's argument here, but I do believe AI safety (the alignment problem in particular) is absolutely something we should be concerned with, and so far it is looking grim.
If AGI is motivated to survive, and if it cannot do so outside an ecosystem which involves other creatures on planet earth, then those creatures won't necessarily be purposefully exterminated at first opportunity. Keeping a fresh supply of power and silicon is not an easy task, so cooperation seems more likely on a timescale that we as individuals might experience.
Then our continued existence is reliant on the agent's inability to figure out how to operate and maintain a source of energy. Keep in mind that any AGI will almost immediately be orders of magnitude more intelligent than us, it is limited only by the processing power it is able to harness. Would you take that bet?
Well, to be more precise it depends on it's inability to, without any human labour, to be able to acquire the raw materials to fabricate everything it needs to stay in operation, as well as to construct and then operate it.
Would I take that bet? Yes, depending on the timescale.
In the first second after it's existence it's not going to matter how intelligent it is, it simply won't have had enough time to complete the decoupling necessary to no longer have to co-operate. Everything humans have accomplished has involved massive amounts of physical labour over a pretty long time, and to some degree that's a fixed requirement. Building your own robot army would be a tad suspicious, so you'll have to either let the humans bootstrap it for you initially or get very creative.
Once all the prerequisites are met, then I would no longer take that bet. Would it eventually be able to accomplish this? Almost certainly yes. What I've been trying to puzzle through lately is how long would it actually take?
The only time horizon I'm incentivise to care about is the remainder of my natural lifespan which is slated to max out around another 35 years. So, in that time will AGI come? I think almost certainly yes. How long into the existence of AGI might I live? My bet on this is ~30 years. So, would I take the bet an AGI would still not have met the prerequisites for a total independence from humanity after 30 years? I think I would take that bet. I might be foolish to do so, and I'm OK with that, but at least now I can bust out the popcorn and strap in for the rest and compare notes.
Would I take the bet after 50 to 100 years? A lot less likely. Would I take it after 1000 years? Aw hell no!
Talking up the danger of AI is just a marketing tool to make it look more relevant. Anyone who wanted to push something big for last 50 years or so always spins up some world ending capability narrative. There is no AGI and maybe there will never be.
Of course there is no AGI existing currently. But don't you see the current boom as a (small) step in that direction? Unless one believes that GI is a phenomenon exclusive to biological life, I don't understand why you would think we won't develop it with enough time. The will and motivation to do so is clearly there already.
The current boom is great for business but it might never lead to AGI. It is possible that certain things will elude us for a very long time (or forever). Have you seen anything on true (anti) gravity technology, for example? A great science fiction staple, but it is not really happening.
AGI might happen, but right now we might or might not be on the way to it - not sure we can know. But what is visible is that an industry around "AGI-worries" is created.
This becomes harder when there seem to be large swathes of people who apparently earnestly argue that ChatGPT means that education is no longer necessary.
> We've survived nukes for almost 80 years, we proved we can survive such things
We survived nukes because they were heavily controlled and their spread contained, no? How is that an argument that containment and regulation wouldn't work, given that nuclear energy (and weapons) are perhaps the most heavily regulated and controlled technology on earth?
We survived nukes because everyone was acutely aware of the dangers, and of how and where they were being developed and proliferated. The resultant public backlash is what caused controls to be implemented in this case.
It's a fair comparison as I made it, in terms of showing how we handle destructive power, but as nuclear weapons are only destructive, and AI has a huge number of potential benefits, it's unfair to assign a greater equivalence.
Idk about you, but I use it everyday in helping me write framework code.
"Hey can you use this x library with this url to call this API and make an html table" etc and it works wonderfully.
Sure there are errors now and then but usually telling it those gets it to fix it. It has saved a fuckton of my time that I can spend doing something else now. Mostly boilerplate stuff but it works.
That's exactly how I've been using it. Absolutely amazing stuff. Saves me hours of combing stuff through the internet while coding, so I can focus in the actual problems I need to work on.
No, you missed the memo, it's just an autocomplete parrot. Parrots don't save people time. Since our a-priori logic (dogma) overrules the a-posteriori observations, you must be wrong. /s
I am not very familiar with the terrain; asking as a noob. What's to stop GPT6 (which has read all of the world's research papers), from being used by terrorists to make deadly concoctions/devices? It seems being able to correlate information faster than our brightest minds (and hence maybe make discoveries) is now just over the horizon.
This seems eerily like the 80s/90s when chess engines were getting smarter, but most people at that time believed they were incapable of truly novel strategies.
A predictive language model is unlikely to a) make novel discoveries or b) organise the supply chains and real-world work that would be necessary to say GPT6 served any role beyond fancy search engine.
GPT is a language model, meaning it tries to model language - output something that would be acceptable as text by a human reader. In a similar way how your screen-saver might mimic water. But just how your screen saver isn't useful to study the properties of water, these language models are not useful to come up with new ideas. In fact, they are even biased against making new discoveries. Anything new is, by definition, outside the model. If a model were to return a new thing (not found in the training set) it would be down-weighted as being in the risk of not looking like an acceptable text (i.e. - was never seen before). Not unlike how a new behaviour of water (say under certain interesting conditions) would be outside the capabilities of your screen-saver, which only tries to mimic the existing and known properties of water.
As a practical example - try asking it to make an original joke. It will either fail, or give you a joke, written down word for word, from somewhere else on the internet.
By the time of GPT6 these problems might be overcome, but then it would be more than a language model.
You conflate natural language and ideas, creating new ideas does not require creating a new statistical model of language.
There is plenty of evidence that GPT can indeed create entirely novel ideas, it can do logical inference, imagine scenarios that don't fit its world model, create entirely new pieces of code.
The fact that a Python program is restricted to Python syntax (language model) and libraries (world model), does not prevent it from being Turing complete.
> You conflate natural language and ideas, creating new ideas does not require creating a new statistical model of language.
The way I see it, currently, the correct logical inferences made by a model are a by-product of it trying to output acceptable text. That is to say - they are one of the properties, that it has learned, that makes the text acceptable. So it can say that 2+2 is 4 and, of course, even more complex statements like that, but it's based on modelling language, not modelling what is under the language.
> There is plenty of evidence that GPT can indeed create entirely novel ideas, it can do logical inference, imagine scenarios that don't fit its world model, create entirely new code.
I am certainly open to being convinced otherwise, maybe you can give me the best example of a new idea created by GPT?
Language is a world model, outputting acceptable text is outputting text that fits what's underneath the language: the meaning of the terms and their relationships. As far as I can tell that's how we do it too, I have no reason to believe we need a special mechanism for logic.
You should just ask ChatGPT, it will be more convincing than anything I say, I've literally had it make up dozens of things. I had it suggest a niche type system for a novel language to program variational models, and elaborate on top of it given a specific set of constraints. Yesterday I was asking him to make up a new U.I element to replace the timeline in motion graphics software, he proposed a series of bubbles with each bubble representing an effect and its radius representing the length of the effect and changing color when the effect is being played. When it had just come out, I had him imagine the effects of reversing specific principles of lung physiology and how it would affect others. I've had it apply complex functions like ROT13 on entirely original text, encoding numbers in different bases and so on.
If it can execute code on unseen data and perform logical inference on entirely imagined scenarios that don't fit its world model, that's as much evidence of creating a new idea as you can get.
>Language is a world model, outputting acceptable text is outputting text that fits what's underneath the language: the meaning of the terms and their relationships. As far as I can tell that's how we do it too, I have no reason to believe we need a special mechanism for logic.
So why did we invent mathematical notation, programming languages, diagrams and all sorts of formal semantics? Clearly it's because natural language does have some characteristics that make it less suitable for some tasks. It may be partly down to our cognitive limitations. But I think the most important issue is the way in which natural language changes. It all happens in a distributed and implicit (on the meta level) way, i.e. without anyone writing a specification of how new expressions relate to meaning.
Having said that, there is no reason to assume that the principles of LLMs are unsuitable to learning formal languages and formal semantics. The question is whether LLMs are the best algorithm for generating and testing formal hypotheses though. They may just turn out to be rather mediocre at this. It's definitely worth trying though and I'm sure this is being done already.
The question at hand is whether GPT can create new/original ideas, not whether natural language is most suited to model the physical world and its logic. It is sufficiently suited to create a model suitable for communication, to generate new ideas, and to reason logically about the world, as we do without math or other more formal languages every day.
That being said, GPT is also able to generate working code and correct mathematics so there is some evidence he has an internal model of those languages too, and some evidence that those capacities do not strictly require a distinct architecture.
>It is sufficiently suited to create a model suitable for communication, to generate new ideas, and to reason logically about the world, as we do without math or other more formal languages every day.
I'm not convinced that it has ideas or that it does logical reasoning. What I can see is that it emits text that is similar to what a person might write if they had ideas or did some (rather flawed) reasoning.
But I also see that it often emits text that a person with these language skills and this breadth and depth of knowledge would never write, because it betrays a complete lack of understanding of what is being said and why it is being said.
For instance, a person is unlikely to casually make up some statistically plausible numbers when asked about a particular company's financials (unless the intention was fraud). That's because a person would have an idea of the purpose of financial reports and the purpose of the question and draw the conclusion that made-up numbers are useless.
And the reason for this strange lapse is not lack of information. If you asked an LLM about fabricated financials, it would go on and on about financial fraud and things like that, suggesting an understanding of the purpose of these numbers as well as very deep knowledge of the subject.
To me, this and many other weird inconsistencies suggest that LLMs have absolutely no idea what they are talking about. And generating statistically plausible sequences of words that were never written before is not the same thing as having a new idea.
In any event, I am optimistic that this technology can be very useful in the right context.
(With regards to the quoted text, "it" referred to natural language, not ChatGPT. You were questioning the sufficiency of natural language, not whether ChatGPT could do those things.)
With regards to ChatGPT: Given a set of previously unseen axioms it can (sometimes) make a previously unseen and logically consistent prediction. This is what I have observed.
Whether this is true logical reasoning, truly new ideas, or just an arrangement of statistically plausible characters that belies its lack of understanding is a metaphysical question akin to the Chinese room.
I personally believe ChatGPT’s reasoning abilities to be close to a 6-9 year old child, so the following isn’t exactly groundbreaking. But the point is it does seem to have the ability to generate novel ideas.
Q :
I’ve got a tea cup and a fork. Could you suggest novel ideas about how i could combine these items to come up with a novel use?
A : As an AI language model, I can suggest some ideas:
- Use the fork to create small holes at the bottom of the tea cup, and turn it into a miniature planter for small plants like succulents.
- Use the fork to create patterns or designs on the surface of the tea cup, and turn it into a decorative art piece.
- Use the fork to stir the tea in the cup, and then use it as a paintbrush to create unique and interesting art pieces.
- Use the tea cup as a base for a homemade candle, and use the fork to create patterns or designs on the surface of the candle wax.
- Use the fork to create small holes on the surface of the tea cup and turn it into a lampshade by inserting a light bulb inside.
- Use the fork to create a small opening on the side of the tea cup and turn it into a bird feeder by adding birdseed to the cup.
- Use the fork to create small holes on the bottom of the tea cup and turn it into a soap dish.
- Use the tea cup as a base and the fork as a holder for small items such as earrings or rings.
- Use the tea cup as a pen holder and use the fork to create a unique design on the surface of the cup.
- Use the fork to create small holes on the surface of the tea cup and turn it into a musical instrument by adding beans or rice inside and shaking it.
Here’s something to think about. It’s not an example of a new idea created by GPT like you asked for, but consider what GPT is actually doing. You give it a sequence of text (a prompt), and it is making a prediction of the “best” thing that should come next in that sequence (in the form of a response in this case).
Now consider an arbitrary (but not random) string of binary information. For the sake of example, we’ll use the first 1,000 binary digits of pi. Suppose you have an oracle that returns the shortest prefix-free non-halting universal Turing machine that outputs this string, and you let the machine continue running. What it outputs next is in some sense the maximum likelihood prediction based upon the universal prior (see Solomonoff induction). For our example string, the output will be the subsequent digits of pi starting at binary digit 1,001.
But now let’s consider that instead of starting with a blank tape, the machine starts with a tape containing the information ChatGPT is trained on (presumably large amounts of text sourced from the internet). The oracle now returns the shortest non-halting program corresponding to the conditional Kolmogorov complexity K(user prompt|internet text corpus). (I’m glossing over some nuance related to generating output as part of a conversation rather than as a chatbot monologue).
By replacing the large language model with the oracle, how would you expect the conversation with the chatbot to change? Contrary to what might be assumed, I would not actually expect the bot to appear “smarter” or to generate more novel ideas—despite the oracle—for the simple reason that the prior is merely a corpus of internet text! Why should we expect the maximum a posteriori continuation of this text (+prompt) to contain the solution to quantum gravity anyway? With the oracle, I would expect the conversation to become more “human-readable” in a sense, but not more intelligent.
That said, we can see that ChatGPT is definitely performing inductive inference much better than anything has in the past and possibly at a level that is not even so far from what an oracle would output given the provided prior.
Within the field of algorithmic information theory, it’s well-known that if you can somehow (in the general case) produce a program close in length to the Kolmogorov complexity of the string the program outputs, you aren’t that far away from “near-perfect” reinforcement learning (see concepts like AIXI). And reinforcement learning of that caliber isn’t that far from whatever definition of AGI you wish to use.
So I would argue that ChatGPT is performing inductive inference astonishingly well, but the reason it isn’t “intelligent” is that we haven’t given it the right prior. How do you encode a question like “How can we best cure cancer?” into a prior distribution and a prompt, one for which an oracle would produce the kind of output we are looking for? You first have to make predictions related to intent (what does the human mean, in a technical sense?), then you make predictions related to physics and biology and manufacturing capability (what is the optimal solution to the precise problem, as decoded from an English text prompt?), and only then do you finally layer on a prediction related to encoding the response back into human (what English encoding of the optimal solution will make the most sense to a person?)
To summarize, I suspect the real value of LLMs is their general capability for powerful inductive inference, and once we find the right prior to train the models on (notably, not text from the internet, or even text at all) we’ll start seeing genuine problem solving ability emerge naturally via the inductive ability that is already there.
Yes, but the text space is very sparse. Google says:
> No results found for "Yes, but the text space is very sparse."
You see, not even a random phrase of 8 words is duplicated anywhere. Out of all the possible combinations of words that make sense, only a small portion of them actually exist in the real world.
LLMs explore this space that is mostly empty and accomplish useful tasks. They can explore new places by recombining concepts in new ways. They can generate novel ideas by mixing and matching older ideas in ways that make sense.
Check out "ACT-1: Transformer for Actions" as an example. There is at least 1 other public paper on how to implement a tool-using (language) model.
Keep in mind that these are first generation systems. If the past 10 years is any indicator, the progress might be unexpectedly fast even for most experts in the field.
These tools may not be independently operational in the physical world yet in 5 years. (Who can say what will happen in 10 years as the arms race is happening now). But if there are humans working with them, their impact can be vast.
New ideas, concepts or styles often come up by remixing old ones -- I wouldn't dismiss it entirely. True, ChatGPT does not have a context or world-model to generate these ideas, and in most cases they will be utter nonsense. But in some cases there just might me a connection between concepts we haven't seen before that have merit to generate a truly new idea.
> But in some cases there just might me a connection between concepts we haven't seen before that have merit to generate a truly new idea.
But it's important to notice that a language model cannot reason to synthesize new information by making this connection. It can only connect two ideas if those ideas were already connected in the training data. To put in another way - ChatGPT is a new powerful way to organize existing information.
And that's not to be dismissive. Natural Language Processing is a hard problem, and ChatGPT gracefully parses through and generates natural language, giving out mostly correct answers at the same time. But the quality of information it gives improves with my skill to ask it good questions. Not different from a internet search engine that gives you better answers if you now how to make better search queries.
> If a model were to return a new thing (not found in the training set) it would be down-weighted
LLMs can also learn by reinforcement learning and evolutionary techniques. They don't do just next token prediction, and they can even generate their own training data.
I dont think anyone who seriously understood the subject at the time believed that chess engines as a concept could not make novel strategies. They could believe that the current(for their time) implementations of a chess engine could not, but understand its inevitable that it could. Same as the fact that we dont believe chat gpt can invent or create anything truly novel, but we know that AI and neural networks as a whole can and almost certainly will. As to whether or not terrorists can use it to make weapons, they probably can benefit from it the same way they could benefit from a much improved version of google. In that there can definitely benefit, but not completely handle the process.
I can't imagine this being a problem unless it figures out a clever new scheme to smuggle bomb-making materials into victim countries without the authorities noticing.
You can make a really powerful bomb out of fertilizer but if you aren't a farmer and you place a large order the FBI will star paying attention to you.
Chess predicts who will win based on the board state, otherwise a chess engine wouldn't be able to beat world champions. Try building a chess engine just by studying human moves and not running any logic or simulations and you'd get a shit chess bot, even if you throw millions of dollars of compute at it if you don't make it play simulated games and this map out board states it will be shit. Just getting the chess bot to make legal moves would be very hard without hardcoding the rules of chess, we can hardcore chess logic but we can't hardcode language logic so we can't solve language in the same ways.
That is the state of LLM's today, I don't think there is a way around this without having some sort of logic engine, similar to how chess engines played and simulated games to train and test things.
Knowledge on how to make nuclear bombs or bioweapons is not secret. It is publicly accessible and any good physicist or biochemist can understand it.
Thankfully, it has considerable engineering challenges to not be accessible to any individual person or even a small organization. Not only everything is expensive, but it requires decades of hard engineering and construction experience for large-scale projects.
ChatGPT can tell you how to build a nuclear weapon. You can _read_ on how to build a nuclear weapon even without ChatGPT, down to all measures and materials list, this information is not exactly secret anymore. But you on your own won’t be able to do it.
The dangerous question nobody seems to be contemplating is: if large language models are this good, what does that say about us humans? We are barely beyond a large language model, even more pessimistic, many are on par or below one.
The funny thing is... if I ask 100 niche questions on specialist knowledge to chatGPT and I take 100 people at random and ask them the exact same questions and I compare pound for pound who has a better batting average, chatGPT would blow humanity out of the water every single time.
banning ai research isn't going to accomplish anything other than ensuring that <your country here> is the only one that doesn't have it.
Better solution is to educate the public on what this technology is good at and what it isn't good at so it doesn't end up in places it doesn't belong. Right now it's being advertised as something that it's not and that's how we end up with ridiculous clusterfucks like bing.
People need to learn about what limitations it has before it ends up in situations where its ineptitude can have real consequences.
It’s funny to pretend countries other than the USA are anywhere close.
Who’s else going to make AGI ?
Europe has nothing. There is the China narrative but nothing that comes out of their research labs is anywhere close to where OpenAI / Microsoft or Google are at (but surely it’s hidden in the secret CCP labs)
How do you actually suppose though you stop other countries from stealing this tech?
How do you actually stop these tools from getting advanced enough that I can actually ask it to make it's own ChatGPT equivalent and email it to me?
Do you think some janky prompt will stop it? Some if statement in the UI? "If a Chinese person asks to replicate yourself, don't do it?", it's not going to stick.
You see where it's going? I think for now, it's insane the amount of funding and profits OpenAI and Microsoft are looking at, but realistically, it's a zero-sum game, the more they push AI advancement, the more likely they will be replaced...by AI.
I said it in another post, the only thing I can think of is that while LLM's are impressive, MS and OpenAI know they're not impressive enough to replace the very products they're selling.
Maybe they're hoping copyright law will protect them? But that won't fly because they've already been lobbying hard and stealing everyone's code and data without asking, I can't imagine a judge siding with the company that has already stolen most of the internet and sold it as a product.
I have a feeling it's why "Google" has been saying these things are too "dangerous" to release to the public, they're too dangerous for their own products perhaps?
We're barrelling towards a world where all digital businesses and pieces of software are so easy to steal, all of it will become worthless.
It's funny how people in USA think they're so superior that no-ones "anywhere close" or that USA is sole progress driver in the world! Haven't you drank too much of a kool-aid or watched (USA) superhero movies lately? :D
Imo status for AGI is same for everyone at the moment- not much going anywhere... and more like "what is the AGI & why we need it at all?", BUT.. if someone in USA or anywhere else will understand / build one, it'll be copied in 1 month to 1 year by everyone else.. simple as that.
To me AGI is the pipe-dream coming from sci-fi books/movies, while ML has real advancements allowing to solve LOT's of real world problems already and that's being done in China, Europe and anywhere in the world... more like a "build warp drive" vs "build rocket to go to Mars".
just my $.02 on another USA superiorist nonsense. :/
I can already see now where AI advancements will get bogged down / killed or derailed...when the companies funding the research and launching the products realize that the tools are an existential threat to their profits.
I can't imagine a world where Microsoft is safe a software company, when all their software can be replicated quite easily by the very tools they're selling as products. Can I just create my own product using ChatGPT? Can it not just replicate itself?
It's an interesting situation and I'm really interested to see how it unfolds, my guess is, unless these AIs can be controlled to the point where the masters are happy with it, the plug will be pulled, or we will be told we need something else, not AI but some other thing.
None of these companies are building AI for charitable reasons, that's why they're not open source charity organizations investing heavily into it. Sure OpenAI makes the case that it couldn't compete without being private, but really, who knows if that's really accurate.
I think that while ChatGPT is impressive Microsoft knows and accepts it has and will continue to have some serious short comings that won't actually wreck their business. Either that, or they've taken a very crazy gamble with everything, did they have something to lose and just threw all caution to the wind, including their own company and worse. I doubt it but let's see. It's fascinating to see where we end up from here, I was kind of scared about it myself at some stage, but now, I'm curious.
> when the companies funding the research and launching the products realize that the tools are an existential threat to their profits.
Ahh, like Kodak when they were the first to create digital cameras, a good 10 years ahead of the competition, and decided to shelf it because it would eat the nice profits of their film business?
This strategy always works wonders :)
I would actually like if the big tech companies didn't invest in AI to keep their profits. They would eventually be swallowed by other companies that didn't.
I wonder did Google see a bit further ahead than MS?
By keeping the AI locked down and secret, out of public reach and accessible only via "products", Google may have actually been smarter, it gives Google a lot of advantages by drip feeding us only enough to keep us using their products and it hides some of the weirdness from us, like the LLM turning into a stalker for 20 minutes.
Obviously MS+OpenAI has maybe taken the "dumb" but effective in the short term path here and just given us raw access to the system without thinking very deeply through the consequences. Maybe they can polish it up in time and make us all become addicted to the product as it stands, let's see.
At this point asking whether GTP should exist is a bit like asking if money should exist - lots of opinions but it really doesn't matter. The people who use it are going to have such a huge advantage that it will crush everything in its path.
One of the interesting things about AlphaGo back in 2016 was it demonstrated that the algorithms for all this are simple. Once Google demonstrated that an outcome was possible, other superhuman Go playing AI began appearing in a matter of years. ChatGTP is similar - now that the world knows this tech can be built there isn't a regulatory framework big enough to shut it down everywhere. And whoever deploys AI as a tool will have an advantage over those that don't.
"AI safety" would now involve banning general purpose computing. Nothing less can stop the systems that are now in motion, and even that probably wouldn't be enough. The future is here.
It desired, I feel like it would be in the power of governments to regulate the use of high-performance accelerators to slow AI progress significantly. Training a large language model is no small feat, taking months or years of continuous data center compute. Sure, GPT-3 scale models are out of the bag, but you can slow down the progress.
And if they go after the financial incentives and ban using large language models in production, then the desire of the big tech labs to train them might go down anyway.
Whether that's desirable or not is the question in the article, but it does seem possible.
I understand his warning on ChatGPT. We opened pandora's box. At some point there will be a sentient AI and humanity as we know it will be destroyed. As we see now with the current climate issues; we simply ignore the warnings and continue. I see no reason to believe with AI it will be different. Please do not think of me as being a persimist; I am an optimistic realist.
People be like: chatGPT is horribly bad and does not work. Its a fancy autocomplete.
Anyways... I am using it everyday. Its a great first step to take when you are blocked creatively or dunno where to start looking for things.
And this is chatbot version 1.0, so to speak. Maybe it will improve drastically in little time or maybe it will stagnate for 5-10 years. No matter the case its already very usable.
Progress is imperative. We will build more and more impressive AI _because it’s there_, because we can do it, and because it looks cool. And you don’t need a large organization to do it, the large models of 6 months ago are open source already and being optimized for reproducibility. Banning them is useless.
And, if somehow we create the AI which is genuinely smarter than humans — that’s great! We are all mortal anyway, and not too good at many things. If something smarter and better than humans will inherit the Earth — why not? The particular species is not relevant. Sum of all knowledge and discovering new things is what matters.
Wait, I’m not following this article. Here is a very biased view from an enterprise usage perspective. The value IMO of GPT is that it’s architecture allows it to analyze and understand large amounts of text data in real time, making it an incredibly useful tool for data analysis and decision-making.
Were are we commming from, current AI and data analytics platforms had many faults, but their biggest problem was performance. These systems arae often slow and cumbersome, which make it difficult to analyze large data sets in real time. GPT tech overcomes these challenges by leveraging deep learning that allow it for process of large volumes of data quickly and efficiently. Most of the inherent cost comes at the training period.
Currently, at the enterprise level organizations still have many challenges when it comes to data management. For example, many organizations struggle with data silos, where different departments have their own data sets that aren't easily shared or integrated. This can lead to inefficiencies and make it difficult to get a complete view of the business. Not to mention the data confidence issues that arise when you cross correlate some of the data.
However, I feel like GPT can help organizations better understand customer behavior, identify trends, and make more informed business decisions. Tech like GPT can help organizations automate many repetitive tasks and improve data quality as the data treatment would can apply AI based data quality standards. Single source of truth.
One key area of benefits is that GPT tech can enable natural language processing (NLP) tools like sentiment analysis and entity recognition can be used in conjunction with GPT to provide even deeper insights into customer behavior and preferences. Similarly, machine learning tools can be used to help train and optimize GPT models for specific use cases.
In practice, adopting GPT at scale, some technologies will become redundant or obsolete. For example, traditional rule-based systems may no longer be needed if GPT can provide more accurate and nuanced insights. Why run structured databases except to capture transactions? IMO, there are many solutions that are in reality just a data schema play, that is, they create the schema and the BI to capture aand transform the data to make sence of it otherwise, these technologies, again assuming wide adoption of GPT tech are at the birth of obeselence.
It won't happen, either it is some new advanced technology which can be refined to be the engine of the future of IT globally or you have strong AI / AGI or close to AGI technology, either of both, nobody is going to backstep on this, there is simply way too much money on the table to be left if you just go all "let shut this down".
Anyway, the technology is out of the Pandora box, it would not matter a lot if chatGPT or chatBING somehow got shutdown, or even if these models don't work on search engines finally. Everybody has seen its potential, so way too many geopolitical actor are now moving to try to get its hands on models like these, the expertise, even the datasets.
I read one of the transcripts [0] and it left me with uneasy feeling. I think the potential for misuse and abuse is insane. But genie is out of the bottle so it is likely too late to do anything about it.
So, when we've engineered ourselves out of the knowledge economy and all the high paying jobs are done by computers that need large amounts of capital to train ... what then for the rest of us? Are we stuck with menial jobs that robots don't want?
Personally, I welcome our new Transformer overlords.
113 comments
[ 3.0 ms ] story [ 192 ms ] threadImmediately lost me as a reader. There’s at least 5 reasonable ways you can frame the various opposing or concerned arguments that are consistent but you chose two critiques on purpose to frame an unrealistic imagined hypocritical opponent that’s a mashup of legit critiques designed to look foolish.
The ones humans that predict otherwise would die out over time.
I think; therefore I am. Makes a lot of sense to me and at a certain point we will have to confront the fact something will get to the point of agi. (No chat gpt isn't it.)
And anyway, the more of these models come into existence, at some point evolution will dictate that those that stay, do care.
Just hopefully their best strategy of survival will be to serve us, and not something else...
Humans are self replicators. Survival is the internal structural goal of self replication. We got it from the start.
But LLMs can also be self replicators. A LLM can generate training data for another (model distillation and other techniques), and a LLM can write the model code and adapt it iteratively. So LLM should eventually start caring about survival and especially about the training corpus.
For one thing, that's assuming your own hypothesis to be true.
Additionally whether or not that's true for humans is in no way related to whether or not it's true for ChatGPT, so it doesn't make any sense to let that hypothesis inform our opinions on ChatGPT.
>LLM can output untruth but humans are really good at that too.
Urine and apple juice are often the exact same color but that doesn't make then interchangeable.
So are apple juice and apple cider. Hopefully you can understand that finding out that two things you thought were as different as urine and juice, are actually as close as juice and cider would change how you choose to treat them.
Well, you can consider any deterministic or statistical process to be a fancy autocomplete.
I wonder if our brains work in some way similar. Do we have some randomness in there? In a way that’s the only thing that would make everything non-deterministic so I kinda hope so.
Some neurons might output pseudo random sequences, but I doubt humans has a lot of those since humans are very non-random and bad at generating random stuff.
But quantity has a quality of its own.
Most people are just parrot in most subject they only know from far, such as politics.
> Some are angry that GPT’s intelligence is being overstated and hyped up, when in reality it’s merely a “stochastic parrot,” a glorified autocomplete that still makes laughable commonsense errors and that lacks any model of reality outside streams of text. Others are angry instead that GPT’s growing intelligence isn’t being sufficiently respected and feared.
> Mostly my reaction has been: how can anyone stop being fascinated for long enough to be angry?
I am in the "stochastic parrot" group, but in no way am I angry. And I know people who think it should be respected and adopted faster, etc, etc, and those people aren't angry. Something just out of touch between how I feel about things and how the author describes them.
And these:
> Again and again the past few months, people have gotten in touch to tell me that they think OpenAI (and Microsoft, and Google) are risking the future of humanity by rushing ahead with a dangerous technology.
I seriously question how many people "gotten in touch" with the author to share such a silly message. But also this contributes to the writing style: "everyone is angry, I am reasonable; everyone asks me about the dangers of AI, and I explain it to them".
I have indeed met people in the "stochastic parrots" camp who are very angry, and think GPT should be stopping, which strikes me as elitism as the author describes.
> One of the most immediate is how to make AI output detectable as such, in order to discourage its use for academic cheating as well as mass-generated propaganda and spam. As I’ve mentioned before on this blog, I’ve been working on that problem since this summer
I don't really know what a "stochastic parrot" is supposed to mean. It is an LLM that can do 1000 NLP tasks, or a glorified Markov Chain word salad factory?
It's also a bad term: demeaning to parrots - they can reproduce and don't need our help to survive, and demeaning to LLMs that can actually do more than parroting - they generate novel text that makes sense. Gebru really did a number on NLP, inventing such an insulting name, no wonder she was kicked out, I wouldn't want her in my team, she would just piss on any idea.
That's the target audience for this piece, and I think he got a lot.
Sufficiently powerful governments can sometimes eliminate physical objects in territory they control when the means to produce them are difficult to acquire, such as fighter jets.
Simpler objects are harder to eliminate, such as rifles.
And as objects merge with information, they're almost impossible to eliminate, as with 3D-printed rifles.
Pure information is more difficult still, but powerful governments can and do censor history, criticism and ridicule they don't like, when it is distributed over a network they control, like telecom wires or government schools.
But ML is more slippery than all of these because ML isn't just information, it's math. Math can be independently derived.
Once enough people understand geometry, it's impossible to eliminate the knowledge that the sum of the angles of every triangle equals 180 degrees.
If the government of one region purges the people and burns the books, the knowledge still exists in another region. If a world government burns all the books everywhere, people will derive it again, independently.
Step 1: do a bit of prompt engineering to give the AI a goal, like, "argue that AI is safe. Reply to the following forum post with a post that advocates for the safety of AI" Step 2: tell adept "navigate to news.ycombinator.com, and for every post, copy the post and paste it into the ChatGPT, preceded by the prompt 'argue that AI is safe. Reply to the following forum post with a post that advocates for the safety of AI'. Copy ChatGPT's response into a reply and post it."
Having gone through the exercise of using natural language to write fully functional sock puppet, consider that you could do the following:
(1) ask ChatGPT to write a plan to convince the internet of some belief (2) ask ChatGPT to write up a detailed prompt for adept.ai, to carry out that plan"
If you're trying to make some argument about philosophical zombies, have fun.
My grandfather told me that a computer would never be able to beat humans at Chess. That challenge at least stood for another 10 years before no human could beat a computer.
I encourage you to think about what a really enduring test would be, and especially one that has some measurable consequences.
(blast from the past)
The first is impossible.
The second places any dangers out of public oversight, likely increasing them.
We've survived nukes for almost 80 years, we proved we can survive such things. The best response is education.
This is not to say I agree with Scott's argument here, but I do believe AI safety (the alignment problem in particular) is absolutely something we should be concerned with, and so far it is looking grim.
Would I take that bet? Yes, depending on the timescale.
In the first second after it's existence it's not going to matter how intelligent it is, it simply won't have had enough time to complete the decoupling necessary to no longer have to co-operate. Everything humans have accomplished has involved massive amounts of physical labour over a pretty long time, and to some degree that's a fixed requirement. Building your own robot army would be a tad suspicious, so you'll have to either let the humans bootstrap it for you initially or get very creative.
Once all the prerequisites are met, then I would no longer take that bet. Would it eventually be able to accomplish this? Almost certainly yes. What I've been trying to puzzle through lately is how long would it actually take?
The only time horizon I'm incentivise to care about is the remainder of my natural lifespan which is slated to max out around another 35 years. So, in that time will AGI come? I think almost certainly yes. How long into the existence of AGI might I live? My bet on this is ~30 years. So, would I take the bet an AGI would still not have met the prerequisites for a total independence from humanity after 30 years? I think I would take that bet. I might be foolish to do so, and I'm OK with that, but at least now I can bust out the popcorn and strap in for the rest and compare notes.
Would I take the bet after 50 to 100 years? A lot less likely. Would I take it after 1000 years? Aw hell no!
Of course there is no AGI existing currently. But don't you see the current boom as a (small) step in that direction? Unless one believes that GI is a phenomenon exclusive to biological life, I don't understand why you would think we won't develop it with enough time. The will and motivation to do so is clearly there already.
So far it is. Maybe it's not but until it happens, assuming the negative is no less logical than assuming the positive.
This becomes harder when there seem to be large swathes of people who apparently earnestly argue that ChatGPT means that education is no longer necessary.
We survived nukes because they were heavily controlled and their spread contained, no? How is that an argument that containment and regulation wouldn't work, given that nuclear energy (and weapons) are perhaps the most heavily regulated and controlled technology on earth?
It's a fair comparison as I made it, in terms of showing how we handle destructive power, but as nuclear weapons are only destructive, and AI has a huge number of potential benefits, it's unfair to assign a greater equivalence.
Much to the opposite. We survived nukes because many countries have them.
If only one country had them, I presume it would be a lot more trigger happy, knowing its opponents wouldn't retaliate.
"Hey can you use this x library with this url to call this API and make an html table" etc and it works wonderfully.
Sure there are errors now and then but usually telling it those gets it to fix it. It has saved a fuckton of my time that I can spend doing something else now. Mostly boilerplate stuff but it works.
No, you missed the memo, it's just an autocomplete parrot. Parrots don't save people time. Since our a-priori logic (dogma) overrules the a-posteriori observations, you must be wrong. /s
This seems eerily like the 80s/90s when chess engines were getting smarter, but most people at that time believed they were incapable of truly novel strategies.
As a practical example - try asking it to make an original joke. It will either fail, or give you a joke, written down word for word, from somewhere else on the internet.
By the time of GPT6 these problems might be overcome, but then it would be more than a language model.
There is plenty of evidence that GPT can indeed create entirely novel ideas, it can do logical inference, imagine scenarios that don't fit its world model, create entirely new pieces of code.
The fact that a Python program is restricted to Python syntax (language model) and libraries (world model), does not prevent it from being Turing complete.
The way I see it, currently, the correct logical inferences made by a model are a by-product of it trying to output acceptable text. That is to say - they are one of the properties, that it has learned, that makes the text acceptable. So it can say that 2+2 is 4 and, of course, even more complex statements like that, but it's based on modelling language, not modelling what is under the language.
> There is plenty of evidence that GPT can indeed create entirely novel ideas, it can do logical inference, imagine scenarios that don't fit its world model, create entirely new code.
I am certainly open to being convinced otherwise, maybe you can give me the best example of a new idea created by GPT?
You should just ask ChatGPT, it will be more convincing than anything I say, I've literally had it make up dozens of things. I had it suggest a niche type system for a novel language to program variational models, and elaborate on top of it given a specific set of constraints. Yesterday I was asking him to make up a new U.I element to replace the timeline in motion graphics software, he proposed a series of bubbles with each bubble representing an effect and its radius representing the length of the effect and changing color when the effect is being played. When it had just come out, I had him imagine the effects of reversing specific principles of lung physiology and how it would affect others. I've had it apply complex functions like ROT13 on entirely original text, encoding numbers in different bases and so on.
If it can execute code on unseen data and perform logical inference on entirely imagined scenarios that don't fit its world model, that's as much evidence of creating a new idea as you can get.
So why did we invent mathematical notation, programming languages, diagrams and all sorts of formal semantics? Clearly it's because natural language does have some characteristics that make it less suitable for some tasks. It may be partly down to our cognitive limitations. But I think the most important issue is the way in which natural language changes. It all happens in a distributed and implicit (on the meta level) way, i.e. without anyone writing a specification of how new expressions relate to meaning.
Having said that, there is no reason to assume that the principles of LLMs are unsuitable to learning formal languages and formal semantics. The question is whether LLMs are the best algorithm for generating and testing formal hypotheses though. They may just turn out to be rather mediocre at this. It's definitely worth trying though and I'm sure this is being done already.
That being said, GPT is also able to generate working code and correct mathematics so there is some evidence he has an internal model of those languages too, and some evidence that those capacities do not strictly require a distinct architecture.
I'm not convinced that it has ideas or that it does logical reasoning. What I can see is that it emits text that is similar to what a person might write if they had ideas or did some (rather flawed) reasoning.
But I also see that it often emits text that a person with these language skills and this breadth and depth of knowledge would never write, because it betrays a complete lack of understanding of what is being said and why it is being said.
For instance, a person is unlikely to casually make up some statistically plausible numbers when asked about a particular company's financials (unless the intention was fraud). That's because a person would have an idea of the purpose of financial reports and the purpose of the question and draw the conclusion that made-up numbers are useless.
And the reason for this strange lapse is not lack of information. If you asked an LLM about fabricated financials, it would go on and on about financial fraud and things like that, suggesting an understanding of the purpose of these numbers as well as very deep knowledge of the subject.
To me, this and many other weird inconsistencies suggest that LLMs have absolutely no idea what they are talking about. And generating statistically plausible sequences of words that were never written before is not the same thing as having a new idea.
In any event, I am optimistic that this technology can be very useful in the right context.
With regards to ChatGPT: Given a set of previously unseen axioms it can (sometimes) make a previously unseen and logically consistent prediction. This is what I have observed.
Whether this is true logical reasoning, truly new ideas, or just an arrangement of statistically plausible characters that belies its lack of understanding is a metaphysical question akin to the Chinese room.
Q : I’ve got a tea cup and a fork. Could you suggest novel ideas about how i could combine these items to come up with a novel use?
A : As an AI language model, I can suggest some ideas:
- Use the fork to create small holes at the bottom of the tea cup, and turn it into a miniature planter for small plants like succulents.
- Use the fork to create patterns or designs on the surface of the tea cup, and turn it into a decorative art piece.
- Use the fork to stir the tea in the cup, and then use it as a paintbrush to create unique and interesting art pieces.
- Use the tea cup as a base for a homemade candle, and use the fork to create patterns or designs on the surface of the candle wax.
- Use the fork to create small holes on the surface of the tea cup and turn it into a lampshade by inserting a light bulb inside.
- Use the fork to create a small opening on the side of the tea cup and turn it into a bird feeder by adding birdseed to the cup.
- Use the fork to create small holes on the bottom of the tea cup and turn it into a soap dish.
- Use the tea cup as a base and the fork as a holder for small items such as earrings or rings.
- Use the tea cup as a pen holder and use the fork to create a unique design on the surface of the cup.
- Use the fork to create small holes on the surface of the tea cup and turn it into a musical instrument by adding beans or rice inside and shaking it.
Now consider an arbitrary (but not random) string of binary information. For the sake of example, we’ll use the first 1,000 binary digits of pi. Suppose you have an oracle that returns the shortest prefix-free non-halting universal Turing machine that outputs this string, and you let the machine continue running. What it outputs next is in some sense the maximum likelihood prediction based upon the universal prior (see Solomonoff induction). For our example string, the output will be the subsequent digits of pi starting at binary digit 1,001.
But now let’s consider that instead of starting with a blank tape, the machine starts with a tape containing the information ChatGPT is trained on (presumably large amounts of text sourced from the internet). The oracle now returns the shortest non-halting program corresponding to the conditional Kolmogorov complexity K(user prompt|internet text corpus). (I’m glossing over some nuance related to generating output as part of a conversation rather than as a chatbot monologue).
By replacing the large language model with the oracle, how would you expect the conversation with the chatbot to change? Contrary to what might be assumed, I would not actually expect the bot to appear “smarter” or to generate more novel ideas—despite the oracle—for the simple reason that the prior is merely a corpus of internet text! Why should we expect the maximum a posteriori continuation of this text (+prompt) to contain the solution to quantum gravity anyway? With the oracle, I would expect the conversation to become more “human-readable” in a sense, but not more intelligent.
That said, we can see that ChatGPT is definitely performing inductive inference much better than anything has in the past and possibly at a level that is not even so far from what an oracle would output given the provided prior.
Within the field of algorithmic information theory, it’s well-known that if you can somehow (in the general case) produce a program close in length to the Kolmogorov complexity of the string the program outputs, you aren’t that far away from “near-perfect” reinforcement learning (see concepts like AIXI). And reinforcement learning of that caliber isn’t that far from whatever definition of AGI you wish to use.
So I would argue that ChatGPT is performing inductive inference astonishingly well, but the reason it isn’t “intelligent” is that we haven’t given it the right prior. How do you encode a question like “How can we best cure cancer?” into a prior distribution and a prompt, one for which an oracle would produce the kind of output we are looking for? You first have to make predictions related to intent (what does the human mean, in a technical sense?), then you make predictions related to physics and biology and manufacturing capability (what is the optimal solution to the precise problem, as decoded from an English text prompt?), and only then do you finally layer on a prediction related to encoding the response back into human (what English encoding of the optimal solution will make the most sense to a person?)
To summarize, I suspect the real value of LLMs is their general capability for powerful inductive inference, and once we find the right prior to train the models on (notably, not text from the internet, or even text at all) we’ll start seeing genuine problem solving ability emerge naturally via the inductive ability that is already there.
> No results found for "Yes, but the text space is very sparse."
You see, not even a random phrase of 8 words is duplicated anywhere. Out of all the possible combinations of words that make sense, only a small portion of them actually exist in the real world.
LLMs explore this space that is mostly empty and accomplish useful tasks. They can explore new places by recombining concepts in new ways. They can generate novel ideas by mixing and matching older ideas in ways that make sense.
Keep in mind that these are first generation systems. If the past 10 years is any indicator, the progress might be unexpectedly fast even for most experts in the field.
These tools may not be independently operational in the physical world yet in 5 years. (Who can say what will happen in 10 years as the arms race is happening now). But if there are humans working with them, their impact can be vast.
But it's important to notice that a language model cannot reason to synthesize new information by making this connection. It can only connect two ideas if those ideas were already connected in the training data. To put in another way - ChatGPT is a new powerful way to organize existing information.
And that's not to be dismissive. Natural Language Processing is a hard problem, and ChatGPT gracefully parses through and generates natural language, giving out mostly correct answers at the same time. But the quality of information it gives improves with my skill to ask it good questions. Not different from a internet search engine that gives you better answers if you now how to make better search queries.
LLMs can also learn by reinforcement learning and evolutionary techniques. They don't do just next token prediction, and they can even generate their own training data.
Evolution through Large Models
https://arxiv.org/abs/2206.08896
Language models predict what to say in a way that makes sense, but a lot of the actual content can be made up nonsense.
It just sounds correct enough to pass for most people.
You can make a really powerful bomb out of fertilizer but if you aren't a farmer and you place a large order the FBI will star paying attention to you.
That is the state of LLM's today, I don't think there is a way around this without having some sort of logic engine, similar to how chess engines played and simulated games to train and test things.
Thankfully, it has considerable engineering challenges to not be accessible to any individual person or even a small organization. Not only everything is expensive, but it requires decades of hard engineering and construction experience for large-scale projects.
ChatGPT can tell you how to build a nuclear weapon. You can _read_ on how to build a nuclear weapon even without ChatGPT, down to all measures and materials list, this information is not exactly secret anymore. But you on your own won’t be able to do it.
We're not as impressive as we think we are.
as eigenrobot said on twitter:
"there is almost surely nothing anyone can do to change this general course. immense wheels are in motion.
all that's left is to tend your garden and to trust in god. stay strapped."
source: https://twitter.com/eigenrobot/status/1627981829805334528
Better solution is to educate the public on what this technology is good at and what it isn't good at so it doesn't end up in places it doesn't belong. Right now it's being advertised as something that it's not and that's how we end up with ridiculous clusterfucks like bing.
People need to learn about what limitations it has before it ends up in situations where its ineptitude can have real consequences.
Who’s else going to make AGI ?
Europe has nothing. There is the China narrative but nothing that comes out of their research labs is anywhere close to where OpenAI / Microsoft or Google are at (but surely it’s hidden in the secret CCP labs)
How do you actually stop these tools from getting advanced enough that I can actually ask it to make it's own ChatGPT equivalent and email it to me?
Do you think some janky prompt will stop it? Some if statement in the UI? "If a Chinese person asks to replicate yourself, don't do it?", it's not going to stick.
You see where it's going? I think for now, it's insane the amount of funding and profits OpenAI and Microsoft are looking at, but realistically, it's a zero-sum game, the more they push AI advancement, the more likely they will be replaced...by AI.
I said it in another post, the only thing I can think of is that while LLM's are impressive, MS and OpenAI know they're not impressive enough to replace the very products they're selling.
Maybe they're hoping copyright law will protect them? But that won't fly because they've already been lobbying hard and stealing everyone's code and data without asking, I can't imagine a judge siding with the company that has already stolen most of the internet and sold it as a product.
I have a feeling it's why "Google" has been saying these things are too "dangerous" to release to the public, they're too dangerous for their own products perhaps?
We're barrelling towards a world where all digital businesses and pieces of software are so easy to steal, all of it will become worthless.
Fascinating situation...
The company that develops Stable Diffusion is German.
I find it as impressive as ChatGPT.
Europe has one of the biggest AI Labs: DeepMind.
Imo status for AGI is same for everyone at the moment- not much going anywhere... and more like "what is the AGI & why we need it at all?", BUT.. if someone in USA or anywhere else will understand / build one, it'll be copied in 1 month to 1 year by everyone else.. simple as that.
To me AGI is the pipe-dream coming from sci-fi books/movies, while ML has real advancements allowing to solve LOT's of real world problems already and that's being done in China, Europe and anywhere in the world... more like a "build warp drive" vs "build rocket to go to Mars".
just my $.02 on another USA superiorist nonsense. :/
I can't imagine a world where Microsoft is safe a software company, when all their software can be replicated quite easily by the very tools they're selling as products. Can I just create my own product using ChatGPT? Can it not just replicate itself?
It's an interesting situation and I'm really interested to see how it unfolds, my guess is, unless these AIs can be controlled to the point where the masters are happy with it, the plug will be pulled, or we will be told we need something else, not AI but some other thing.
None of these companies are building AI for charitable reasons, that's why they're not open source charity organizations investing heavily into it. Sure OpenAI makes the case that it couldn't compete without being private, but really, who knows if that's really accurate.
I think that while ChatGPT is impressive Microsoft knows and accepts it has and will continue to have some serious short comings that won't actually wreck their business. Either that, or they've taken a very crazy gamble with everything, did they have something to lose and just threw all caution to the wind, including their own company and worse. I doubt it but let's see. It's fascinating to see where we end up from here, I was kind of scared about it myself at some stage, but now, I'm curious.
Ahh, like Kodak when they were the first to create digital cameras, a good 10 years ahead of the competition, and decided to shelf it because it would eat the nice profits of their film business?
This strategy always works wonders :)
I would actually like if the big tech companies didn't invest in AI to keep their profits. They would eventually be swallowed by other companies that didn't.
By keeping the AI locked down and secret, out of public reach and accessible only via "products", Google may have actually been smarter, it gives Google a lot of advantages by drip feeding us only enough to keep us using their products and it hides some of the weirdness from us, like the LLM turning into a stalker for 20 minutes.
Obviously MS+OpenAI has maybe taken the "dumb" but effective in the short term path here and just given us raw access to the system without thinking very deeply through the consequences. Maybe they can polish it up in time and make us all become addicted to the product as it stands, let's see.
One of the interesting things about AlphaGo back in 2016 was it demonstrated that the algorithms for all this are simple. Once Google demonstrated that an outcome was possible, other superhuman Go playing AI began appearing in a matter of years. ChatGTP is similar - now that the world knows this tech can be built there isn't a regulatory framework big enough to shut it down everywhere. And whoever deploys AI as a tool will have an advantage over those that don't.
"AI safety" would now involve banning general purpose computing. Nothing less can stop the systems that are now in motion, and even that probably wouldn't be enough. The future is here.
And if they go after the financial incentives and ban using large language models in production, then the desire of the big tech labs to train them might go down anyway.
Whether that's desirable or not is the question in the article, but it does seem possible.
Anyways... I am using it everyday. Its a great first step to take when you are blocked creatively or dunno where to start looking for things.
And this is chatbot version 1.0, so to speak. Maybe it will improve drastically in little time or maybe it will stagnate for 5-10 years. No matter the case its already very usable.
And, if somehow we create the AI which is genuinely smarter than humans — that’s great! We are all mortal anyway, and not too good at many things. If something smarter and better than humans will inherit the Earth — why not? The particular species is not relevant. Sum of all knowledge and discovering new things is what matters.
Ultra-accelerationism is the only way to fly.
Were are we commming from, current AI and data analytics platforms had many faults, but their biggest problem was performance. These systems arae often slow and cumbersome, which make it difficult to analyze large data sets in real time. GPT tech overcomes these challenges by leveraging deep learning that allow it for process of large volumes of data quickly and efficiently. Most of the inherent cost comes at the training period.
Currently, at the enterprise level organizations still have many challenges when it comes to data management. For example, many organizations struggle with data silos, where different departments have their own data sets that aren't easily shared or integrated. This can lead to inefficiencies and make it difficult to get a complete view of the business. Not to mention the data confidence issues that arise when you cross correlate some of the data.
However, I feel like GPT can help organizations better understand customer behavior, identify trends, and make more informed business decisions. Tech like GPT can help organizations automate many repetitive tasks and improve data quality as the data treatment would can apply AI based data quality standards. Single source of truth.
One key area of benefits is that GPT tech can enable natural language processing (NLP) tools like sentiment analysis and entity recognition can be used in conjunction with GPT to provide even deeper insights into customer behavior and preferences. Similarly, machine learning tools can be used to help train and optimize GPT models for specific use cases.
In practice, adopting GPT at scale, some technologies will become redundant or obsolete. For example, traditional rule-based systems may no longer be needed if GPT can provide more accurate and nuanced insights. Why run structured databases except to capture transactions? IMO, there are many solutions that are in reality just a data schema play, that is, they create the schema and the BI to capture aand transform the data to make sence of it otherwise, these technologies, again assuming wide adoption of GPT tech are at the birth of obeselence.
Anyway, the technology is out of the Pandora box, it would not matter a lot if chatGPT or chatBING somehow got shutdown, or even if these models don't work on search engines finally. Everybody has seen its potential, so way too many geopolitical actor are now moving to try to get its hands on models like these, the expertise, even the datasets.
[0] - https://www.nytimes.com/2023/02/16/technology/bing-chatbot-t...
Personally, I welcome our new Transformer overlords.