The fun part is when it hallucinates a library like that and it's simple enough that you can ask it to implement it from its own usage examples. I got that working with a Sequel (Ruby ORM) plugin that it insisted existed that didn't - the implementation I asked it for only needed a couple of minor tweaks.
I got it to write a Python module that would intercept calls to missing functions, create the missing function based on the name and calling parameters, write the function to a new .py file, modify the file that called the missing function to import the new file, then execute the new function and return without error.
It was very unhappy about doing so; I had to stress that it was just for research in the prompt to avoid it flat out refusing to write self-modifying code. But it worked!
That makes me wonder where you can go if combining GPT with Koza et.al's genetic programming research, to instead of doing "blind" mutation and crossover, let GPT do targeted changes at high temperature, driven by knowing the goal, and pit different runs against each other.
Hallucinating is the wrong word here. If the prompt explicitly asks it to "simulate" something then it's just following directions.
Hallucinating it would be if it came up with the concept all by itself. Like I asked where I could see the latest Avatar film and it said "easy, just go onto ftp.disney.com and download it".
How far away are we from an AI which can hallucinate and simulate.
The year is 2026
"hey, how can I watch 'avatar 3 - tokyo drift' today? it's not out yet."
"easy, just go onto ftp.disney.com and download it".
"really?? can you do it for me?"
"yeah here it is"
suspiciously fast response arrives, with a video which is a movie about blue aliens that gpt6 has just asked midjourney 8 to create for it.
Thank-you for that. Fans could share prompts as episodes. Wonder how canon would be established, or if it's just throughlines in a wonderful multiverse, remixed to infinity.
Not really challenges, but business models. There is no free lunch so to say. Maybe there will be highly personalized subscriptions that allow for hyperparameter tuning in this way.
I doubt this is something Netflix fears, it is something they gonna embrace. Computation of such movies won't be cheap.
What may save them is that self-service generated movies might end up being worthless as entertainment.
A large part of value of movies, TV shows, books, videogames - even paintings and sculptures - is that they're part of external and shared reality. With self-generated movies (shows, books, games, ...), there will be no process of discovery to enjoy, no immersion to experience. There will be nothing to share or discuss with others, as everyone will have their own version of the story. There will be no canonical version to talk about - there will only be everyone's headcanon, all equivalent, all personal. All boring and worthless.
That kinda sounds like fanfiction to me? And people share and talk about that all the time. It would definitely be more niche, but I could see sites and communities for it springing up, the video equivalent of AO3 or so on. I do think it would be more satisfying to let other people do the "self generated" personal versions of a movie and watch that though, even if it's hobbyist level.
Right, but still, there is only so much fanfiction produced for any given franchise, as there are only so many fanfiction writers able and inclined to do the work.
Now imagine if anyone could casually just fork a piece of writing and adapt to their personal tastes. "Like this, but make X be in love with Y instead of Z. Also make Z not really die but make a surprise comeback in the final chapter." It might be fun to play with such capability for a moment or two, but I can't imagine anyone enjoying exchanging such stories - if everyone can trivially change anything, then everything becomes arbitrary. There is no story, only everyone looking at their own selfie.
Incidentally, GPT-3.5 and GPT-4 already give us this capability with smaller pieces of fiction. You can literally alter any short story like I described above, today. Anyone can. And somehow, I don't see any takers. I think people realize there's no entertainment value in it. Hence, this is what might just save Hollywood et al.
Erotica is one exception to what I described above, as it's by nature the kind of fiction to be enjoyed personally, not in a wider context. And, of course, erotic story generators are as old as the Internet.
> Incidentally, GPT-3.5 and GPT-4 already give us this capability with smaller pieces of fiction. You can literally alter any short story like I described above, today. Anyone can. And somehow, I don't see any takers. I think people realize there's no entertainment value in it. Hence, this is what might just save Hollywood et al.
Writing, even prompting, is more work than reading, and it takes some of the fun out of it - plot twists are no fun if you told someone what they should be.
Even assuming the prompting is near-zero work (i.e. equivalent to commenting on the movie while watching it), you're spot on about the plot twists. Which is one of those reasons why I argue that the ability to self-generate movies, TV shows, etc. will not kill Hollywood and the entertainment industry, simply because people will quickly realize there's no fun in such works (whether self-generated by yourself, or by others).
The beauty is that you don't need to write a complex prompt, you already have a language model to do that for you. You can have GPT write an outline, come up with some plot twists and than write a full story based on all that. You don't have to do anything here.
Plain ChatGPT still requires a bit back and force to get complex tasks going, but Auto-GPT already has a loop around it that makes it possible for GPT to feed into itself and perform complex tasks that are beyond a singular prompt line.
Even without that, ChatGPT can already provide quite serviceable short stories, with no effort on the user beyond "write a short story". The biggest practical show stopper right now is that ChatGPT has strict filtering, so the moment you have guns, action and death, the story just ends. BingChat is even more extreme with the filtering and will fail almost every time.
> With self-generated movies (shows, books, games, ...), there will be no process of discovery to enjoy, no immersion to experience. There will be nothing to share or discuss with others, as everyone will have their own version of the story.
I don't think so. I think users will share what they've generated like they do now with midjourney images.
If I decided to "generate" a pilot episode of a show about robot ninjas fighting a secret war on mars, and it turns out really well, I'll share it on youtube. if enough people like it I might do ep2, or ep3. Then it becomes that shared reality.
Obviously if anyone can just say "gimme a show about robots on mars" instead of looking at what others have already created, the audience might not be as big, but no more than midjourney I'd imagine there will still be some talent and finesse involved in getting the prompts just right, and most people will be too lazy and prefer to browse what others have already done
I agree with your sentiment, but everybody wants their own version of the story, which is why fanfiction is overwhelmingly incoherent and terrible. Nobody wants to be challenged by anyone else's differing opinion.
Like it or not, on-demand fanfiction is where we're headed. It's not impossible that a canonical version of a story might be published though, like a video game. There's a main story about you saving the world from nuclear disaster, but if you spend 600 hours trying to romance the barfly in the first tavern and never fulfill your role, well, that's your story about a lecherous loser in the land of the lounge lizards, man.
This isn't anything new either; look at the Protestant Reformation. Everybody wanted a version of The Story that appealed to their own interests then too. Now you can pick a church whose ending you most prefer.
It will not with the current way it goes, because AI models are artificially biased to not offend intellectual property owners and to protect powerful people or influential lobbies.
Mickey Mouse according to OpenAI is just a symptom, but the root of the issue is very deep if the people preparing the dataset intentionally decide to exclude examples due to commercial or political pressure.
You're talking as if weak AI is sentient and performing the act itself. The damages are from actual people and/or companies involved in the act of distributing derivative works from copyrighted material using forward diffusion, gradient descent, and other techniques.
I've noticed that ChatGPT will often suggest paying for SaaS solutions over using FOSS or building from scratch even for simple things. Definitely heavily biased towards "buy" rather than "build." Hard to say if it's a general trend but it did tickle my cynicism.
I think that's more likely due to internet commentators generally suggesting SaaS solutions over using FOSS or building from scratch. How many more blog posts are there about how great a SaaS is than there are about how they really liked building a product in house. ChatGPT reflects the internet, and the internet is a SaaS seller's world.
And why would your example be considered a hallucination, then?
The truth is, we have no need for the notion as it applies to AI. It's just a catchy term that sounds great in the media. Nothing the algorithm spits out is a hallucination (or else ALL of it is) because, quite literally, It's all the same uncontrollable output.
It'd be as weird as saying your autocorrect feature "hallucinated" any time it presented the "wrong" word to you. (As if it's of clear mind the other times?!)
I think the distinction is utterly useless in such a use case and ultimately just expresses our understanding or interpretation of the outcome. And trying to categorize it's output in those terms is highly misleading regarding how it works.
We speak of "hallucination" when GPT provides completions that are not connected to any facts it was "provided with" in either training data or prompts, but ultimately factuality makes no difference to GPT.
It ALWAYS just "simulates" text, that's what it does.
If you'd want to use these words to describe factual accuracy, or to express if GPT "performs" an action, as opposed to "faking" a result, you have to bear in mind either of these qualifications are nothing but a human interpretation of the completion.
It being "correct" more often than not with the content of it's simulation is just a logical, but ultimately accidental consequence of being so well fitted to simulate the structure of data it trained itself on.
And a "halucination" is nothing but this simulation deviating from the context you provided, that fundamentally was simulated by the same rules that produced any other content which didn't deviate.
This is a good example for what ChatGPT does: It makes shit up and presents it in a compelling format. You should always keep this in mind when trying to get information on a subject you are not familiar with yourself.
I use ChatGPT daily, but always specify the information I want formatted. (In bullet points, a table or for example a news article.)
I use it for first drafts of some less demanding writing work. My workflow is more or less research with Bing Chat for summarisation and key point extraction, then feeding the results to GPT-4 so it uses the research as context for a first draft. Then the longest phase is editing, fact-checking, adding new information as required, etc. I’ve also created vector indexes for large documentation sites I work with and used langchain so I can query them, get sources, and have ChatGPT write summaries. Honestly, it doesn’t save a huge amount of time because then research phase was always the most time consuming to begin with: I still have to read and understand the material.
Of course, that’s only possible if the context content exists in the first place. If not, then it’s back to the old-fashioned technique of figuring things out for myself and writing about them.
Not OP but I am using it to finally shape my blog into what I always wanted. It still takes me up to 100 hours to finish an article (sometimes more than what I used to spend on an article before ChatGPT) but my articles are so much better and the research process is so much more insightful and fun.
It's great at little tedious tasks, as long as you review its work.
Like converting a code sample from one language to another, or one format to another (e.g. raw css to object syntax), or complex replaces where the regex gets tricky- also great at writing simple docs.
Also use it for micro tutorials if it's on a topic older than a couple years old.
Or converting unstructured (or semi-structured) text to structured format. My most recent example with GPT 3.5: copy-pasting a piece of a website with a list of dates and events, and prepending it with "Please rewrite it into ICAL format".
I use it daily, sort of like a professional site/forum where someone immediately answers any question you may have. It may not be 100 % accurate, just like real people, but it's instant and personalized. (And in my experience GPT-4 is just spot on for anything a beginner might ask.) It's like training wheels for anything.
How do I set up some backup infrastructure for my home server? How does Btrfs handle read errors? How do I cook X if I don't have ingredient Y? What's the correct syntax for this command? How to analyse and interpret these measurements from this vermicomposting experiment (helping my girlfriend with statistics)? How do noise cancelling headphones REALLY work? Is it okay to have my samba.conf set like this (it wasn't)?
I can't say anything specific about the accuracy, other than that it's good enough, but the difference between asking GPT-4 and googling the same question is night and day. And that's comparing it to Kagi, with my own search filters with boosted wikipedia, stackoverflow and generally scientific sources. Do you know the feeling that you want to ask a specific question about a specific, technical aspect of something, and you wish you could just tell the search engine somehow that you're just not interested in hand-wavy popular science average Joe half-incorrect answers? And if you actually find the one good answer on page 14 of some god-forgotten thread from 2006, the question differs in some tiny but important detail from what you're after?
Well that feeling doesn't exist with GPT-4. So far we've always been able to come up with something, together. If you don't like the first answer, you ask more questions. You can dig deeper. You can tell it what your guess is, what your intuition tells you about the problem, where exactly your uncertainty lies, e.g.:
You: I don't understand phenomenon X.
GPT: Oh that's easy, X is just [parrots wikipedia].
You: That's fine, but I don't understand how X differs from Y, they sound like the same phenomena.
GPT: I see why you'd think that, but Y and X actually differ in this detail called Foo that makes all the difference.
You: I still don't get it, compare Foo to something similar that I know from normal life.
GPT: Okay, so Foo is like an elephant who's too large to drive a car.
You: Ooh, I get it now.
Me : I don't understand this publicly documented AWS service with plenty of OSS examples, can you suggest how I would solve some edge case X
GPT : Oh that's easy, invents bullshit that sounds exactly like what I need
Me : Googles the shit the GPT came up with and can't find any references
Me : Maybe there is something similar - spends more time searching
Me : GPT you're wrong - your solution doesn't exist
GPT : Oh sorry, here's the correct solution, comes up with more bulshit
Me : Googles the shit that GPT came up with and can't find any references
Me : Starts googling and solving the problem on my own
Me : Finds out X can't be done with AWS service
I've had this flow many times now, it never resulted in valuable output from GPT vs just Googling since I need to verify everything anyway.
Or
Me : Review and come up with improvements to this code : CP code
GPT : Suggest bullshit improvements, gives factually wrong reasoning, makes code worse
Me : Points out everything wrong
GPT : Suggest other bullshit improvements
Me : Points out everything wrong
GPT : Returns basically identical code to input
Or
Me : Propose a design to this technical problem
GPT : Implements the solution in the most roundabout way and suboptimal way
IMO if I'm not sure what the output should be GPT is less than worthless it's actually convincingly misleading.
I've upgraded to 4 ASAP, all those examples are with 4. I tried 3.5 buy it was completely useless, 4 actually started giving sane output but still lying and hard to verify.
My only use case so far is stuff like "translate this model to openapi schema" because it was faster than setting up the tool, and similar tasks that are easy to verify and boring to type.
This is what I'm most curious about with AI. How often do you catch it being incorrect? The questions you're asking a fairly inconsequential if wrong, but how often do you fact check what it says?
Not daily, but often: questions about history. GPT3/4 demonstrates good historical insight and the ability to make nuanced arguments. For instance, I like the mention of some of Reagan's "liberal" policies in this answer:
"Can Ronald Reagan be considered a liberal?
Ronald Reagan is generally considered a conservative politician. He was a member of the Republican Party and served as the 40th President of the United States from 1981 to 1989.
Reagan's political philosophy was based on a belief in limited government, individual freedom, and free markets. He advocated for a reduction in government regulation and taxes, increased defense spending, and a strong national defense. He was also a strong supporter of traditional social values, including opposition to abortion and same-sex marriage.
While Reagan did take some positions that could be considered more liberal, such as his support for amnesty for undocumented immigrants and his advocacy for the abolition of nuclear weapons, overall his policies and beliefs aligned more with conservative ideology. Therefore, he is not typically considered a liberal."
Now ask this to a human. You will get a polarized answer "No, Reagan is a conservative icon!!!!!!" lacking any nuance.
'Considered a liberal' by who? ChatGPT doesn't tell us. I only mention this because the academic definition of the term 'liberal' is very different from the colloquial understanding.
> Now ask this to a human. You will get a polarized answer "No, Reagan is a conservative icon!!!!!!" lacking any nuance.
Depends on the human. Ask a political science professor and the result will probably be pretty similar, plus an earful on the evolution of the two major US political parties' positions since the 1980s and how that complicates these kinds of retrospective judgements.
Quite off topic, but: When dreaming I imagine reddit posts, comments, tweets, HN submissions, code, UI, etc. This also happens when I have had 4 days of no working at all. Then I wake up and am unsure whether the information I read is real or not.
I've been struggling with jet lag, the other day I surprised myself by getting up at 6am and being very productive, enjoying the sunrise and my morning coffee until I looked at a clock and it said something like "O8:X4f" and I was like "No! This is a dream!" and woke up, finding it to be 11am. Stupid brain.
Hallucinating is not quite accurate. I'd say, GPT4 role-plays an FTP server. From the script it appears that GPT4 is fully 'aware' that the FTP server is not real, but that they are playing make-believe together.
A true hallucination would be, if I were to ask where can I get Disney songs, and it referred me to a non-existent FTP server.
No, it’s not. If I ask ChatGPT to write a story about Susan B. Anthony going to the moon, it would not be accurate to say it “hallucinated” that she had gone to the moon.
OpenAI are themselves shoring up this kind of anthropomorphization by having ChatGPT respond to queries in a human way. I think it’s fine. We know it’s not really human. But it’s convenient to think of the GPT models as though they are.
And like someone said before, perhaps we can think of the GPT models not so much as “artificial intelligence”, but more like a different kind of intelligence from what humans have. With strengths and weaknesses of its own.
I’d like to think that we are making some progress in our conversations about AI, but comments like this really just pull us back.
You even ignored the quotes that OP put around ‘aware’.
As someone working with GPT-3 day-to-day at $dayjob, these words - usually reserved for sentient beings - are simply put typically the most apt to use in pretty much all contexts other than some sections of an academic paper. The fact that you yourself couldn’t get the sentence to work right is a testament to that.
Again, as someone that’s working with and having conversations about this day-to-day, we all find ourselves using these words, first kind of awkwardly, with laughs and air quotes, and we try to mentally reach for the best word to use. Eventually we all just know what we mean, and that’s that.
So, if I can ask directly, how do I talk about this online in a way that allows for productive conversations about the very obviously disruptive abilities of LLMs but that still avoids your snark?
Probably? I mean, if you were to ask me, why I wrote any of my comments the way I did, half of the time the honest answer would be "because it fits" or "it sounds right". Surely I'm not the only one who's aware they're speaking and writing mostly by generating completions and picking ones that sound best.
The way it feels to me is that I have a fully formed concept almost immediately upon hearing or reading something then I employ language to try and convey the same concept into to someone else’s brain.
During that process additional ideas may occur to me, and that may augment the final output.
I experience some of what you describe, too, when dealing with ideas I already am familiar with, or explaining them to others. But when I'm trying to work through some novel thoughts, or am engaging in a conversation - including the one that has me writing this comment - I function in the way that's peculiarly similar to LLMs. That is, what pops into my head are words, phrases, sometimes entire sentences (often movie or videogame quotes). I pick one that fits best as an extension of what was already said - the previous words, the previous sentences, the response from the other party - and then repeat with the next completion, and the next, etc., until I feel it's time to stop.
I've joked for many years that, based on my personal experience and introspection, human brains at the highest level may just be glorified Markov chains. I sometimes still say that about LLMs - but I do so more as a compliment now. GPT-3 and GPT-4 may be just fancy autocomplete, but so are we. I feel we've brute-forced our way into a turning point in our understanding of what intelligence is and how it works.
The concept is always fully formed (or at least SOME concept is fully formed) before I start typing.
Like I read your comment. I had thoughts during that process, but I read until the end, then I made a decision about what I thought, and that decision contained all the information that I want to convey in my response.
Now: when I am constructing my response, I am of course modulating which words might come next that would best fit into my language model to convey that concept, but the concept is an instantaneous understanding.
That instantaneous understanding may change during the process of articulation, but not fundamentally -- I think that any changes during the process of articulation could be considered "elaboration" rather than a reconceptualisation.
All the language that I just typed was designed to convey a concept that was more or less instantaneously formed and fully complete prior to my typing anything. The way in which I construct the language to articulate that concept may be very similar to the way in which a LLM constructs its responses, but I think the core difference is that original and instantaneous conceptual understanding of what it is trying to articulate.
I’m a mix between you two, I think. I know what thought I want to convey, before writing, but not the exact words I want to use. Even as I’m writing, I’m pausing for a few moments to figure out the best word for the thought, determining if it will lead to the next thought, how you might accept it, and then writing it down. And so on.
Only moments before I write the word, do I know what that word will be, but I know the complete thought I want to convey. I don’t know if this is important, but my thoughts are not words, so I’m basically just describing them using words.
Yeah I think that's basically what I'm saying. The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but if you were to describe that elephant to someone you'd have to go through all sorts of decisions about what to say and how to say it and you'd change your mind, and you'd say in one instance that the ears were flat, then you'd say no not really flat in the sense that they're level but very thin and floppy. And the person would ask questions and you'd have to reply to them.
The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression is like an instantaneous graphical image of an elephant, that one must then put into words.
I think that a LLM doesn't have that. I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language. The concepts are all pre-existing in the same way that all the chess move possibilities pre-exist in the programming of a chess computer.
EDIT: In fact even the way I described concepts as "pre-existing" there might be giving it too much credit. Like the concept only arises on the receiving end, from a pool of concepts that were pre-existing but about which the LLM has no "awareness" whatsoever.
> The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but if you were to describe that elephant to someone you'd have to go through all sorts of decisions (...)
The question is, do you know its attributes before you describe them to yourself? To me, at least, an "elephant" feels like a pointer, a handle, a promise - I know it's a thing, but to know more than that, I have to dereference the pointer. And that usually means something that's getting close to stringing words together.
Possible confounder: I suffer from aphantasia[0]. I have nearly zero ability to perceive images in my mind, except when dreaming. Maybe it's easier to "know immediately what it [the elephant] is and all it's attributes", if it just pops into your head as a complete image. I, unfortunately, don't experience that.
> The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression (...)
> I think that a LLM doesn't have that.
Crazy idea: what if a concept is... a partial set of coordinates in the latent space? Enough to describe a pretty narrow region.
> I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language.
My tentative guess is that language will turn out to not be the fundamental part of what makes LLMs so powerful/spectacular. I think the magic is the latent space itself - the high-dimensional constructs where all kinds of associations end up represented as spatial distance. Language is just a serialization format[1] for latent space associations. By being fed lots and lots of text in training, LLMs build up the latent space, piece by piece, until there's enough dimensions to it that it is no longer a simple mapping between words learned from the training data. I think this is how understanding and concepts look like.
Also, I think it takes so much input text for LLMs to get to their current performance, because LLMs learn like babies - they soak up everything they can, letting the inputs self-arrange by correlations. We learn to understand what an elephant is by hearing stories about it, seeing it alive, and/or being told facts about it in a structured way (i.e. school). LLMs learn what elephant is by soaking up so much text that (the token representing) the word "elephant" is, in the latent space, close to all the words people used to describe its attributes - which themselves are close to other words with whatever various meanings and aspects we relate to those attributes, etc.
EDIT:
And connecting the two parts of the comment: LLMs are AI's poster child of aphantasia. If they have concepts, how else could they work with them if not by... articulating them? And when they articulate a concept, what makes us so sure they're just parroting associations, and not truly having a concept?
[1] - Quite literally, as communicating via any language happens via ingesting or producing a series of some kind of tokens (letters, words, ideographs, etc.) over time.
> Like I read your comment. I had thoughts during that process, but I read until the end, then I made a decision about what I thought, and that decision contained all the information that I want to convey in my response.
I recognize that mode. I have it to. I had it now; this comment is aiming to transmit a thought that came to me fully formed while reading your response. That said, it's a bit suspect how it popped up as if my mind was doing an associative search. It's also interesting that, however fully formed the thought might have been, it's also small, as if a conceptual token, and I usually have to run further associative lookups to extend it before the comment (or train of thought) feels satisfactory to me.
In the previous paragraph, the last two sentences were not part of expressing the fully-formed thought described earlier. I'm not lying to you, it's just I didn't expect to write the second half of the previous paragraph (and this paragraph) when I was writing the first one; they came from conceptual autocomplete.
> That instantaneous understanding may change during the process of articulation, but not fundamentally
For me, it quite often does. I rely on it happening often. Example cases:
- On HN, I post maybe 1/3 of the comments I write. 2/3 of the time, the act of trying to express my thoughts makes me realize that a) I'm wrong, and/or b) it's a bad idea to continue (e.g. I'm just procrastinating, or posting something impolite, or not useful, etc.). Of the 1/3 that survive long enough to be posted, I still delete some - as sometimes the realization from the previous sentence hits me only a couple seconds after posting.
- When facing difficult problems, I do a lot of internal monologue. In it, exploring whether certain phrases feel right, and kind of sculpting the chain of thought, is a critical part of the process.
- Because I often feel I lack the working memory to keep enough thoughts in flight, I often "talk" with myself via a text file - externalizing my internal monologue. That process also involves sculpting sentences, paragraphs or bullet points until they feel right, so it's arguably still articulation. And the more I think about it, the more this looks to me like the kind of tricks you do to work around the token limit / window of GPT-3 / GPT-4...
EDIT:
Ultimately, I think we're both doing thinking in similar ways, perhaps with different proportions between modes, and different conception of it.
One relevant conceptualization I have that I remembered just now: in conversational situations, I often feel like my subconsciousness is a Markov chain / LLM autocomplete, spitting out phrases, and my consciousness is busy judging them, and choosing whether or not to pass them to the actuators (speech organs, or fingers on the keyboard).
The two "models" - the autocomplete and the censor - is something I can clearly see in my mind when I'm processing a problem in a calm manner, or participating in a slow-paced discussion. But I've noticed that the more excited I get (or intoxicated), the weaker the "censor" part is; in the limit (when I'm really excited, angry, or drunk), I feel as if the Markov chain bit is feeding phrases straight to my mouth/fingers, cutting out the middleman of consciousness.
(Of course, what I wrote is all "how the algorithm feels from inside", so it's not very objective or reliable...)
Good question. I recently started to wonder about it myself. In my train of thoughts, I tend to switch between languages on the fly, based on which one feels like it's bringing me closer conceptually and emotionally to the thought I want to express.
Or at least I used to, because I only recently realized that, over the last few years, I've mostly switched to thinking in English only. I'm now in a weird state: my native language (Polish) is still easier for me to use, but also using it feels wrong somehow. Often enough, when talking to people around me, I notice I'm translating my own thoughts from English.
It might be that this process of learning and internalizing a second language, to the point it eventually took over my first one, is what changed my experience of the thinking process itself.
What's to stop a Markov chain or very simple language model (something that prompting doesn't work on, like an RNN with a modest hidden state) from meeting this definition? Because I wouldn't buy that either of these are aware, and both of them predict the next token.
I think it's a matter of degree, not of kind. That is, the complexity of the model predicting the next token matters, and so does the space of possible tokens.
Ultimately, I think the definite answer would require solving the halting problem. The approximate answer we should be able to get when we let models interact with some complex environment over time. That is, the model predicts the next token, which may cause some change of a shared state, and then the model receives a response.
I dunno. Is there a testable prediction of this hypothesis? I just don't really see why I should believe it other than that chatting to LLMs is compelling, which isn't enough for me.
This seems like an answer to the question, "why is the LLM so compelling?" And not "what makes something conscious/aware?" Which is an important question but feels like not quite the right one to ask and then come to this conclusion. And it bothers me if I try to accept an idea where I don't know what conditions I'd reject it (eg, I don't know what evidence would cause me to reject this hypothesis, I can't see a test I can perform to next reject it).
Simple models will very quickly produce content that doesn't fit. Just stringing a bit of grammar and a dictionary together will produce locally correct sentences, but it will completely fail at generate larger texts with context. GPT in contrast knows what it was talking about five sentences ago and can refer to it in newly formed sentences.
Go and write a Markov chain, you need some text, a hashtable and a few minutes, you'll quickly see that they generate rubbish which doesn't fit anywhere, no sense of context, no sense of 'chat', no sense of intelligence to them.
ChatGPT 'predicts the next token' so much better than other 'next token predictors' that using that as the comparison for a low effort dismissal is like saying "what's to stop a human tracking a criminal by scent, humans have noses and dogs have noses so they must be the same".
I'm aware of how these models work (I have trained simple LSTM text generators); I didn't dismiss anything (other than that simple models aren't aware, which you don't seem to contest).
If we agree that Markov chains aren't aware, then surely we also agree this definition of what makes them aware is either incomplete or incorrect.
Predicting the next token turns out to be quite profound, I'm just not convinced, at this time, it's sufficient to be aware of conscious. If you have a reason I should believe it, I'm all ears.
Predicting the next token perfectly would make a chatbot an omniscient god, as it could tell you things like the lottery numbers from next week. Chatbots "just" being a token predictors does not limit their power or intelligence in any way. It however also doesn't tell you how good they are at their job.
If you want to test how good a chatbot is at predicting tokens, well, you just have to test it. You can throw a Winograd schema challenge at it, or let it take part in code competition or exams, let it solve puzzles and all that. Quick simple test will for example show you that ChatGPT isn't any good at calculating, as the answers will just be wrong.
As for awareness, we know what ChatGPT is aware of: Its 4000 token history and whatever information it learned from the training data. That's the grand total of its existence, it doesn't get any other input, it doesn't perceive time, it doesn't learn, that history is all there is to it. And we can call that awareness as ChatGPT is able to perform correct prediction based on that token history.
For consciousness there are still a lot of basic parts missing, such as the perception of time, senses to act with some external world, the ability to think for itself without getting explicitly prompted and all that.
I'm not really interested in how well or poorly the models perform (in this context), I'm trying to understand why people have certain ideas about them.
I think you and I are generally in agreement about consciousness and awareness. I think consciousness is an interaction of an agent with it's environment, rather than an agent in isolation. I guess that's why I find this attitude that consciousness is a matter of feeding enough data into a static training process confusing. I think it's a kind of is-ought problem where I ask about the "ought" and people express the "is" and we end up talking past each other.
But I don't really have a good hypothesis to disprove this either, so I'll have to meditate on that. I think it may be testable but it's challenging to model.
I guess I have been using the term hallucination "correctly" then.
Because GPT often tells me to use programming libraries that I can find no record of anywhere and generates URLs of references that I can also find no reference of ever existing.
Nice. I adapted the prompt for the good old gatekeeper.dec.com and got similarly detailed and reasonably plausible output, though it seemed to be a mix of realistic output and possible anachronisms (e.g. software and research that feels too new, though I can't say for sure as gatekeeper lingered on for years after it lost its importance).
In 3.5 days I had it "hallucinate" you could share documents with it via Google Drive.
Transcript:
Can I give you a document and ask for a summary?
Yes, you can give me a document and ask for a summary. However, please note that as an AI language model, my ability to summarize a document may depend on the length and complexity of the document, as well as the quality and clarity of the writing. Additionally, please be aware that I am an automated system and may not always capture the main points or nuances of a document as accurately as a human reader.
How do I give you the document?
You can provide me with the document by copying and pasting the text into our conversation. Alternatively, you can upload the document to a cloud storage service such as Google Drive or Dropbox, and then share the link with me so that I can access and read the document. Please note that I will only be able to access public documents or those for which you provide me with appropriate access.
How do I share a document on a google drive with you?
To share a document on Google Drive with me, please follow these steps:
1. Upload the document to your Google Drive account.
2. Right-click on the document and select "Share" from the dropdown menu.
3. In the "Share with people and groups" box, type "chatgpt@openai.com" to add me as a collaborator.
4, Choose the appropriate sharing settings (e.g., "Can edit," "Can comment," or "Can view") for the document.
5. Click "Send" to share the document with me.
Once you have shared the document, I will be able to access it and provide a summary if you request it.
I'm sorry, but as an AI language model, I do not have access to external sources or links. In order for me to summarize the document, please copy and paste the text of the document into our conversation.
I think it's likely that ChatGPT made up the email address, but what if this is some functionality Open AI has already added, but not enabled, in the system? Or instructed ChatGPT to say that it's unable to access external sources/links when that's not exactly the truth.
My take is it would get the general pattern of how to share a file wirh someone on Google Drive, even "inventing" a likely email adress from a pattern, regardless of the presence or absense of the functionality existing, as it has no "knowledge" of the fact.
Once 3.5 suggested me that there was an upload button by the input box. After spending a few minutes confused not finding it, I asked for clarifications: GPT said it was sorry, it is possible that not all interfaces have this feature and the one I'm using doesn't. It was extremely convincing and to this day I'm still not sure if the, entirely plausible, upload button isn't some premium feature I don't have access to.
Whether or not the button exists in someone's interface is likely separate than the means used by GPT to reason it was the case.
GPT responds that there is an upload button because a lot of interfaces are described that way around the web -- there is not likely a hard reference for its answer in regard to its own interface.
There was a similar incident a while back where someone asked it whether you could make queries via Telegram, and it just handed out a random person's phone number saying they were ChatGPT.
I experimented a lot with 3.5 and the extent to which it would hallucinate. It was wild. It would argue with me and I'd say something was wrong and ask for its source. It would make up a URL. I would then give it my own URL and then it would start to agree with me based on the website I provided that it couldn't actually access. But sometimes it wouldn't agree, despite my "source", for reasons that didn't make sense. Or sometimes it would agree for reasons that also didn't make sense.
I also had it suggest there was emailing capability in the playground interface, or it would say that it emailed me something I asked for. It would claim to have abilities to do a bunch of stuff like this that it couldn't actually.
Also recently discovered it hallucinates dates and times and has 100% confidence they're correct
Confabulation adding data which was not provided. Example making up the plot of an episode.
Hallucination to claim stimuli which was not provided. Claiming my prompt included something it did not.
Delusions refusing to admit an error or mistake and the best thing to do is reset the history and explicitly exclude the topic. Examples accusing me of spreading harmful disinformation when I brought up Kirstie Alley's death.
Often it's mistaken and it's none of those, you can simply correct it or direct it better.
I've experienced both, and deal with both in different ways.
It was never accurate. The concatenated strings just failed to conform to reality as it often does. People are projecting too hard on a chatbot. It's just a chatbot people.
I have never seen hype precede significant change. It's usually the other way around. It took people years to realize how much PC and smartphones and social media were impacting society.
And I never seen lower entry barriers improve quality of systems. Also, I don't think generating code in a probabilistic manner is a win.
I'm not in a camp. What is it with current discourse that only sees adversary teams as valid positions?
> I have never seen hype precede significant change. It's usually the other way around. It took people years to realize how much PC and smartphones and social media were impacting society.
Great counter-point. It is a false dichotomy to view this as a binary matter (terrible pun not intended).
During the information superhighway era of the nascent internet, opinions on what would become varied as well (I recall with a bit less sarcasm and absolutist thinking).[1]
The hype was not widespread, and even though some prognostications were fairly accurate in broadly outlining societal and economic benefits, predicting the downsides was an oversight.
Maybe failing to foresee these difficulties in the past has left some people jaded? Could this be driving much of the cynicism and skepticism in the great AI debate?
One scholarly but quite compact book charting this story of early, techno-centric and hyper-optimistic expectations being frustrated by unappreciated socioeconomic, political and cultural realities is 'Misunderstanding the Internet', by James Curran, Natalie Fenton and Des Freedman.
That must be a US centric view point, maybe due to FPTP voting? You can be in many camps at once, there is more than two camps and they don't have to be necessarily adversarial towards each other either. However, if you think that LLMs are just stochastic parrots, then that probably guides a lot of other arguments and lines of thinking.
> I have never seen hype precede significant change.
I think you have to separate what's hype from what's not. I personally see three things here:
- The usual hype, now AI, previously Bitcoin/NFT, some time earlier it was "sharing economy"[0] / "gig economy", earlier it was the cloud. Some things change, some stay the same, but there's little point paying attention to it unless you're in for a quick buck - for the shrewd, that hype is effectively a pyramid scheme with extra steps.
- The fear of losing your job to the AI. This, IMHO, is a quite realistic fear - more so than ever - because GPT-3.5 is already[1] good enough to replace plenty of jobs. It's just the market hasn't caught up to it yet. But it will soon enough.
- GAI, consciousness. There's no hype there. It may seem there is, because you're part of the small group that cares and can productively discuss the topic; for most people, it flies entirely over their heads. On that topic, however, I feel quite strongly[2] that LLMs are a major breakthrough in understanding the nature of intelligence. Disregarding the language model aspects, whatever is going on in the latent space, it seems unexpectedly powerful.
My speculative take: it might be that intelligence is mostly a natural consequence of building a high-dimensional association map between sensory inputs. It would definitely make sense in the context of evolution, and the spectrum of intelligence that various forms of life fall on. Perhaps we didn't realize it before because we can't imagine high-dimensional spaces all that well.
Even if my hypothesis turns out to be false, I can't imagine something akin to LLMs, with high-dimensional latent space, not becoming one of the fundamental concepts in AI and cognitive science. I expect we'll find examples that fit this model in nature. I expect we'll find that a part of our brains fits that model too.
But back to the point, the GAI/cognitive/consciousness aspects are hardly hyped. They're just becoming hot again among the small crowd that is interested in this. As you say, it's usually that hype follows real change. I imagine if LLMs end up shedding light on the topic of consciousness, both hype and hate will follow, particularly from the world's religions.
--
[0] - $deity knows how crazy it was to hear the excitement about "shareconomy" coming from the startup/entrepreneurial crowd, seemingly oblivious to the fact that anything they do in this space will be abusing, exploiting and otherwise ruining it...
[1] - Consider all the things people use ChatGPT for. All the things that you may be using it for. There are plenty of jobs that are entirely contained within the "area of effectiveness" of ChatGPT. And I'm talking the GPT-3.5 ChatGPT here (sadly, I still don't have the API keys to GPT-4 :( ).
[2] - I'll give it 90% confidence, though I'm not exactly well calibrated.
>My speculative take: it might be that intelligence is mostly a natural consequence of building a high-dimensional association map between sensory inputs.
What if consciousness actually is the high dimensional entity that can map to sensory input and motor output of neural networks? Then expecting it to emerge from highly manipulated memory spaces would be like shocking cadaver brains with lighting to get a person... Physics is looking more and more like it cannot actually describe an objective reality that is perceived, pointing more to reality being the descriptions that we can build of experience in high dimensional spaces.
But speculation apart, it doesn't much matter how we think of chatbots or what they can do. Ultimately all AI/ML stuff is just up front compute stored in models, and they are just forms of capital. They will be used as other forms of capital.
To think that we can build people surrogates because chatbots are getting impressive is... sales hype pure and simple. Much like thinking you don't need servers because you use "the cloud". You're just outsourcing engineering, something that has been done for centuries.
I don't follow. Thinking of LLMs as stochastic parrots is a viewpoint I've seen repeatedly and one that probably informs how you think about LLMs and their potential. I don't know where memes or 4chan fit into that.
I'm talking about your trying to split comments into camps. That's the root of internet flame wars. That's where 4chan fits.
Allegedly, HN is a place to hold curious conversation. What is curious in attempting to explain a comment as coming from a certain "camp"? Isn't that just going to polarise conversations and shut down any curious discussion?
> You can be in many camps at once, there is more than two camps and they don't have to be necessarily adversarial towards each other either. However, if you think that LLMs are just stochastic parrots then that probably guides a lot of other arguments and lines of thinking.
We are a long ways off from a flame war and 4chan here.
>> However, if you think that LLMs are just stochastic parrots then that probably guides a lot of other arguments and lines of thinking.
Making this sort of assumption about your interlocutor deprives the discussion of any kind of interest it would have. It's like we have seen that match before and we know what each side is going to say, so what's the point in debating anything? Everyone's convinced, everyone's stuck in their little corner, and calling each other group-names.
More to the point, it makes people reply not to each other, but to the views they assume the other person holds, which never leads to productive conversation, only to confusion and chaos.
There has to be a way to debate that doesn't involve picking an enemy to fight.
Not that "stochastic parrots" is a 4chan thing. The polarised debate with "us" vs. "them" is what reminds me of 4chan. And not just 4chan, it's how people generally behave on the internet: it's always "us" vs "them" and there's no space for interesting conversation.
fair enough lad - all i'm saying if this phrase has 200 more citations in literature than posts on /g/ i wouldn't invoke the stichwort "4chan"
Extreme Polarization in debate FAR easily predates imageboards and even USENET, vi vs emacs probably harkens back to some of the earliest messages on DARPANET (ok, probably the text editor wars were close to a decade later, because i think it was more like the "ed" vs "god do we have anything else" era, but you get the idea -- i'm sadly not old enough to know the details of this golden era)
or look upon ye olde days of Slashdot trolling (some netcraftian oracles say that BSD is still, indeed, dying)
if you really do want to track the modern internet manifestation back you have to go back before 4chan. We'd have to talk about SA and GBS vs FYAD, and some of the earlier predecessors of SA such as.... Maddox.. and others, i guess? I can't think of other sites, only other names.
In any case - don't give 4chan credit where it's undue. Us vs Them goes back a long, long while, perhaps even pre-internet!
Anyway, what's with the assumptions? "Lad"? A lad named YeGoblynQueenne? Not that there isn't precedent for that sort of thing [1], but, let's not jump to conclusions.
it's been used 6 times on /g/ -- 3/6 are mocking the term, 1/6 is calling people stochastic parrots (haha), and 2/6 are using it in the sense you're implying.
So not really a 4chan type thing. It's been cited by other papers apparently 200x more than it's been posted there.
I'm still setting up the tiny tent for the "delusional parrot" camp, right next to the "it's gonna get better for real and change the world" camp, which seems to have a quite an enormous tent and has a suspiciously large flashing banner that draws a lot of attention. There's a sign next to it that says: "Mt. AI goldrush is now open, buy your shovels and pickaxes now!111!!"
Hallucination is what it does, it never stops hallucinating. Whether and how we assign truth values to it is a different matter and quite arbitrary. I suspect we need to cooperate with another AI fit for that specific purpose.
I think where people misunderstand these LLMs is they think they have creativity. But they don't. They cannot think of something new. They can simply rehash. So it is useless for any actual creative work. Once you have the creative idea, it can generate a few paragraphs which sound OK but you need to carefully double check.
I spent about 10 hours one day trying to convince my professional artist friend that AI could produce legitimate works of art, against his insistence that they could at best produce "kitsch" collections of art-like patterns, the kind of stuff that belongs on the wall at malls and not being discussed by academics in galleries.
I have to say that, although I still disagree with him, I was unable to get GPT to produce midjourney prompts which I would consider creative. Even when I prompted GPT to take the perspective of a known artist, it produced an endless series of banal landscapes, cliche compositions, overplayed metaphors, and insipid imagery.
Midjourney only really accels when a human is involved with the prompting.
I had to concede to him at the end of the discussion that /today's/ AI are not creative.
I firmly believe though that the inclusion of all mankind's perspectives possesses the raw material to generate creative ideas. My understanding is that current AI lack the ability to "turn off" the vanilla oversaturation of averages that pervade when you are exposed to all stimuli. One day, perhaps soon, there will be AI able to combine ideas they are exposed to without trying to combine ALL of the ideas at once.
how is that different from humans ? how common is truly original thought in humans ? to my knowledge humans work exactly the same, we "simply" rehash and mix existing ideas and repeat ending up with "novel" concepts. To common people these ideas may seem original, while to an expert it's just rehashed information.
original work often does not mean doing something already done in a slightly different way (paint a different set of sunflowers than Van Gogh)
What these models do is more similar to pastiche, if we really want to compare it to some technique.
But the value of the pastiche depends on the intent and its perception by the viewer(s), because there's nothing inherently original in it.
So basically what models produce has no value, unless we are able to attach some to it. [1]
If we asked chat-gpt to analyze something it has produced, it will probably say that it is "similar to" or "in resemblance of" but it's unlikely that it would say "this is the work of genius, how original! lovely!"
[1] edit: you don't read "pizza maker creates fabulous art on AI" but <person who's already in the business> won a contest of <some art form> submitting something created by <AI of your choosing>. Why? Because they know how to market it and can rely on other people believing that's their creation. Nobody says it upfront "I will submit an AI generated work", because they know it won't be judged the same way. The pizza maker was probably trying it for fun or to make a new logo for the pizza place or simply has no instrument to assign it a value and convince other people that it is true (including using their professional card).
This isn't true. You can ask it basic logic problems that it's never seen before, and it will apply the rules of logic to them. It can also identify correctly which rules of logic would make sense to apply in more complex situations, even when it doesn't get the answer right straight away. At doing this stuff GPT4 is better than GPT 3.5 which is better than previous GPTs. I fully expect that future models will be able to tackle more complex applications of logic to new domains successfully.
If you use only examples that weren't in its training set, you'll get to its limits quickly, but basic level first order logic is definitely within its ability.
It can reason but it has limited ability to transfer its reasoning skills into other domains.
For example, it can take code and add types to it. This involves a lot of reasoning ability. It can do this because it’s been trained on a vast amount of code. But it can’t yet fully transfer those reasoning abilities outside the narrow domain of code.
What is a domain? You talk of it like you know what it means. These things have no concept of a domain. If they use reasoning they use it all over the place. It is up to us to judge domainness.
Domain as in a context, the type of data that they were trained on. Code vs text. English vs Russian. Python vs APL.
For instance, OpenAI found that GPT4 is much better at reasoning in some human languages than others. It is best at reasoning in English, but struggles reasoning in less resourced languages.
There is clearly some context-independent reasoning going on (i.e. generalization) otherwise the model would not be able to reason at all in languages that it hasn’t seen a particular problem in. But there also appears to be a large context-dependent factor.
It is a side effect of human language and a huge corpus of text it was trained on. I know that people sometimes use this kind of reasoning, words have attached meaning and then you can do some simple reasoning. I would say though, that it is not a real reasoning. It can do some of the things real reasoning can do, but the same as with people it can get attached to the meaning of words too much. When a person can't go beyond that they are called lazy or sloppy thinkers. It is a heuristic, not a real reasoning. I think it is a cheaper processing and it is of course useful, but it has its limits. For me it will mean that ChatGPT will not get to see some patterns that in creative works would be obvious for a creative human.
As you said, when things are not in its training set it can struggle. If there is a plausible looking text for the question I asked it will give it to me, that's how it is designed. For example I asked it about Windows command line debugger - CDB. It gave me an example command line for it: cdb -c "your-app" -o "logfile". It is very wrong. -c requires an argument which are debugging commands to run on start, -o is to attach to all created attached processes. Real command line looks something like this: cdb -logo "logfile" "your-app" (and it still does not exactly behave as you would imagine having experience with Unix CLI). The problem ChatGPT has with CDB is probably, because it has much much bigger corpus on Unix-like command line tools and because the documentation for CDB is abysmal. From this ChatGPT session I would have more examples.
I'm not saying it is useless. It just is not designed to do that. It might improve or there might be an another algorithm needed on top or instead of what it does use.
For me it is like a kind of a step up from a search engine. It helps me to find something to start with. When I get to some details it is often wrong. I get a starting point from it and then find a proper source for the rest.
just like humans. I'm not saying chatgpt can do everything humans can and I'm not saying it's AGI, but it certainly is overlapping with what the human brain cam do
It’s a step towards something that can act like it reasons. I agree that many people perceive its capabilities inaccurately, but ChatGPT is an impressive machine. Just think how long it took us to evolve into being able to reason(billions of years). Here, we(humanity) went from first transistors to ChatGPT in, what, 100 years? The next 100 years might bring true AI.
In fiction it is often the case that an AI can reason better than humans do, but it doesn't understand emotions. But we now in general that emotions and reading of emotions is simpler than general problem solving. A child picks up on parent's emotions without extensive training. Animals can sense them. The fear response is a basic instinct. I would imagine it should be easier to make a machine being able to almost perfectly recognize emotions than general reasoning or this big heuristic machine which is GPT. I guess it all goes to the training set available.
It is very wrong to compare human creative process with a very elaborate text generator (AI).
If you write some simple code to generate art procedurally with randomness and it creates really beautiful pieces sometimes, was your software being creative?
I'd think if it had the ability to generate text as it does now, then iteratively apply the sorts of value judgments humans do on said output, and refine/discard/preserve accordingly, it wouldn't necessarily be so far from "genuine" creativity. Indeed maybe it does already do something like that.
I've played around a bit with getting it to write stories etc., and they do often seem quite "creative", the problem is eventually they often end up making little sense or contain fairly obvious contradictions or non-sequiturs in a way I wouldn't expect to see in the output of a typical human author (and certainly not a skilled writer). Indeed I'm not sure ChatGPT has the ability to formulate any sort of longer "story arc" within which to frame the text it generates. But I also suspect it will gradually be able to develop that capacity with future refinements.
I have positively had GPT-4 create genuinely creative answers. "More creative" is even listed as one of its traits in the ChatGPT model drop-down.
Ironically I think anyone who believes LLMs cannot be creative are themselves uncreative in their prompt crafting and use of the model, I've even seen this from senior developers and experienced writers. It's painful, like watching your mom try to find the downloads folder or enter terrible search engine keywords.
The technology provides very interesting and impressive results, but to me it seems people are interpreting very wrongly its underlying process, trying hard to see emerging consciousness and whatever any time the result doesn’t manifestly show that it’s not having such a characteristic.
I tested GPT on its ability to handle word creation with suffixes. Some results are really interesting, but sometime it pretends a word is a sample of some suffix use when the word doesn’t encompass it. I’m rather confident that this kind of artifact can be overcome, I won’t slander the technology because it still has some obvious weakness. But I’m afraid that once overcome, the lake of snake will convince most people all the more that this impressive technological achievement is more than what it is actually.
I think that better philosophers than us debated this question for thousands or years but I'll give my take. Suppose that creativity means to find a particular place in the n-dimensional space of all possible ideas. Artists find some interesting places in that space in part because they started from other interesting places nearby. They know those nearby places and we don't, because we trained to do a different job. However we have our share of creativity.
Algorithms like GPT or Stable Diffusion explore that n-dimensional space starting from possibly more places than a single human has experienced. They will miss many of the starting points that any human experiences, because of limits of those algorithms' interfaces with the world. However they can still find interesting places. Maybe it's not creativity as we mean it, but they can show us interesting places that no artist has found before.
Obviously we can say things like "What about a car that ran on pencils" but real, useful, entropy comes from inferences and abduction rather than the new.
What do humans who exhibit "creativity" typically do but rehash themes and ideas from others? Trends rather slowly evolve as they reverberate back and forth between thousands of humans -- whether in music, painting, clothing, books, memes, ...
If an artist just thinks of something completely new not inspired by others it only appeals to a small minority. It is a minor portion of human creativity, and avoided by perhaps most humans.
Nice one. Indeed there's tremendous opportunities for using GPT for deception purposes.
I did something similar but had it pretend to be a Windows machine, and for a specific type of corporation. The problem, or maybe opportunity, here is to have this happen live (as a user types commands in.) Currently the response rate is way too slow for that to happen, but probably will be mitigated as time passes.
I decided to give in to the hype and let GPT “help” me on a recent project doing some ansible automation. Instead of googling things or looking things up in the docs I asked chatgpt first. It was really nice putting in a question and getting back a direct answer, but it made up so much shit that looked like it should work, but was dead wrong. The worst one I can remember is I asked it how to extract a specific capture group when using the regex_search filter. It was adamant that I could do {{ value | regex_search(regex).group(1) }}. I was totally fooled and tinkered with this for an hour before realizing no such facility exists and it will never work.
Imagine this with a video gpt. You can click on things and get hallucinated responses back. It’ll require tons of compute but we’re probably only 5 years away from it
Well, the prompt says it all, fictional, simulate... No surprise that it "made things up". What I find more concerning is that it keeps making things up in rather concrete questions. I tried its map-reading capabilities today, asking it in what part of my town a certain street is, and what well-known things are nearby. It came up with a confidently-sounding answer. But without looking at a map, I immediately knew one of supposedly near-by things were actually on the other side of the town. So I let it know that I thought it was wrong. It appologized, and mentioned another near-by which was also wrong. Corrected it again, and it appologized again. At least it didnt make up something impossible the third time around... However, the confidence it implies combined with a high rate of hallucinations is quite worrying.
I am slowly coming to the conclusion that everything it tells you which you dont already know, is highly likely to be wrong. Question is, how useful it actually is, given that failure rate.
It feels like a person that really likes to talk, and has no regard for truth. If you catch an acquaintance of yours deliberately making things up just to sound eloquent, you'd put em into a "dont trust with anything"-category pretty quickly.
>I am slowly coming to the conclusion that everything it tells you which you dont already know, is highly likely to be wrong
It can also be wrong about the things you already know.
My rule of thumb is if the cost of generation + verification + estimated correction time is higher than cost of doing it your self - avoid AI.
So if you have stuff where you already know what needs to be done depending on how well it fits in the AI tool workflow, it might be a good choice. If not just don't bother.
206 comments
[ 2.6 ms ] story [ 231 ms ] threadIt was very unhappy about doing so; I had to stress that it was just for research in the prompt to avoid it flat out refusing to write self-modifying code. But it worked!
Hallucinating it would be if it came up with the concept all by itself. Like I asked where I could see the latest Avatar film and it said "easy, just go onto ftp.disney.com and download it".
The year is 2026
"hey, how can I watch 'avatar 3 - tokyo drift' today? it's not out yet." "easy, just go onto ftp.disney.com and download it". "really?? can you do it for me?" "yeah here it is"
suspiciously fast response arrives, with a video which is a movie about blue aliens that gpt6 has just asked midjourney 8 to create for it.
In a not so distant future you will create your own movies.
Or even better: There will be models teached on a show you really liked but which was cancelled and you will be able to continue it to eternity.
You can't take the sky from me~
"Firefly season 2 by rustyBones447 is the best one"
Not really challenges, but business models. There is no free lunch so to say. Maybe there will be highly personalized subscriptions that allow for hyperparameter tuning in this way.
I doubt this is something Netflix fears, it is something they gonna embrace. Computation of such movies won't be cheap.
A large part of value of movies, TV shows, books, videogames - even paintings and sculptures - is that they're part of external and shared reality. With self-generated movies (shows, books, games, ...), there will be no process of discovery to enjoy, no immersion to experience. There will be nothing to share or discuss with others, as everyone will have their own version of the story. There will be no canonical version to talk about - there will only be everyone's headcanon, all equivalent, all personal. All boring and worthless.
Now imagine if anyone could casually just fork a piece of writing and adapt to their personal tastes. "Like this, but make X be in love with Y instead of Z. Also make Z not really die but make a surprise comeback in the final chapter." It might be fun to play with such capability for a moment or two, but I can't imagine anyone enjoying exchanging such stories - if everyone can trivially change anything, then everything becomes arbitrary. There is no story, only everyone looking at their own selfie.
Incidentally, GPT-3.5 and GPT-4 already give us this capability with smaller pieces of fiction. You can literally alter any short story like I described above, today. Anyone can. And somehow, I don't see any takers. I think people realize there's no entertainment value in it. Hence, this is what might just save Hollywood et al.
Erotica is one exception to what I described above, as it's by nature the kind of fiction to be enjoyed personally, not in a wider context. And, of course, erotic story generators are as old as the Internet.
Writing, even prompting, is more work than reading, and it takes some of the fun out of it - plot twists are no fun if you told someone what they should be.
Plain ChatGPT still requires a bit back and force to get complex tasks going, but Auto-GPT already has a loop around it that makes it possible for GPT to feed into itself and perform complex tasks that are beyond a singular prompt line.
Even without that, ChatGPT can already provide quite serviceable short stories, with no effort on the user beyond "write a short story". The biggest practical show stopper right now is that ChatGPT has strict filtering, so the moment you have guns, action and death, the story just ends. BingChat is even more extreme with the filtering and will fail almost every time.
So no John Wick fanfic then
I don't think so. I think users will share what they've generated like they do now with midjourney images.
If I decided to "generate" a pilot episode of a show about robot ninjas fighting a secret war on mars, and it turns out really well, I'll share it on youtube. if enough people like it I might do ep2, or ep3. Then it becomes that shared reality.
Obviously if anyone can just say "gimme a show about robots on mars" instead of looking at what others have already created, the audience might not be as big, but no more than midjourney I'd imagine there will still be some talent and finesse involved in getting the prompts just right, and most people will be too lazy and prefer to browse what others have already done
Like it or not, on-demand fanfiction is where we're headed. It's not impossible that a canonical version of a story might be published though, like a video game. There's a main story about you saving the world from nuclear disaster, but if you spend 600 hours trying to romance the barfly in the first tavern and never fulfill your role, well, that's your story about a lecherous loser in the land of the lounge lizards, man.
This isn't anything new either; look at the Protestant Reformation. Everybody wanted a version of The Story that appealed to their own interests then too. Now you can pick a church whose ending you most prefer.
Would be awful.
The best movies I have seen I would not have been able to come up with.
https://i.ibb.co/XLFGdWp/DALL-E-2023-04-05-12-55-36-Mickey-M...
Mickey Mouse according to OpenAI is just a symptom, but the root of the issue is very deep if the people preparing the dataset intentionally decide to exclude examples due to commercial or political pressure.
https://preview.redd.it/n6wdyhyz9dra1.png?width=960&crop=sma...
https://preview.redd.it/9djj6tdflu3a1.png?width=1080&crop=sm...
https://www.reddit.com/r/midjourney/search/?q=mickey%20mouse...
And if we've learned anything about AI in the last 6 months we've learned:
1) AI is probably trained on data obtained without the creators permission
2) AI responses are very hard to control, and easily jailbroken
[1] https://petapixel.com/2023/04/03/midjourney-bans-ai-images-o...
He might just want to be able to visit china in the future
0: https://brlcad.org/
Wonder how much other open source software I am unaware of?
> that gpt6 has just asked midjourney 8
at the current rate gpt16 and midjourney --v 20 are more likely
I'll pay, just gimme a file to download
The truth is, we have no need for the notion as it applies to AI. It's just a catchy term that sounds great in the media. Nothing the algorithm spits out is a hallucination (or else ALL of it is) because, quite literally, It's all the same uncontrollable output.
It'd be as weird as saying your autocorrect feature "hallucinated" any time it presented the "wrong" word to you. (As if it's of clear mind the other times?!)
We speak of "hallucination" when GPT provides completions that are not connected to any facts it was "provided with" in either training data or prompts, but ultimately factuality makes no difference to GPT.
It ALWAYS just "simulates" text, that's what it does.
If you'd want to use these words to describe factual accuracy, or to express if GPT "performs" an action, as opposed to "faking" a result, you have to bear in mind either of these qualifications are nothing but a human interpretation of the completion.
It being "correct" more often than not with the content of it's simulation is just a logical, but ultimately accidental consequence of being so well fitted to simulate the structure of data it trained itself on.
And a "halucination" is nothing but this simulation deviating from the context you provided, that fundamentally was simulated by the same rules that produced any other content which didn't deviate.
I use ChatGPT daily, but always specify the information I want formatted. (In bullet points, a table or for example a news article.)
Of course, that’s only possible if the context content exists in the first place. If not, then it’s back to the old-fashioned technique of figuring things out for myself and writing about them.
Like converting a code sample from one language to another, or one format to another (e.g. raw css to object syntax), or complex replaces where the regex gets tricky- also great at writing simple docs.
Also use it for micro tutorials if it's on a topic older than a couple years old.
https://cloud.typingmind.com/share/dc9b6a87-cd00-43e9-a7ea-6...
The result I then could paste into a text editor, save as .ical, and import straight to my calendar.
How do I set up some backup infrastructure for my home server? How does Btrfs handle read errors? How do I cook X if I don't have ingredient Y? What's the correct syntax for this command? How to analyse and interpret these measurements from this vermicomposting experiment (helping my girlfriend with statistics)? How do noise cancelling headphones REALLY work? Is it okay to have my samba.conf set like this (it wasn't)?
Well that feeling doesn't exist with GPT-4. So far we've always been able to come up with something, together. If you don't like the first answer, you ask more questions. You can dig deeper. You can tell it what your guess is, what your intuition tells you about the problem, where exactly your uncertainty lies, e.g.:
Or
Or IMO if I'm not sure what the output should be GPT is less than worthless it's actually convincingly misleading.My only use case so far is stuff like "translate this model to openapi schema" because it was faster than setting up the tool, and similar tasks that are easy to verify and boring to type.
"Can Ronald Reagan be considered a liberal?
Ronald Reagan is generally considered a conservative politician. He was a member of the Republican Party and served as the 40th President of the United States from 1981 to 1989.
Reagan's political philosophy was based on a belief in limited government, individual freedom, and free markets. He advocated for a reduction in government regulation and taxes, increased defense spending, and a strong national defense. He was also a strong supporter of traditional social values, including opposition to abortion and same-sex marriage.
While Reagan did take some positions that could be considered more liberal, such as his support for amnesty for undocumented immigrants and his advocacy for the abolition of nuclear weapons, overall his policies and beliefs aligned more with conservative ideology. Therefore, he is not typically considered a liberal."
Now ask this to a human. You will get a polarized answer "No, Reagan is a conservative icon!!!!!!" lacking any nuance.
Depends on the human. Ask a political science professor and the result will probably be pretty similar, plus an earful on the evolution of the two major US political parties' positions since the 1980s and how that complicates these kinds of retrospective judgements.
A true hallucination would be, if I were to ask where can I get Disney songs, and it referred me to a non-existent FTP server.
It's entirely inaccurate...
It’s literally doing what it was asked to do.
Is "this word fits here" now synonymous to "fully aware"?
And like someone said before, perhaps we can think of the GPT models not so much as “artificial intelligence”, but more like a different kind of intelligence from what humans have. With strengths and weaknesses of its own.
You even ignored the quotes that OP put around ‘aware’.
As someone working with GPT-3 day-to-day at $dayjob, these words - usually reserved for sentient beings - are simply put typically the most apt to use in pretty much all contexts other than some sections of an academic paper. The fact that you yourself couldn’t get the sentence to work right is a testament to that.
Again, as someone that’s working with and having conversations about this day-to-day, we all find ourselves using these words, first kind of awkwardly, with laughs and air quotes, and we try to mentally reach for the best word to use. Eventually we all just know what we mean, and that’s that.
So, if I can ask directly, how do I talk about this online in a way that allows for productive conversations about the very obviously disruptive abilities of LLMs but that still avoids your snark?
The way it feels to me is that I have a fully formed concept almost immediately upon hearing or reading something then I employ language to try and convey the same concept into to someone else’s brain.
During that process additional ideas may occur to me, and that may augment the final output.
I experience some of what you describe, too, when dealing with ideas I already am familiar with, or explaining them to others. But when I'm trying to work through some novel thoughts, or am engaging in a conversation - including the one that has me writing this comment - I function in the way that's peculiarly similar to LLMs. That is, what pops into my head are words, phrases, sometimes entire sentences (often movie or videogame quotes). I pick one that fits best as an extension of what was already said - the previous words, the previous sentences, the response from the other party - and then repeat with the next completion, and the next, etc., until I feel it's time to stop.
I've joked for many years that, based on my personal experience and introspection, human brains at the highest level may just be glorified Markov chains. I sometimes still say that about LLMs - but I do so more as a compliment now. GPT-3 and GPT-4 may be just fancy autocomplete, but so are we. I feel we've brute-forced our way into a turning point in our understanding of what intelligence is and how it works.
Like I read your comment. I had thoughts during that process, but I read until the end, then I made a decision about what I thought, and that decision contained all the information that I want to convey in my response.
Now: when I am constructing my response, I am of course modulating which words might come next that would best fit into my language model to convey that concept, but the concept is an instantaneous understanding.
That instantaneous understanding may change during the process of articulation, but not fundamentally -- I think that any changes during the process of articulation could be considered "elaboration" rather than a reconceptualisation.
All the language that I just typed was designed to convey a concept that was more or less instantaneously formed and fully complete prior to my typing anything. The way in which I construct the language to articulate that concept may be very similar to the way in which a LLM constructs its responses, but I think the core difference is that original and instantaneous conceptual understanding of what it is trying to articulate.
Only moments before I write the word, do I know what that word will be, but I know the complete thought I want to convey. I don’t know if this is important, but my thoughts are not words, so I’m basically just describing them using words.
The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression is like an instantaneous graphical image of an elephant, that one must then put into words.
I think that a LLM doesn't have that. I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language. The concepts are all pre-existing in the same way that all the chess move possibilities pre-exist in the programming of a chess computer.
EDIT: In fact even the way I described concepts as "pre-existing" there might be giving it too much credit. Like the concept only arises on the receiving end, from a pool of concepts that were pre-existing but about which the LLM has no "awareness" whatsoever.
The question is, do you know its attributes before you describe them to yourself? To me, at least, an "elephant" feels like a pointer, a handle, a promise - I know it's a thing, but to know more than that, I have to dereference the pointer. And that usually means something that's getting close to stringing words together.
Possible confounder: I suffer from aphantasia[0]. I have nearly zero ability to perceive images in my mind, except when dreaming. Maybe it's easier to "know immediately what it [the elephant] is and all it's attributes", if it just pops into your head as a complete image. I, unfortunately, don't experience that.
> The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression (...)
> I think that a LLM doesn't have that.
Crazy idea: what if a concept is... a partial set of coordinates in the latent space? Enough to describe a pretty narrow region.
> I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language.
My tentative guess is that language will turn out to not be the fundamental part of what makes LLMs so powerful/spectacular. I think the magic is the latent space itself - the high-dimensional constructs where all kinds of associations end up represented as spatial distance. Language is just a serialization format[1] for latent space associations. By being fed lots and lots of text in training, LLMs build up the latent space, piece by piece, until there's enough dimensions to it that it is no longer a simple mapping between words learned from the training data. I think this is how understanding and concepts look like.
Also, I think it takes so much input text for LLMs to get to their current performance, because LLMs learn like babies - they soak up everything they can, letting the inputs self-arrange by correlations. We learn to understand what an elephant is by hearing stories about it, seeing it alive, and/or being told facts about it in a structured way (i.e. school). LLMs learn what elephant is by soaking up so much text that (the token representing) the word "elephant" is, in the latent space, close to all the words people used to describe its attributes - which themselves are close to other words with whatever various meanings and aspects we relate to those attributes, etc.
EDIT:
And connecting the two parts of the comment: LLMs are AI's poster child of aphantasia. If they have concepts, how else could they work with them if not by... articulating them? And when they articulate a concept, what makes us so sure they're just parroting associations, and not truly having a concept?
--
[0] - https://en.wikipedia.org/wiki/Aphantasia.
[1] - Quite literally, as communicating via any language happens via ingesting or producing a series of some kind of tokens (letters, words, ideographs, etc.) over time.
I recognize that mode. I have it to. I had it now; this comment is aiming to transmit a thought that came to me fully formed while reading your response. That said, it's a bit suspect how it popped up as if my mind was doing an associative search. It's also interesting that, however fully formed the thought might have been, it's also small, as if a conceptual token, and I usually have to run further associative lookups to extend it before the comment (or train of thought) feels satisfactory to me.
In the previous paragraph, the last two sentences were not part of expressing the fully-formed thought described earlier. I'm not lying to you, it's just I didn't expect to write the second half of the previous paragraph (and this paragraph) when I was writing the first one; they came from conceptual autocomplete.
> That instantaneous understanding may change during the process of articulation, but not fundamentally
For me, it quite often does. I rely on it happening often. Example cases:
- On HN, I post maybe 1/3 of the comments I write. 2/3 of the time, the act of trying to express my thoughts makes me realize that a) I'm wrong, and/or b) it's a bad idea to continue (e.g. I'm just procrastinating, or posting something impolite, or not useful, etc.). Of the 1/3 that survive long enough to be posted, I still delete some - as sometimes the realization from the previous sentence hits me only a couple seconds after posting.
- When facing difficult problems, I do a lot of internal monologue. In it, exploring whether certain phrases feel right, and kind of sculpting the chain of thought, is a critical part of the process.
- Because I often feel I lack the working memory to keep enough thoughts in flight, I often "talk" with myself via a text file - externalizing my internal monologue. That process also involves sculpting sentences, paragraphs or bullet points until they feel right, so it's arguably still articulation. And the more I think about it, the more this looks to me like the kind of tricks you do to work around the token limit / window of GPT-3 / GPT-4...
EDIT:
Ultimately, I think we're both doing thinking in similar ways, perhaps with different proportions between modes, and different conception of it.
One relevant conceptualization I have that I remembered just now: in conversational situations, I often feel like my subconsciousness is a Markov chain / LLM autocomplete, spitting out phrases, and my consciousness is busy judging them, and choosing whether or not to pass them to the actuators (speech organs, or fingers on the keyboard).
The two "models" - the autocomplete and the censor - is something I can clearly see in my mind when I'm processing a problem in a calm manner, or participating in a slow-paced discussion. But I've noticed that the more excited I get (or intoxicated), the weaker the "censor" part is; in the limit (when I'm really excited, angry, or drunk), I feel as if the Markov chain bit is feeding phrases straight to my mouth/fingers, cutting out the middleman of consciousness.
(Of course, what I wrote is all "how the algorithm feels from inside", so it's not very objective or reliable...)
Or at least I used to, because I only recently realized that, over the last few years, I've mostly switched to thinking in English only. I'm now in a weird state: my native language (Polish) is still easier for me to use, but also using it feels wrong somehow. Often enough, when talking to people around me, I notice I'm translating my own thoughts from English.
It might be that this process of learning and internalizing a second language, to the point it eventually took over my first one, is what changed my experience of the thinking process itself.
Ultimately, I think the definite answer would require solving the halting problem. The approximate answer we should be able to get when we let models interact with some complex environment over time. That is, the model predicts the next token, which may cause some change of a shared state, and then the model receives a response.
This seems like an answer to the question, "why is the LLM so compelling?" And not "what makes something conscious/aware?" Which is an important question but feels like not quite the right one to ask and then come to this conclusion. And it bothers me if I try to accept an idea where I don't know what conditions I'd reject it (eg, I don't know what evidence would cause me to reject this hypothesis, I can't see a test I can perform to next reject it).
ChatGPT 'predicts the next token' so much better than other 'next token predictors' that using that as the comparison for a low effort dismissal is like saying "what's to stop a human tracking a criminal by scent, humans have noses and dogs have noses so they must be the same".
If we agree that Markov chains aren't aware, then surely we also agree this definition of what makes them aware is either incomplete or incorrect.
Predicting the next token turns out to be quite profound, I'm just not convinced, at this time, it's sufficient to be aware of conscious. If you have a reason I should believe it, I'm all ears.
If you want to test how good a chatbot is at predicting tokens, well, you just have to test it. You can throw a Winograd schema challenge at it, or let it take part in code competition or exams, let it solve puzzles and all that. Quick simple test will for example show you that ChatGPT isn't any good at calculating, as the answers will just be wrong.
As for awareness, we know what ChatGPT is aware of: Its 4000 token history and whatever information it learned from the training data. That's the grand total of its existence, it doesn't get any other input, it doesn't perceive time, it doesn't learn, that history is all there is to it. And we can call that awareness as ChatGPT is able to perform correct prediction based on that token history.
For consciousness there are still a lot of basic parts missing, such as the perception of time, senses to act with some external world, the ability to think for itself without getting explicitly prompted and all that.
I think you and I are generally in agreement about consciousness and awareness. I think consciousness is an interaction of an agent with it's environment, rather than an agent in isolation. I guess that's why I find this attitude that consciousness is a matter of feeding enough data into a static training process confusing. I think it's a kind of is-ought problem where I ask about the "ought" and people express the "is" and we end up talking past each other.
But I don't really have a good hypothesis to disprove this either, so I'll have to meditate on that. I think it may be testable but it's challenging to model.
I wouldn't buy it, but you can sell it to plenty of people.
Because GPT often tells me to use programming libraries that I can find no record of anywhere and generates URLs of references that I can also find no reference of ever existing.
Transcript:
Can I give you a document and ask for a summary?
Yes, you can give me a document and ask for a summary. However, please note that as an AI language model, my ability to summarize a document may depend on the length and complexity of the document, as well as the quality and clarity of the writing. Additionally, please be aware that I am an automated system and may not always capture the main points or nuances of a document as accurately as a human reader.
How do I give you the document?
You can provide me with the document by copying and pasting the text into our conversation. Alternatively, you can upload the document to a cloud storage service such as Google Drive or Dropbox, and then share the link with me so that I can access and read the document. Please note that I will only be able to access public documents or those for which you provide me with appropriate access.
How do I share a document on a google drive with you?
To share a document on Google Drive with me, please follow these steps:
Once you have shared the document, I will be able to access it and provide a summary if you request it.I have uploaded a document for you to summarize https://drive.google.com/file/d/...(redacted)
I'm sorry, but as an AI language model, I do not have access to external sources or links. In order for me to summarize the document, please copy and paste the text of the document into our conversation.
GPT responds that there is an upload button because a lot of interfaces are described that way around the web -- there is not likely a hard reference for its answer in regard to its own interface.
I also had it suggest there was emailing capability in the playground interface, or it would say that it emailed me something I asked for. It would claim to have abilities to do a bunch of stuff like this that it couldn't actually.
Also recently discovered it hallucinates dates and times and has 100% confidence they're correct
Confabulation adding data which was not provided. Example making up the plot of an episode.
Hallucination to claim stimuli which was not provided. Claiming my prompt included something it did not.
Delusions refusing to admit an error or mistake and the best thing to do is reset the history and explicitly exclude the topic. Examples accusing me of spreading harmful disinformation when I brought up Kirstie Alley's death.
Often it's mistaken and it's none of those, you can simply correct it or direct it better.
I've experienced both, and deal with both in different ways.
And I never seen lower entry barriers improve quality of systems. Also, I don't think generating code in a probabilistic manner is a win.
I'm not in a camp. What is it with current discourse that only sees adversary teams as valid positions?
Great counter-point. It is a false dichotomy to view this as a binary matter (terrible pun not intended).
During the information superhighway era of the nascent internet, opinions on what would become varied as well (I recall with a bit less sarcasm and absolutist thinking).[1]
The hype was not widespread, and even though some prognostications were fairly accurate in broadly outlining societal and economic benefits, predicting the downsides was an oversight.
Maybe failing to foresee these difficulties in the past has left some people jaded? Could this be driving much of the cynicism and skepticism in the great AI debate?
[1] https://en.wikipedia.org/wiki/Information_superhighway
I think you have to separate what's hype from what's not. I personally see three things here:
- The usual hype, now AI, previously Bitcoin/NFT, some time earlier it was "sharing economy"[0] / "gig economy", earlier it was the cloud. Some things change, some stay the same, but there's little point paying attention to it unless you're in for a quick buck - for the shrewd, that hype is effectively a pyramid scheme with extra steps.
- The fear of losing your job to the AI. This, IMHO, is a quite realistic fear - more so than ever - because GPT-3.5 is already[1] good enough to replace plenty of jobs. It's just the market hasn't caught up to it yet. But it will soon enough.
- GAI, consciousness. There's no hype there. It may seem there is, because you're part of the small group that cares and can productively discuss the topic; for most people, it flies entirely over their heads. On that topic, however, I feel quite strongly[2] that LLMs are a major breakthrough in understanding the nature of intelligence. Disregarding the language model aspects, whatever is going on in the latent space, it seems unexpectedly powerful.
My speculative take: it might be that intelligence is mostly a natural consequence of building a high-dimensional association map between sensory inputs. It would definitely make sense in the context of evolution, and the spectrum of intelligence that various forms of life fall on. Perhaps we didn't realize it before because we can't imagine high-dimensional spaces all that well.
Even if my hypothesis turns out to be false, I can't imagine something akin to LLMs, with high-dimensional latent space, not becoming one of the fundamental concepts in AI and cognitive science. I expect we'll find examples that fit this model in nature. I expect we'll find that a part of our brains fits that model too.
But back to the point, the GAI/cognitive/consciousness aspects are hardly hyped. They're just becoming hot again among the small crowd that is interested in this. As you say, it's usually that hype follows real change. I imagine if LLMs end up shedding light on the topic of consciousness, both hype and hate will follow, particularly from the world's religions.
--
[0] - $deity knows how crazy it was to hear the excitement about "shareconomy" coming from the startup/entrepreneurial crowd, seemingly oblivious to the fact that anything they do in this space will be abusing, exploiting and otherwise ruining it...
[1] - Consider all the things people use ChatGPT for. All the things that you may be using it for. There are plenty of jobs that are entirely contained within the "area of effectiveness" of ChatGPT. And I'm talking the GPT-3.5 ChatGPT here (sadly, I still don't have the API keys to GPT-4 :( ).
[2] - I'll give it 90% confidence, though I'm not exactly well calibrated.
What if consciousness actually is the high dimensional entity that can map to sensory input and motor output of neural networks? Then expecting it to emerge from highly manipulated memory spaces would be like shocking cadaver brains with lighting to get a person... Physics is looking more and more like it cannot actually describe an objective reality that is perceived, pointing more to reality being the descriptions that we can build of experience in high dimensional spaces.
But speculation apart, it doesn't much matter how we think of chatbots or what they can do. Ultimately all AI/ML stuff is just up front compute stored in models, and they are just forms of capital. They will be used as other forms of capital.
To think that we can build people surrogates because chatbots are getting impressive is... sales hype pure and simple. Much like thinking you don't need servers because you use "the cloud". You're just outsourcing engineering, something that has been done for centuries.
It's already useful to me as my job involves writing reports to various audiences.
Taking to a computer(programming) is like using a knife to undo screws. It works mostly but you need to be careful.
Maybe we should petition dang to allow memes to be posted and get it over with.
Allegedly, HN is a place to hold curious conversation. What is curious in attempting to explain a comment as coming from a certain "camp"? Isn't that just going to polarise conversations and shut down any curious discussion?
> You can be in many camps at once, there is more than two camps and they don't have to be necessarily adversarial towards each other either. However, if you think that LLMs are just stochastic parrots then that probably guides a lot of other arguments and lines of thinking.
We are a long ways off from a flame war and 4chan here.
Making this sort of assumption about your interlocutor deprives the discussion of any kind of interest it would have. It's like we have seen that match before and we know what each side is going to say, so what's the point in debating anything? Everyone's convinced, everyone's stuck in their little corner, and calling each other group-names.
More to the point, it makes people reply not to each other, but to the views they assume the other person holds, which never leads to productive conversation, only to confusion and chaos.
There has to be a way to debate that doesn't involve picking an enemy to fight.
Seriously though, "stochastic parrot" is not a 4chan type of meme, it's the "industry term of art" kind of meme.
it's been used 6 times on /g/ and almost all are mocking the term -- with one calling people stochastic parrots (haha)
Meanwhile it's been cited 1200 or so times.
>> "Camp stochastic parrot"? HN is looking more and more like 4chan.
(https://news.ycombinator.com/item?id=35451667)
Then I clarified that:
>> I'm talking about your trying to split comments into camps.
(https://news.ycombinator.com/item?id=35451944)
Not that "stochastic parrots" is a 4chan thing. The polarised debate with "us" vs. "them" is what reminds me of 4chan. And not just 4chan, it's how people generally behave on the internet: it's always "us" vs "them" and there's no space for interesting conversation.
Extreme Polarization in debate FAR easily predates imageboards and even USENET, vi vs emacs probably harkens back to some of the earliest messages on DARPANET (ok, probably the text editor wars were close to a decade later, because i think it was more like the "ed" vs "god do we have anything else" era, but you get the idea -- i'm sadly not old enough to know the details of this golden era)
or look upon ye olde days of Slashdot trolling (some netcraftian oracles say that BSD is still, indeed, dying)
if you really do want to track the modern internet manifestation back you have to go back before 4chan. We'd have to talk about SA and GBS vs FYAD, and some of the earlier predecessors of SA such as.... Maddox.. and others, i guess? I can't think of other sites, only other names.
In any case - don't give 4chan credit where it's undue. Us vs Them goes back a long, long while, perhaps even pre-internet!
Anyway, what's with the assumptions? "Lad"? A lad named YeGoblynQueenne? Not that there isn't precedent for that sort of thing [1], but, let's not jump to conclusions.
______________
[1] https://www.youtube.com/watch?v=WOHPuY88Ry4
it's been used 6 times on /g/ -- 3/6 are mocking the term, 1/6 is calling people stochastic parrots (haha), and 2/6 are using it in the sense you're implying.
So not really a 4chan type thing. It's been cited by other papers apparently 200x more than it's been posted there.
Isn’t most creativity mashing things together that no one before thought of putting together?
I have to say that, although I still disagree with him, I was unable to get GPT to produce midjourney prompts which I would consider creative. Even when I prompted GPT to take the perspective of a known artist, it produced an endless series of banal landscapes, cliche compositions, overplayed metaphors, and insipid imagery.
Midjourney only really accels when a human is involved with the prompting.
I had to concede to him at the end of the discussion that /today's/ AI are not creative.
I firmly believe though that the inclusion of all mankind's perspectives possesses the raw material to generate creative ideas. My understanding is that current AI lack the ability to "turn off" the vanilla oversaturation of averages that pervade when you are exposed to all stimuli. One day, perhaps soon, there will be AI able to combine ideas they are exposed to without trying to combine ALL of the ideas at once.
What these models do is more similar to pastiche, if we really want to compare it to some technique.
But the value of the pastiche depends on the intent and its perception by the viewer(s), because there's nothing inherently original in it.
So basically what models produce has no value, unless we are able to attach some to it. [1]
If we asked chat-gpt to analyze something it has produced, it will probably say that it is "similar to" or "in resemblance of" but it's unlikely that it would say "this is the work of genius, how original! lovely!"
[1] edit: you don't read "pizza maker creates fabulous art on AI" but <person who's already in the business> won a contest of <some art form> submitting something created by <AI of your choosing>. Why? Because they know how to market it and can rely on other people believing that's their creation. Nobody says it upfront "I will submit an AI generated work", because they know it won't be judged the same way. The pizza maker was probably trying it for fun or to make a new logo for the pizza place or simply has no instrument to assign it a value and convince other people that it is true (including using their professional card).
When it is wrong it may correct itself, it may double down on being wrong and often just make something up again.
This isn't true. You can ask it basic logic problems that it's never seen before, and it will apply the rules of logic to them. It can also identify correctly which rules of logic would make sense to apply in more complex situations, even when it doesn't get the answer right straight away. At doing this stuff GPT4 is better than GPT 3.5 which is better than previous GPTs. I fully expect that future models will be able to tackle more complex applications of logic to new domains successfully.
If you use only examples that weren't in its training set, you'll get to its limits quickly, but basic level first order logic is definitely within its ability.
For example, it can take code and add types to it. This involves a lot of reasoning ability. It can do this because it’s been trained on a vast amount of code. But it can’t yet fully transfer those reasoning abilities outside the narrow domain of code.
For instance, OpenAI found that GPT4 is much better at reasoning in some human languages than others. It is best at reasoning in English, but struggles reasoning in less resourced languages.
There is clearly some context-independent reasoning going on (i.e. generalization) otherwise the model would not be able to reason at all in languages that it hasn’t seen a particular problem in. But there also appears to be a large context-dependent factor.
As you said, when things are not in its training set it can struggle. If there is a plausible looking text for the question I asked it will give it to me, that's how it is designed. For example I asked it about Windows command line debugger - CDB. It gave me an example command line for it: cdb -c "your-app" -o "logfile". It is very wrong. -c requires an argument which are debugging commands to run on start, -o is to attach to all created attached processes. Real command line looks something like this: cdb -logo "logfile" "your-app" (and it still does not exactly behave as you would imagine having experience with Unix CLI). The problem ChatGPT has with CDB is probably, because it has much much bigger corpus on Unix-like command line tools and because the documentation for CDB is abysmal. From this ChatGPT session I would have more examples.
I'm not saying it is useless. It just is not designed to do that. It might improve or there might be an another algorithm needed on top or instead of what it does use.
For me it is like a kind of a step up from a search engine. It helps me to find something to start with. When I get to some details it is often wrong. I get a starting point from it and then find a proper source for the rest.
In fiction it is often the case that an AI can reason better than humans do, but it doesn't understand emotions. But we now in general that emotions and reading of emotions is simpler than general problem solving. A child picks up on parent's emotions without extensive training. Animals can sense them. The fear response is a basic instinct. I would imagine it should be easier to make a machine being able to almost perfectly recognize emotions than general reasoning or this big heuristic machine which is GPT. I guess it all goes to the training set available.
If you write some simple code to generate art procedurally with randomness and it creates really beautiful pieces sometimes, was your software being creative?
I've played around a bit with getting it to write stories etc., and they do often seem quite "creative", the problem is eventually they often end up making little sense or contain fairly obvious contradictions or non-sequiturs in a way I wouldn't expect to see in the output of a typical human author (and certainly not a skilled writer). Indeed I'm not sure ChatGPT has the ability to formulate any sort of longer "story arc" within which to frame the text it generates. But I also suspect it will gradually be able to develop that capacity with future refinements.
Ironically I think anyone who believes LLMs cannot be creative are themselves uncreative in their prompt crafting and use of the model, I've even seen this from senior developers and experienced writers. It's painful, like watching your mom try to find the downloads folder or enter terrible search engine keywords.
I tested GPT on its ability to handle word creation with suffixes. Some results are really interesting, but sometime it pretends a word is a sample of some suffix use when the word doesn’t encompass it. I’m rather confident that this kind of artifact can be overcome, I won’t slander the technology because it still has some obvious weakness. But I’m afraid that once overcome, the lake of snake will convince most people all the more that this impressive technological achievement is more than what it is actually.
- Oscar Wilde
Algorithms like GPT or Stable Diffusion explore that n-dimensional space starting from possibly more places than a single human has experienced. They will miss many of the starting points that any human experiences, because of limits of those algorithms' interfaces with the world. However they can still find interesting places. Maybe it's not creativity as we mean it, but they can show us interesting places that no artist has found before.
Obviously we can say things like "What about a car that ran on pencils" but real, useful, entropy comes from inferences and abduction rather than the new.
If an artist just thinks of something completely new not inspired by others it only appeals to a small minority. It is a minor portion of human creativity, and avoided by perhaps most humans.
Maybe a real honeypot could send the attacker commands to the bot for an assessment. I wonder how it would compare to the existing assessment systems.
I did something similar but had it pretend to be a Windows machine, and for a specific type of corporation. The problem, or maybe opportunity, here is to have this happen live (as a user types commands in.) Currently the response rate is way too slow for that to happen, but probably will be mitigated as time passes.
We can do text, images, audio. What you want, smell?
I would LOVE to read the gpt version of Disney plots.
I am slowly coming to the conclusion that everything it tells you which you dont already know, is highly likely to be wrong. Question is, how useful it actually is, given that failure rate.
It feels like a person that really likes to talk, and has no regard for truth. If you catch an acquaintance of yours deliberately making things up just to sound eloquent, you'd put em into a "dont trust with anything"-category pretty quickly.
It can also be wrong about the things you already know.
My rule of thumb is if the cost of generation + verification + estimated correction time is higher than cost of doing it your self - avoid AI.
So if you have stuff where you already know what needs to be done depending on how well it fits in the AI tool workflow, it might be a good choice. If not just don't bother.
Amusingly enough, this is the exact strategy you should use for outsourcing development too.