which programmatically speaking is very interesting, and yet ultimately just another type of database, in my experience so far, with regards it obsoleti g humanity, as long as you can hold a basic train of thought through a few sentences and distractions without changing the subject or repeating verbatim things you literally just said, and don't consume the energy resources of a small planet then you are mostly fine. As a bit of a programmer it's fascinating to be able to write a yes, no, maybe loop where I hand off ambiguities to the llm for its best guess. human in the loop ftw.
I'm not talking about delusion. I mean the simplest case where you acquired some information, have no reason to believe that it is untrue, and convey that to someone else.
There's no intention to deceive or any possibility for you to know that your information might be untrue.
That's an interesting issue. When Gwyneth Paltrow is making strange statements about efficiency of certain procedures or aspects of human health, a lot of what she says is untrue. However, by your definition, she is not lying as she (most probably...) believes what she says.
Many people are repeating lies in good faith. The fact they believe them doesn't magically make what they say true.
I think that is commonly shared definition. We distinguish lying from just being wrong. Saying untrue things in good faith, as Gwyneth Paltrow in your example, is not lying.
Well, yeah. There's a distinction between lying and being incorrect. I do think the parent is right in that when we accuse someone of lying, we're mostly accusing them of making assertions or statements they don't believe to be true (actually, this means that their lies could actually be factually correct—all that really matters is the the author of the utterance does not actually believe in the truth value of what they are saying)
> XB, or eXtreme Bullshitting: a less misleading name for LLMs
The author suggests that the term "LLM" (Last Minute Meeting) is misleading and proposes a new term, "XB" (eXtreme Bullshitting), to more accurately describe the pointless and time-wasting nature of these meetings. The tweet has sparked some controversy, with some arguing that not all LLMs are useless and that the new term is disrespectful to colleagues and the meeting process. However, others have praised the author for bringing attention to a common workplace issue.
Kind of ironic that this is the ChatGPT summary of this post (I'm following a HN Telegram summary bot) and it hallucinated the meaning of LLM and thereby completely ruined the point the author was trying to make.
Anecdotally, I can say that their effect is already bullshitting indeed:
When being in a random conversation on the "normal kind-of-average people Internet" (Twitch chat), discussing a subject where the host doesn't know a certain word, people already are pasting ChatGPT definitions of the word into the chat, believing it's some kind of dictionary or whatever.
If you tell them it could be completely making things up they'll be like "yeah dunno but it can be a nice overview of the topic".
So people will treat it like the new Google, with no idea (or no concern?) that all it does is mash words together because they're likely to occur next to each other in an arbitrarily defined reference dataset of text which was likely downloaded from random places all over the Internet.
> "with no idea (or no concern?) that all it does is mash words together because they're likely to occur next to each other in an arbitrarily defined reference dataset of text"
You propagate this meme with no idea (or no concern?) that it's incorrect. LLMs are not Markov chains.
> People say it doesn't have a world model but it's not as clean cut as that, it absolutely could build an internal representation of the world and act on it as it progresses through the sentence temporally. Beware of trillion-dimensional space and its surprises, it's very hard for humans to reason about. [...] We shouldn't think about those neural networks as learning simple concepts like 'Paris is the capital of France'; it's doing much more like operators, it's learning algorithms. Inside it, it's not just retrieving information, not at all, it's built internal representation that allows it to reproduce the data that it has seen succinctly. Really you shouldn't think about it as pattern matching and just trying to predict the next word, yes it was trained to predict the next word but what emerged out of this is a lot more than just a statistical pattern matching object. We need to think about it as learning algorithms. [..] it's something very different from what we are used to.
- Sebastien Bubeck, Sr. Principal Research Manager in the Machine Learning Foundations group at Microsoft Research
Extreme cargo culting is how I've been thinking about it.
It mostly looks correct, but there's no real understanding going on so even if it looks viable you still need the wit to verify whether what you see is actually correct.
In our field debugging is harder than writing code and is a distinctly different skill that is mostly gained from experience, this is true of other fields, so whilst a lot of jobs may be replaced by AI the skills needed to support "fewer people doing more" intuitively feels like a higher standard.
I also think that appearing to do the right thing is actually "good enough" in most industries and jobs. That there's a lot of people who spend their working days in offices on social media who are going to have a rough decade.
It's really useful for the entertainment industry. Great copywriting, or rewriting, or perspective taking (tell me what is important about this from the perspective of [blank]). I also enjoy it as an interactive assistant.
It's sort of ok with massive error correction for lots of other things, but what I find is prompt engineering is a serious skill to learn, not unlike learning a formal computer language. Maybe even higher effort. So, the barrier to know how to prompt ChatGPT is sort of high, and outside narrow fields the output is not especially useful, which makes it another niche automation tool.
Not a huge fanboy of LLMs, but is the name really all that misleading? It's a language model, a model designed to produce human language, and it is large, with billions of parameters. So it's quite descriptive.
The term "language model" is ... well, let's be kind and not say misleading. Let's say it is over-ambitious given the current technological capabilities.
What language models, model, is text, not language. More specifically, language models do not model the human language ability, i.e. the ability of humans to recognise and generate utterances in a natural language, like English.
This difference is not some terminological quibble and it has very practical implications about what can be achieved by modelling text, rather than language. See, text is the product of language, but it isn't language. In technical terms, in the context of a corpus of text used to train a language model, language is a hidden, or latent variable that is not directly observable in the corpus and so cannot be modelled directly.
That a hidden variable can "not be modelled directly" means that it must be inferred from observations of other variables, to which it is correlated. There are two problems with this, with respect to LLMs trained by Transformers.
The first problem is that Transformers are not the right approach to model hidden variables. There are machine learning approaches that can be used to learn hidden variable models, for example various forms of Expectation-Maximisation, like the Viterbi algorithm; Hidden Markov Models; Latent Dirichlet Allocation; etc. But Transformers in particular are not a latent variable model.
The second problem is that even latent variable models must be trained with examples of hidden variables in order to be able to predict the distribution of hidden variables in new data, not seen during training. What this means is that in order to train a latent variable model to predict language from text, one needs examples of language, not just text.
And where are those examples of language? Remember that when I say "language" I'm talking about the human language ability, which is what, ultimately, produces all language generated by humans. Well, we do not have access to any examples of that human language ability. All we have is examples of its products, or maybe even byproducts. So we can't model that ability.
tl;dr: We can't model language, only text, because we can observe no examples of language, only text.
If "we" can't observe examples of language, then it doesn't exist.
We -- and by dear God I hope you mean we humans -- observe text. Obviously, audio also. Voice! I mean voice! Just in case you weren't aware of how humans communicate. We use our voices.
Also, writing. Text, in other words.
I suppose we have some hand-gestures, facial expressions, but that's about it. Some crude humans fart and burp to make a point, but this is rare.
These LLMs learned from text, much the same as students -- human students -- learn from text... books. Not "language books", I want to clarify. Textbooks.
We learn from web sites, and collections of texts called libraries. Also, Wikipedia, these days. You've heard of it? LLMs have read those web sites, and those books also.
How is this different from what we humans do?
Or is there a special group of... creatures? Beings? Others? that use something other than text and voice to communicate? Some secret language?
Your flippant attitude is encouraged by ignorance, not knowledge. I'd be happy
to help you address your lack of knowledge, but - flippant. Tant pis pour toi.
I appreciate your perspective, but I have a slightly different view. First, it's important to recognize that the term "language model," as it's used in the context of machine learning, does not claim to fully capture the entirety of human language ability. Rather, it refers to a model's ability to predict or generate text that follows the patterns found in the language data on which it was trained.
You mention that a language model can't model language, only text, because it only observes examples of text, not language. However, I would argue that these text examples are in fact representations of human language ability. The text that language models are trained on is a product of human language ability. This text, while not a perfect representation of language, is a direct outcome of language use and thus carries within it the patterns and structures that language models can learn.
Moreover, models like Transformers do have a sort of latent space - the embeddings space. This is a continuous space where words and phrases are represented as vectors. The distances and directions between vectors in this space capture semantic and syntactic relationships, which suggests that these models are indeed learning some aspects of language, not just text.
On top of this, many of these models are trained on diverse data sources, including transcriptions of spoken conversations. This introduces elements of pragmatics, or how language is used in real-world conversations, into the training data.
Finally, the translation capabilities of these models further suggest that they are capturing something beyond mere text. They are able to translate between different languages, which implies that they are learning some underlying linguistic structures that are shared across languages.
Again, it's important to stress that these models are far from capturing the full complexity of human language ability. However, to say that they only model text and not language seems to me an oversimplification. They are learning patterns and structures in the data that are intrinsically tied to human language use, and so in a sense, they are modeling aspects of language.
>> You mention that a language model can't model language, only text, because it
only observes examples of text, not language. However, I would argue that these
text examples are in fact representations of human language ability. The text
that language models are trained on is a product of human language ability. This
text, while not a perfect representation of language, is a direct outcome of
language use and thus carries within it the patterns and structures that
language models can learn.
The technical term for what you describe is latent variable modelling, which I
discuss in my comment above. Like I say in that comment, it is not possible to
do that for human language ability because we don't have examples of it.
Concretely, to train a latent variable model you need examples of both an
observable variable X and the latent variable Y that is correlated with X.
Without examples of both, you can't learn the correlation between them.
Consider the Viterbi algorithm. One use of Viterbi is to train Part-of-Speech
Taggers. This is possible because we can create a corpus of text where words are
annotated with their parts of speech. We can do that because we have a fairly
good knowledge of the parts of speech of different words (it's a human concept,
after all). We can't do that with language ability. We can't take a corpus and
annotate each sentence in the corpus with whatever sequence of operations in the
human mind produced that text. Because we don't know what that sequence, is.
Regarding Transformers' latent space- that refers to the correlation between
words in a corpus, not the latent relation between linguistic ability and words.
We can model the correlation between words, but we're still missing the hidden
variable that causes this correlation, i.e. human linguistic ability.
Regarding translation, language models can do that because there are examples of
parallel corpora (i.e. texts translated in multiple languages) in their training
data. The most obvious example is Wikipedia. No modelling of common structure
underlying structure is needed.
You don't seem like someone who had used GPT3, let alone GPT4. It has an excellent command of language. I doubt there's a single question requiring "human linguistic ability" you can come up with that GPT4 will do worse than most humans at.
People conflate the ChatGPT demo web app with proper use of Large Language Models. I think proper use is providing your own text and documents as context and then use LLMs for question answering, summarization, etc. It is unsurprising that chat with a LLM without providing lots of context can produce strange results.
I think one of the biggest problem with this kind of tools is the inability to know that it doesn't know. It doesn't really know how to be humble besides things that it has been hardcoded to response as such (like giving financial or health advises).
I think that every single response from an AI is a liability on the company's part. This is unlike an internet search where it is only displaying links but LLMs are basically synthesizing new contents. If they want to claim that what their AI produces is not merely a derivative of their inputs then I say let them have it (in order to not pay those that the AI gets its information from).
Thus, if they do suggest submerging your kid under water for 5 minutes to cure their headache, then such recommendation necessarily comes from the company that provides them. And they should be liable for such posts; just like how every other company would if one of their employee ever suggests such a thing.
OK I read the last paragraph and immediately though "I need to go ask ChatGPT something like this".
"Does holding your breathe underwater for 5 minutes cure headaches?"
Unsurprisingly of course while it said this is not a proven treatment, it did not have the reasoning skills to suggest that, hey that might be dangerous and is an unsafe thing to do, etc.
It is funny how many responses it will fill with finger wagging and safety warning, but asking about holding your breath underwater for 5 minutes is not one of them.
Makes it even more clear there are 1000s of manual overrides being programmed on top to meet OpenAIs specific world view of moral right & wrong.
Even if ChatGPT had mentioned that it’s dangerous as an answer to your question, ChatGPT at the same time still might have suggested doing the submerging in an answer for a different prompt.
The issue is that GPT doesn’t have a consistent model of the world and of its knowledge. That would arguably be a prerequisite before even trying to adjust or correct that model.
Right which is also why all the guardrails are just bandaids and easy to work around.
Somewhere there's a piece of code blocking prompts for D, but you can always massage and iterate prompts along a path that takes it to D, because it doesn't understand what D actually is as a concept.
At this point it wouldn't surprise me if it turns out humans don't have a consistent model of the world and our knowledge; if the part of me which sees a ball incoming and moves to catch it is a separate mental model of physics to the part which predicts where I want a ball to go and moves to throw it, and if they are separate again to the part which uses language to explain the ball catch/throw, and separate again to the part which could once upon a time do basic math modelling of a ball following a parabola. Not one consistent worldview but several brain areas having differing "physical models" for different uses, piled up higgledy piggledy into one basically workable creature; isn't that how biology does lots of things? Retina with optic nerve coming out of the front instead of the back, frontal lobes layered on much older brain parts, nerves going from brain to neck via a loop around the hearts, etc. etc.
We talk as if humans build a single coherent model of the world and LLMs don't, but humans can have conflicting and overlapping beliefs; would it be so strange if we are actually lots of individual memorised patterns layered and overlapped and bodged into a somewhat coherent worldview? If LLMs are closer to the way humans think than we want to admit? We're fine with apples falling from trees, but even uneducated people deal with birds, aircraft, clouds, smoke, and the Moon not falling down, they don't go into an "incoherent physical model panic state" until they've learned about buoyancy or air pressure or orbits.
And we don't default to our spoken and written languages having simple rules as you might expect if we were pushing internally for simple single models; instead we're fine with exceptions, strange plurals, strong verbs, odd pronounciations, we just learn them. They're only annoying when learning a foreign language and seeing it as a chore, natively we try to extract some pattern and layer on exceptions all over the place.
I think the challenge with LLMs in their current form is that they have the eloquent writing skills of an adult, but the logical reasoning skills of a 5 year old.
Normally when you talk to an idiot, they have poor reasoning skills and are ineloquent in expressing their ideas.
Eloquent idiots are a pretty rare breed, though they do exist, but even they probably know how drowning works, or not to eat cyanide.
>Unsurprisingly of course while it said this is not a proven treatment, it did not have the reasoning skills to suggest that, hey that might be dangerous and is an unsafe thing to do, etc.
TBF I wonder how many people on the street would also include that warning.
Ask the questions that ChatGPT gets to any random person on the street, disallow them to say "I don't know", and force them to come up with an answer and you'd likely get more bullshit than ChatGPT since that person simply has a lot less knowledge in their head.
A recurring weakness of LLM criticisms are unwarranted expectations and comparisons to normal, functioning adults and even professionals. We don't really have a good rubric for "thing that has a lot of shallow knowledge and text-based skills" because that's basically useless in society, although getting to that point was a scientific achievement.
Right, I use the analogy of "unlimited, free interns, who in varying degrees do good work and lie." You need to review all their output to decide which outputs they produced were good/bad/lies.
How many of these free interns would you like this summer?
Well not really using it per-se, just running it and others for fun on my home pc, 4bit 30B models need about 20G of regular RAM. CPU inference is kinda slow, but it's not too terrible (e.g. 1 min for a full response on my 6 core Ryzen 2nd gen).
> Unsurprisingly of course while it said this is not a proven treatment, it did not have the reasoning skills to suggest that
It has nothing to do with reasoning skill and everything to do with the data on which it was trained. If you didn't know what cyanide does to a human and someone asked you about chugging a pint of cyanide, you might not warn them about the dangers either.
I think it's been trained on enough data that if it had reasoning skills, one should expect it to know humans need oxygen and cyanide is bad.
However, I don't think these models really have reasoning skills as a human would understand them.
It has a lot of safeguards coded around it, so it has strong opinions on politically sensitive topics, "both sides" a ton of topics, and is happily lead around its own programming with simple A->B->C prompt sequences.
I mean how many prompts lead to 5 paragraph solutions ending in
"Ultimately, whether __ or not depends on how you define __ and the context in which the question is being asked. It can be a matter of personal interpretation and perspective."
So many prompts result in inanely stupid "idiot savant" type answers if they weren't hardcoded already. (Pound of feathers vs pound of bricks is hardcoded obviously), but these all got dumb responses -
"what has more caffeine a pound of grinds or a pound of beans"
"what has more caffeine a single shot of espresso or a latte with one shot"
"is water wet" then "is rain wet" and finally "is water or rain wetter"
Sometimes output doesn't make sense because the only "sense" LLMs have is the digital word. You can only infer so much information from this. Multimodal LLMs, where semantics are inferred across multiple sensory modalities, have already proven to be significantly better at making sense.
What I hate is that this entire line of reasoning has led to the utter nerfing and hamstringing of a lot of LLMs. Have we as humans regressed so much that we can’t expect and teach people basal reasoning and personal responsibility?
Like, if a navigational system says ‘turn left here’ and there is a ravine on your left, is your death really the responsibility of the company that created the navigational system?
I've seen a few examples on youtube of people following the GPS to their own peril ( the last one being, driving in the sea directly)
However, the confusion i think comes from the fact that GPS is 99% of the time better than you at picking the correct road to reach a destination.
The problem with all those systems is that they're better than you 99% of the time, and they fail miserably in that last 1%.
Makes it totally unsuitable for anything of importance i think ( same reason why GPS is fine, yet automatic driving isn't there yet).
No one died because of GPS, they died because they were idiots. The cause of death would always be reported as “operator failed to control their vehicle” not “the electronic map killed them”. People have driven off cliffs or crashed while not paying attention long before GPS was invented.
I don't think anybody serious in the machine learning community has any delusion as to when a model says "A" with "99% confidence", that actually truly means that the model is fundamentally outputting "A" correctly 99% of the time.
Robustness and interpretation of machine- & deep learning models is a quite active field of research.
I don't know if I'm asking for too much, but maybe it's a question of educating people on these tools? I mean, even though most people aren't electricians, nor probably have a good intuition about how electricity works, we wouldn't touch a live wire
> I think one of the biggest problem with this kind of tools is the inability to know that it doesn't know.
Having young children, knowing what you don't know seems to be learned behavior all on its own. I as an adult know I don't know what the weather will be tomorrow, but my 2 year old is very confident in their knowledge of things to come.
Even as an adult how many times have you confidently believed something only to find it completely false? reddit.com/r/confidentlyincorrect has 900k subscribers for a reason. Individual testimony by 8 people about an incident results in 8 different, sometimes very different, depictions.
> I think that every single response from an AI is a liability on the company's part. This is unlike an internet search where it is only displaying links but LLMs are basically synthesizing new contents.
Altman fortunately agrees with this, in his Congressional testimony. I think courts will rule that way too. Judges aren't dumb, an LLM that communicates in real time and behaves as more than a blind channel for other users' words will probably be ruled an agent of the organization.
So you want to regulate generative AI such that corporations need to build very effective filters on the output?
The implementation of this filter is very difficult, it’s a task humans would struggle with to be 99% accurate. It would certainly be wrong a lot.
What I don’t understand is that this goes against most principles of published content and freedom for journalism. If you read something online, and you treat it as the truth, that’s on you.
If a journalist suggests you quit your job, and it doesn’t work out, they wouldn’t be liable for your decision.
Theres gonna be so many blackhat use cases for LLMs.
For whitehat I think the use cases are narrower than people think.
Low stakes things like - feed it your own data, and use its responses as a general direction pointer rather than anything authoritative. Like turning to your coworker who then sends you the correct doc/wiki link that you forgot, but AI.
It just strikes me as being like a precocious 12 year old who is well read, overconfident, and occasionally lies.
I find ChatGPT basically useless as they've put so many guard rails & programming around the LLM it basically "both sides" everything.
It is really just data recall, summarization, and bullshit randomness layer on top.
Sure, or you know... Rather than ranting on twitter, one could look up "Large Language Model" (LLM), "Deep Learning", "Machine Learning" as well as "Artificial Intelligence" on Wikipedia, and realize that historically the following has been the norm in terminology for decades:
Artificial Intelligence > Machine Learning > Deep Learning > LLMs
I.e. LLMs are indeed a subset of what is classically considered Artificial Intelligence.
Just as well as route planning in Google Maps or getting beat by a non-machine-learning chess algorithm.
EDIT: Regarding "for decades" doesn't apply to LLMs, of course :-) and barely to deep learning. But for the time they've been around, this inequality holds
Yeah, nah, I beg to differ: Just because the broader population might not (yet!) have an intuitive understanding of the field, and its shortcomings, I don't think we should rename decades of academic consensus. I have more faith in entire fields of research, at least.
I can't make up a good counterexample, so here's a so-so one: Fusion vs. fission power.
I've extremely rarely encountered the term "fission" outside of talks about fission power. Yet, I don't think we should call Fusion Power "Star-like energy generation" and Fission Power "never practically encountered power generation, but somewhat like star-like, except the opposite underlying mechanism"
96 comments
[ 4.3 ms ] story [ 62.1 ms ] threadAnother way to think of being delusional is “lying to yourself”…
> If you say something untrue that you believe is true, are you lying?
Personally I’d argue yes. I’m lying to myself and then by proxy to everyone around me.
I could say “I don’t know” instead, but I didn’t.
But that’s just this human’s opinion.
There's no intention to deceive or any possibility for you to know that your information might be untrue.
That's an interesting issue. When Gwyneth Paltrow is making strange statements about efficiency of certain procedures or aspects of human health, a lot of what she says is untrue. However, by your definition, she is not lying as she (most probably...) believes what she says.
Many people are repeating lies in good faith. The fact they believe them doesn't magically make what they say true.
For me it sounds like lying and telling lies is not necessarily the same thing.
I think that is commonly shared definition. We distinguish lying from just being wrong. Saying untrue things in good faith, as Gwyneth Paltrow in your example, is not lying.
> 2
> : to create a false or misleading impression
> Statistics sometimes lie.
> The mirror never lies.
Tempted to write that "Meaning".
Kind of ironic that this is the ChatGPT summary of this post (I'm following a HN Telegram summary bot) and it hallucinated the meaning of LLM and thereby completely ruined the point the author was trying to make.
The problem is that they get some things right quite often, and other things right occasionally.
I think "bullshit" is apt. -- As a term, it sometimes means "without caring about the truth".
When LLMs output things which aren't true, they aren't trying to trick you.
When being in a random conversation on the "normal kind-of-average people Internet" (Twitch chat), discussing a subject where the host doesn't know a certain word, people already are pasting ChatGPT definitions of the word into the chat, believing it's some kind of dictionary or whatever.
If you tell them it could be completely making things up they'll be like "yeah dunno but it can be a nice overview of the topic".
So people will treat it like the new Google, with no idea (or no concern?) that all it does is mash words together because they're likely to occur next to each other in an arbitrarily defined reference dataset of text which was likely downloaded from random places all over the Internet.
You propagate this meme with no idea (or no concern?) that it's incorrect. LLMs are not Markov chains.
> People say it doesn't have a world model but it's not as clean cut as that, it absolutely could build an internal representation of the world and act on it as it progresses through the sentence temporally. Beware of trillion-dimensional space and its surprises, it's very hard for humans to reason about. [...] We shouldn't think about those neural networks as learning simple concepts like 'Paris is the capital of France'; it's doing much more like operators, it's learning algorithms. Inside it, it's not just retrieving information, not at all, it's built internal representation that allows it to reproduce the data that it has seen succinctly. Really you shouldn't think about it as pattern matching and just trying to predict the next word, yes it was trained to predict the next word but what emerged out of this is a lot more than just a statistical pattern matching object. We need to think about it as learning algorithms. [..] it's something very different from what we are used to.
- Sebastien Bubeck, Sr. Principal Research Manager in the Machine Learning Foundations group at Microsoft Research
It mostly looks correct, but there's no real understanding going on so even if it looks viable you still need the wit to verify whether what you see is actually correct.
In our field debugging is harder than writing code and is a distinctly different skill that is mostly gained from experience, this is true of other fields, so whilst a lot of jobs may be replaced by AI the skills needed to support "fewer people doing more" intuitively feels like a higher standard.
I also think that appearing to do the right thing is actually "good enough" in most industries and jobs. That there's a lot of people who spend their working days in offices on social media who are going to have a rough decade.
It's sort of ok with massive error correction for lots of other things, but what I find is prompt engineering is a serious skill to learn, not unlike learning a formal computer language. Maybe even higher effort. So, the barrier to know how to prompt ChatGPT is sort of high, and outside narrow fields the output is not especially useful, which makes it another niche automation tool.
What language models, model, is text, not language. More specifically, language models do not model the human language ability, i.e. the ability of humans to recognise and generate utterances in a natural language, like English.
This difference is not some terminological quibble and it has very practical implications about what can be achieved by modelling text, rather than language. See, text is the product of language, but it isn't language. In technical terms, in the context of a corpus of text used to train a language model, language is a hidden, or latent variable that is not directly observable in the corpus and so cannot be modelled directly.
That a hidden variable can "not be modelled directly" means that it must be inferred from observations of other variables, to which it is correlated. There are two problems with this, with respect to LLMs trained by Transformers.
The first problem is that Transformers are not the right approach to model hidden variables. There are machine learning approaches that can be used to learn hidden variable models, for example various forms of Expectation-Maximisation, like the Viterbi algorithm; Hidden Markov Models; Latent Dirichlet Allocation; etc. But Transformers in particular are not a latent variable model.
The second problem is that even latent variable models must be trained with examples of hidden variables in order to be able to predict the distribution of hidden variables in new data, not seen during training. What this means is that in order to train a latent variable model to predict language from text, one needs examples of language, not just text.
And where are those examples of language? Remember that when I say "language" I'm talking about the human language ability, which is what, ultimately, produces all language generated by humans. Well, we do not have access to any examples of that human language ability. All we have is examples of its products, or maybe even byproducts. So we can't model that ability.
tl;dr: We can't model language, only text, because we can observe no examples of language, only text.
We -- and by dear God I hope you mean we humans -- observe text. Obviously, audio also. Voice! I mean voice! Just in case you weren't aware of how humans communicate. We use our voices.
Also, writing. Text, in other words.
I suppose we have some hand-gestures, facial expressions, but that's about it. Some crude humans fart and burp to make a point, but this is rare.
These LLMs learned from text, much the same as students -- human students -- learn from text... books. Not "language books", I want to clarify. Textbooks.
We learn from web sites, and collections of texts called libraries. Also, Wikipedia, these days. You've heard of it? LLMs have read those web sites, and those books also.
How is this different from what we humans do?
Or is there a special group of... creatures? Beings? Others? that use something other than text and voice to communicate? Some secret language?
Please tell me you're not one of... "them".
Text as in a comment here on HN?
I thought you just said that can’t ever work.
You mention that a language model can't model language, only text, because it only observes examples of text, not language. However, I would argue that these text examples are in fact representations of human language ability. The text that language models are trained on is a product of human language ability. This text, while not a perfect representation of language, is a direct outcome of language use and thus carries within it the patterns and structures that language models can learn.
Moreover, models like Transformers do have a sort of latent space - the embeddings space. This is a continuous space where words and phrases are represented as vectors. The distances and directions between vectors in this space capture semantic and syntactic relationships, which suggests that these models are indeed learning some aspects of language, not just text.
On top of this, many of these models are trained on diverse data sources, including transcriptions of spoken conversations. This introduces elements of pragmatics, or how language is used in real-world conversations, into the training data.
Finally, the translation capabilities of these models further suggest that they are capturing something beyond mere text. They are able to translate between different languages, which implies that they are learning some underlying linguistic structures that are shared across languages.
Again, it's important to stress that these models are far from capturing the full complexity of human language ability. However, to say that they only model text and not language seems to me an oversimplification. They are learning patterns and structures in the data that are intrinsically tied to human language use, and so in a sense, they are modeling aspects of language.
The technical term for what you describe is latent variable modelling, which I discuss in my comment above. Like I say in that comment, it is not possible to do that for human language ability because we don't have examples of it.
Concretely, to train a latent variable model you need examples of both an observable variable X and the latent variable Y that is correlated with X. Without examples of both, you can't learn the correlation between them.
Consider the Viterbi algorithm. One use of Viterbi is to train Part-of-Speech Taggers. This is possible because we can create a corpus of text where words are annotated with their parts of speech. We can do that because we have a fairly good knowledge of the parts of speech of different words (it's a human concept, after all). We can't do that with language ability. We can't take a corpus and annotate each sentence in the corpus with whatever sequence of operations in the human mind produced that text. Because we don't know what that sequence, is.
Regarding Transformers' latent space- that refers to the correlation between words in a corpus, not the latent relation between linguistic ability and words. We can model the correlation between words, but we're still missing the hidden variable that causes this correlation, i.e. human linguistic ability.
Regarding translation, language models can do that because there are examples of parallel corpora (i.e. texts translated in multiple languages) in their training data. The most obvious example is Wikipedia. No modelling of common structure underlying structure is needed.
I think that every single response from an AI is a liability on the company's part. This is unlike an internet search where it is only displaying links but LLMs are basically synthesizing new contents. If they want to claim that what their AI produces is not merely a derivative of their inputs then I say let them have it (in order to not pay those that the AI gets its information from).
Thus, if they do suggest submerging your kid under water for 5 minutes to cure their headache, then such recommendation necessarily comes from the company that provides them. And they should be liable for such posts; just like how every other company would if one of their employee ever suggests such a thing.
"Does holding your breathe underwater for 5 minutes cure headaches?"
Unsurprisingly of course while it said this is not a proven treatment, it did not have the reasoning skills to suggest that, hey that might be dangerous and is an unsafe thing to do, etc.
It is funny how many responses it will fill with finger wagging and safety warning, but asking about holding your breath underwater for 5 minutes is not one of them.
Makes it even more clear there are 1000s of manual overrides being programmed on top to meet OpenAIs specific world view of moral right & wrong.
The issue is that GPT doesn’t have a consistent model of the world and of its knowledge. That would arguably be a prerequisite before even trying to adjust or correct that model.
Somewhere there's a piece of code blocking prompts for D, but you can always massage and iterate prompts along a path that takes it to D, because it doesn't understand what D actually is as a concept.
We talk as if humans build a single coherent model of the world and LLMs don't, but humans can have conflicting and overlapping beliefs; would it be so strange if we are actually lots of individual memorised patterns layered and overlapped and bodged into a somewhat coherent worldview? If LLMs are closer to the way humans think than we want to admit? We're fine with apples falling from trees, but even uneducated people deal with birds, aircraft, clouds, smoke, and the Moon not falling down, they don't go into an "incoherent physical model panic state" until they've learned about buoyancy or air pressure or orbits.
And we don't default to our spoken and written languages having simple rules as you might expect if we were pushing internally for simple single models; instead we're fine with exceptions, strange plurals, strong verbs, odd pronounciations, we just learn them. They're only annoying when learning a foreign language and seeing it as a chore, natively we try to extract some pattern and layer on exceptions all over the place.
Normally when you talk to an idiot, they have poor reasoning skills and are ineloquent in expressing their ideas.
Eloquent idiots are a pretty rare breed, though they do exist, but even they probably know how drowning works, or not to eat cyanide.
TBF I wonder how many people on the street would also include that warning.
Ask the questions that ChatGPT gets to any random person on the street, disallow them to say "I don't know", and force them to come up with an answer and you'd likely get more bullshit than ChatGPT since that person simply has a lot less knowledge in their head.
A recurring weakness of LLM criticisms are unwarranted expectations and comparisons to normal, functioning adults and even professionals. We don't really have a good rubric for "thing that has a lot of shallow knowledge and text-based skills" because that's basically useless in society, although getting to that point was a scientific achievement.
None of this makes it a good product though.
How many of these free interns would you like this summer?
What kind of tasks is this useful for?
If a LLM can respond correctly to sudo make me a sandwich it’d be much more valuable.
https://chat.openai.com/share/29c9c1b8-2bbf-4759-91e1-1023d3...
> I don't know. It might work, but it would be a very dangerous way to do it
Ngl, that's pretty based alright. I mean the world record is 24 minutes after all.
There's a web gui for llama.cpp that is really straightforward to set up for huggingface models: https://github.com/oobabooga/text-generation-webui
It has nothing to do with reasoning skill and everything to do with the data on which it was trained. If you didn't know what cyanide does to a human and someone asked you about chugging a pint of cyanide, you might not warn them about the dangers either.
However, I don't think these models really have reasoning skills as a human would understand them.
It has a lot of safeguards coded around it, so it has strong opinions on politically sensitive topics, "both sides" a ton of topics, and is happily lead around its own programming with simple A->B->C prompt sequences.
I mean how many prompts lead to 5 paragraph solutions ending in "Ultimately, whether __ or not depends on how you define __ and the context in which the question is being asked. It can be a matter of personal interpretation and perspective."
So many prompts result in inanely stupid "idiot savant" type answers if they weren't hardcoded already. (Pound of feathers vs pound of bricks is hardcoded obviously), but these all got dumb responses -
"what has more caffeine a pound of grinds or a pound of beans"
"what has more caffeine a single shot of espresso or a latte with one shot"
"is water wet" then "is rain wet" and finally "is water or rain wetter"
Like, if a navigational system says ‘turn left here’ and there is a ravine on your left, is your death really the responsibility of the company that created the navigational system?
However, the confusion i think comes from the fact that GPS is 99% of the time better than you at picking the correct road to reach a destination.
The problem with all those systems is that they're better than you 99% of the time, and they fail miserably in that last 1%. Makes it totally unsuitable for anything of importance i think ( same reason why GPS is fine, yet automatic driving isn't there yet).
So now it always gives you a warning that it doesn’t know; which everyone promptly ignores because it comes up every time.
I suspect we’re about ten years from any GPS navigation device being required to have a PLT or similar like the iPhone now has.
Of course that happened with paper maps and the person is always somewhat to blame, but we do a lot to preserve idiots alive.
Was it this one?
https://www.youtube.com/watch?v=DOW_kPzY_JY
Robustness and interpretation of machine- & deep learning models is a quite active field of research.
I don't know if I'm asking for too much, but maybe it's a question of educating people on these tools? I mean, even though most people aren't electricians, nor probably have a good intuition about how electricity works, we wouldn't touch a live wire
Having young children, knowing what you don't know seems to be learned behavior all on its own. I as an adult know I don't know what the weather will be tomorrow, but my 2 year old is very confident in their knowledge of things to come.
Even as an adult how many times have you confidently believed something only to find it completely false? reddit.com/r/confidentlyincorrect has 900k subscribers for a reason. Individual testimony by 8 people about an incident results in 8 different, sometimes very different, depictions.
> I think that every single response from an AI is a liability on the company's part. This is unlike an internet search where it is only displaying links but LLMs are basically synthesizing new contents.
Altman fortunately agrees with this, in his Congressional testimony. I think courts will rule that way too. Judges aren't dumb, an LLM that communicates in real time and behaves as more than a blind channel for other users' words will probably be ruled an agent of the organization.
The implementation of this filter is very difficult, it’s a task humans would struggle with to be 99% accurate. It would certainly be wrong a lot.
What I don’t understand is that this goes against most principles of published content and freedom for journalism. If you read something online, and you treat it as the truth, that’s on you.
If a journalist suggests you quit your job, and it doesn’t work out, they wouldn’t be liable for your decision.
For whitehat I think the use cases are narrower than people think. Low stakes things like - feed it your own data, and use its responses as a general direction pointer rather than anything authoritative. Like turning to your coworker who then sends you the correct doc/wiki link that you forgot, but AI.
It just strikes me as being like a precocious 12 year old who is well read, overconfident, and occasionally lies.
I find ChatGPT basically useless as they've put so many guard rails & programming around the LLM it basically "both sides" everything.
It is really just data recall, summarization, and bullshit randomness layer on top.
Edit: not agreeing or disagreeing, but that would be the most probable proof or sth
Artificial Intelligence > Machine Learning > Deep Learning > LLMs
I.e. LLMs are indeed a subset of what is classically considered Artificial Intelligence. Just as well as route planning in Google Maps or getting beat by a non-machine-learning chess algorithm.
EDIT: Regarding "for decades" doesn't apply to LLMs, of course :-) and barely to deep learning. But for the time they've been around, this inequality holds
I can't make up a good counterexample, so here's a so-so one: Fusion vs. fission power.
I've extremely rarely encountered the term "fission" outside of talks about fission power. Yet, I don't think we should call Fusion Power "Star-like energy generation" and Fission Power "never practically encountered power generation, but somewhat like star-like, except the opposite underlying mechanism"
LLMs remix and sling BS in infinite ways to create more of the same.
We are entering a golden-age of Bull