Rule #1 of reading CS papers - don't trust the anthropomorphized terms. Rule #2 - definitely don't extrapolate those terms to the real mind.
Of course, attention is almost certainly a heuristic readers use, but to even bring it up as key to intelligence begs so many questions about LLMs and the mind...
The problem lies more in #2 than #1. It's fine to repurpose words in new domains, but there's zero guarantee that translates back into anything useful about the original domain.
As a former perceptual neuroscientist, I can assure you that ML "attention" has very little to do with biological attention.
At the neuronal level, it involves secondary connections biasing receptive fields.
E.g., a high-level auditory neuron might primarily fire in response to high-pitched voices, but if you hear someone say your name in a high-pitched voice at a party, you can bias that neuron to try more selectively fire in response to just their voice, "tuning out" similarly-pitched voices.
That's because we're missing an abstraction, salience. LLMs and biological minds alike have salience functions that both use complex and seemingly inscrutable means to answer the question "why am i paying attention to this right now?" And those salience functions are tuned the same way, the way the Greeks worked out thousands of years ago and called logos, reason.
When you think about it that way, you can bridge the two types of cognition more usefully. The how of intelligence, what attention consists of, can remain completely different, while the why can feel so universal that humans can easily forget they're talking to a machine.
I'm skeptical it requires embracing new meanings, it only requires being unbiased (non human centric in thinking). If an organic looking alien landed on Earth, circa 1950, with the capabilities of whatever the cutting edge version of eg Mythos is, humanity at the time would have had absolutely zero problem recognizing it as de facto intelligent, even if it only responded when prompted during interactions. People would have merely categorized it as a different type of intelligence. Instead of embracing that intelligence can come in many forms, as sci-fi has proposed about the appearance of alien creatures for many decades, when confronted with the arrival of another type of intelligence, humanity becomes cowardly as it always does when its perceived specialness is challenged.
People are doing what people do: they're desperately moving the goal post to keep pretending humanity is particularly special and that intelligence is particularly special. Unique snowflakes and all that, we're the center of the universe. It's nothing more than elaborate human weakness & fragility on display.
We're in the first inning of this, intelligence is not going to end up being special, it's going to be trivial and low cost. See: Chinese models ten years from now.
Astra, Opus, Fable, Argon - they're all radically beyond the top 1% of humanity intellectually already, and it's going to get a lot worse quickly. The sole aspect the top 1% beat them on, and increasingly rarely, is at hyper specialization, where a person is elite at what they do. For $20-$100-$200 per month I can get access to an omnimath genius intellect far beyond any human that has ever existed, and that AI will work for me like a dog. Humanity should be scared, and the reason HN hates AI so much, is because it will wipe out very large numbers of their lucrative jobs that they imagined would form long-term careers (six figures, stock options, path to wealth, elitism re competition).
One of the things I hate most about modern computer science and programming is how everyone takes meaningful words and then completely distorts them into a completely unrelated idea or program name. It not only pollutes search for both the original and new topic, but it distorts peoples idea of what the new version means through the idea of it being some kind of analogy.
"Sentences with pronouns such as “it” in them illustrate the challenge that attention overcomes. Consider: “The animal didn’t cross the street because it was too tired.” And: “The animal didn’t cross the street because it was too wide.” To translate these sentences correctly into French, an AI model must determine that, in the first one, “it” refers to the animal, which is a masculine word in French, so the translation should use that form of the pronoun, il. But in the second sentence, “it” refers to the street, a feminine word that requires a different pronoun, elle. How can the machine figure this out?"
"Bahdanau remembered doing translation exercises when he was learning English in school. To write each next word of his translated sentence, he’d have to glance all around the source text to figure out what should come next. “Literally you have your source sentence on the left and your writing on the right, and you’ll be, like, turning your head, left-right, left-right,” he said. “That process is what I expressed as a neural mechanism.”
That simplicity seems to me the real tell that this is the real insight. The author does a good job of walking the ready past a long list of distractors and conjuring a sense of a path to an insight, and then, there's the insight - a simple encoding of a simple task: "What is the next word? How do I figure that out? Left-right. Left-right"
This is a really neat, approachable way to explain the concept of attention in plain words. Thanks for giving us the reference, though in my understanding the formulation in the first paragraph comes from Natalie Wolchover, the columnist, rather than Bahdanau.
An IMO under-appreciated aspect of attention is that it's an ingredient for building a Turing-complete evaluator, since (for example) it lets you implement mutable RAM by treating the attended data as a write log: https://yager.io/neural/turing/turing.html
The answer is clearly "no." The fact that neural networks can decipher patterns in our language without knowing what the sentence is about or even what the individual words mean makes it obvious that the entire A.I. "boom" is based on quicksand. These A.I. companies may sprinkle in some stuff that makes it look intelligent, but the current transformer/attention system will only get us so far.
As an NLP person it's very hard for me to endorse this explanation, because the transformer as it is described here simply does not work until you add the most important ingredient of all: the positional encoding.
Most of the semantics of a sentence are indicated by the precise order the tokens appear. This is easiest to see in a language like English, since it is visible in two very prominent ways.
First, the morphology is fairly derivational, meaning you can keep adding suffixes to things and English speakers will generally get what you mean. For example: derivation-al-ish-ly. You've probably never heard this specific derivation, but you probably have some idea of what it means anyway. The transformer can only capture these semantics if it has some representation of the fact that the suffixes inflect the meaning of the word's root (presumably something like `deriv`). English encodes this entirely by the position of the suffixes relative to the root. Without the positional information, there is no way to recover it, and sentences would lose much of their meaning, since English depends heavily on this construction.
Second, English has relatively few cases, and thus relies very heavily on the order of words to indicate what is the subject and what is the object. English specifically is Subject-Verb-Object, and the instinct of native English speakers to use SVO ordering is so strong, you will find them inserting nouns and pronouns into sentences that do not actually have subjects at all. "It's raining outside" is the most famous example. This sentence has no subject at all, and yet native English speakers feel a strong need to place "it's" at the beginning.
If the transformer did not have positional encodings it would be completely hopeless to recover what is the subject and what is the object from English sentences. There simply is no other way to reliably tell, other than inspecting the word positions.
It's worth noting we also do have the counterfactual. Generative language models that explicitly exclude positional information, like Latent Dirichlet Allocation, are only able to (essentially) cluster word co-occurrences. That is not as useless as it sounds—if I say "fire house burning ambulence" you kind of get the point even without a subject or an object—but it certainly does not generate the lucid, provoking sentences that modern LLMs do.
> "It's raining outside" is the most famous example. This sentence has no subject at all, and yet native English speakers feel a strong need to place "it's" at the beginning.
Italian, Spanish and Latin (for example) can generally avoid to express explicitly the subject in a dedicated pronoun, as the verb already encode the it through specific flection. In contrast in French and German only in imperative tense one can formally make such an ellipsis (colloquially of course one can encounter more nuances).
The sentence does have a very obvious subject. The environment is producing rain. The subject is the environment and its condition is being elaborated. It = environment.
The rain is not magical, the environment is producing it. The environment is raining. It's raining outside means the environment is producing rain outside.
Who could possibly confuse that as lacking a subject.
And with this reply we begin one of the permanent debates on this subject. Regardless of where anyone here comes down on this particular case, "it" is an indisputable truth that not all English sentences have subjects. Yet, many of them have pronouns where a subject should be. Maybe the famous example isn't one of them, but "it" is not at all hard to come up with a sentence where that is true. :)
French is interesting because it evolved this so-called "dummy subject" pattern (which is actually called "subject-prominence") for an entirely different reason than English.
French is descended from Latin, which is "pro-drop" in the sense that you'd drop the "dummy subject" and simply write "pluit." Later, Old French, like English, lost most of its cases. Because of that, many word endings started sounding very similar (a problem known as "agreement syncretization"), and all of that seems to correspond in the record with the pro-drop "null" subject.
This is an entirely different path than English took, and it's also not a path any other Romance language took. Spanish isn't even subject-prominent. It fully allows the "null" construction, e.g., "llueve", which literally is just "rains."
In Ukrainian it’s «дощить» (from “дощ” — rain) — the form of the word tells you its function in the sentence. Same happens in other Slavic languages: Polish, Belarusian, Slovakian.
Latin languages rely on the order of words to explain their function.
So much so, English speakers don’t say “blue big ball” in German, time before place is mandated.
>Most of the semantics of a sentence are indicated by the precise order the tokens appear.
I would rather say it’s derived from contextual element. And in linguistic context which encodes more interpretation cues, order of morphemes drop symmetrically in semantical importance, though it can still affect interpretation on pragmatic level. Latin for example already offers more flexibility regarding word order, though it’s used mostly in poetry/stylised-rhetoric. And as German show having the case encoded in the nouns don’t even prevent mandatory rigidity in sentence structure.
It's hard to speak for every language on Earth, but generally languages that have less flexible word orderings will have correspondingly more complicated morphology. In those cases, order does obviously still matter, but instead of in word orderings, it would be in things like prefixes and suffixes.
There isn't really a way to get around this. Natural language IS structured as a recursive tree. Our job is to extract this tree from a flat stream of tokens (or similar), and straightforwardly the only way to do that is to make the ordering of the tokens meaningful. There is no other option! This is how 100% of parsing works in any context whatsoever, both theoretically and practically.
There is nothing like a valid criterion to discriminate "natural" in languages. All human languages are praxeostratic, they result from the cumulation of practices through time. Some might be more or less blank slate founded through theoretical jump, but even burning the bridge doesn’t mean creation ex nihilo. I mean, the only thing that would not be "natural", would obviously be "supernatural", like if one pretend a language to be some divine creation and not result of human contingent existence.
And just because utterances can be more or less matched with recursive tree doesn’t mean there is an ontological absolute truth about language there. Even "language" refers to some vague abstraction. Just like the "it" in "it rains" mentioned in this discussion: no entity out there actually respond to "language".
At introspection level, I absolutely don’t interpret utterances like recursive tree. They are more like cues going through living graphs more or less already connected and being able to change topology on the go. Impermanence of interpretation results is the only constant.
Now it’s not to dismiss that some other people might feel like "Natural language IS structured as a recursive tree" is the only way. People have different perspective, different brains, different life experiences. One who dwell in aphantasia¹ don’t perceive the world the same way as people that have hyperphantasia². Different of course here doesn’t mean better/worst or superior/inferior.
¹ A word I discovered in some HN comment
² A word that I had never red before but just popped up in my mind as I was thinking about this sentence, but which is actually already is in use in literature apparently: https://en.wikipedia.org/wiki/Hyperphantasia
A tree is a graph so I’m not sure I understand your idea. But, at the end of the day, the point of the tree is just to represent the relationships between the tokens. There IS a relationship. As long as your theory represents that as precisely and with as much predictive power as trad linguistics, it doesn’t really matter how.
I’m also not sure why you are talking about why natural languages are not different from other ones. That is what I’m saying, it’s the reason behind the comment that all parsing works the same way.
But positional encoding by itself also won't work nearly as well.
And if you look at how transformers work you will immediately realize there's nothing a transformer can do that an RNN cannot do, and frankly that an RNN cannot do 1000x more efficiently at inference time. It's just that training even the best known RNNs is going to take a lot longer because it can't be parallelized. It cannot train on 100 GPUs, in fact it cannot really use a GPUs advantage.
Plus equivalence proofs will get you a lot further than RNNs too. Kernel methods will obviously work as well, for example.
This has been addressed several times now in the peer comments to this one. I am addressing the original formulation of the problem because the article is about the original formulation. There are other options available now, including causal masking, but in the original formulation positional encodings are THE most important single ingredient in success, because many aspects of semantics cannot be recovered from anything except position. One example is the subject in an English sentence.
You are making an excuse for a poor observation. "the Essence of Intelligence" applies to the current models, and they do not require positional encoding, but they do require attention.
> because the Vaswani et al transformer (as it is described here) simply does not work until you add the most important ingredient of all: the positional encoding.
This is factually correct. The Viswani et al transformer DOES NOT WORK without positional encodings. A transformer architecture that involves (e.g.) causal masking is a different architecture. If you want to separately discuss one of those architectures I am happy to do that too.
As a former neuroscientist of visual perception, I always feel compelled to point out that "attention" in the ML sense has very little to do with biological attention.
"Spreading activation" would be a better analogy.
----
It's mostly harmless, though it occasionally results in head-scratchers from the biological POV. I once saw a ML paper talking about "retinal attention", which is funny, because the retina is one of the few places in the central nervous system where afawk, attention is impossible.
There's simply no efferent (brain->retina) non-motor nerves capable of biasing photoreceptor output; the retina is largely ballistic. (Eyeball motion is not attention.)
The term "attention" confuses more than it reveals, trading well-known terminology for a weak analogy to how the human brain works. Kernel smoothing is all you need.
For those who are not math people, this comment is referring to the fact that "attention" is formally equivalent[1] to Nadaraya-Watson kernel smoothing, which is a kind of weighted average. Yes! Incredibly, attention is just a weighted average.
Imagine you have a series of (x,y) coordinates. You receive a new x coordinate, and you'd like to predict the corresponding y. So, the absolute dumbest thing you can do is find the nearest x, and then report its corresponding y as your prediction. The second dumbest thing you can do is find the n closest x coordinates, and then average their y values together, and output that as your predicted value. And the third-dumbest thing you can do is that n-closest-average thing, but give the closest x coordinates the most "weight" when you average them.
That third-dumbest thing is Nadaraya-Watson kernel smoothing. And that's what attention does. You have a big list of vectors. You compute a weighted average of those vectors. You now have a new vector. You can do what you want with it, but LLMs will generally turn that into a distribution over token types, from which you'll generally want to select either the top one, or weighted-randomly one from the top k (known as "increasing the temperature").
With all that said, I'm not sure "Kernel smoothing is all you need" is actually materially better than "attention is all you need" for 99.999% of humans. :)
Yeah, I mean, a lot of things can express attention, because attention is mechanically extremely simple. The "trick" of attention is that it's "relatively easy" to train. Or if not "relatively easy", we at least have a lot of collective experience doing it.
The crucial question, however, is "Where do the weights come from?" :-)
btw, "kernel smoothing" etc. are still fairly complex terms that sound a bit like math magic: 1-nearest-neighbor (1-nn), k-nearest-neighbor (knn), and weighted knn would be even simpler and possibly more descriptive.
Yeah but Nadaraya-Watson kernel smoothing isn't actually 1NN/kNN/weighted kNN.
First of all, the trick relies on imputed 1NN/imputed kNN/imputed weighted kNN. And second of all, you still have to explain how imputed weighted kNN relates to the problem of taking a bunch of x/y pairs and predicting a novel y for some novel x. When you start to unpack all of that it does not look simpler to me than what I wrote.
For the weights, I'm not sure that's as mysterious as you make it seem. The most important thing they learn is, basically syntax. So you have 40k token types to choose from at each prediction, and the vast majority of them are statistically implausible because they are syntactically invalid. At some positions this restriction reduces entropy from ~15-16 bits (or 40k choices) to 2^8 (~200 choices) and in some places it reduces it "only" to 12 or 13 (~8k choices, common when the next token is "approximately a noun").
Once you winnow it down, it's much closer to a moderately big multiple choice problem. And much of that entropy lies just in the model's representation of word co-occurrences.
Attention is not based on Euclidean closeness, therefore it is not kernel smoothing. Saying otherwise would be the same as saying kernel smoothing is just an average (e.g. omitting the important part that it has to be distance-based-weighted).
First and most importantly, attention is, formally, kernel smoothing with a particular exponential kernel function, and if you don't believe me, you can read the paper I linked, which proves it.
Second, and less importantly: kernel smoothing absolutely, definitely does NOT require a kernel function representing Euclidean distance, and in fact the entire point of kernel smoothing is that you can pick basically any function that fits your needs. Many use cases find it helpful to have a real-valued kernel function that is positive and semidefinite, or nonnegative and integrating to 1, or whatever. Different kernel functions in these spaces have different properties and trade-offs. The exponential kernel attention uses (if memory serves) is (1) finite and nonnegative, and (2) has a positive total weight so you can normalize.
Third, and least importantly, you definitely can have a kernel function that is "uniform" (i.e., "just" an average) although it's generally not very helpful (e.g., not positive semidefinite).
Your writing takes a lot of work to follow, I think because English is your second language. I will try one more time.
For those following along, when this user says that attention is kernel smoothing with a "Q-K based distance function", I think (THINK!) what they're trying to say is the second part does more work than it gets credit for when we say attention is "just" a weighted average.
Both of these things are actually true. Attention is indeed "just" a weighted average (there is a proof in the paper in my original reply), AND ALSO, the Q/K are learned, and the similarity between two learned vectors suggests there is more going on than might initially be obvious. (I actually argue in the top comment that the positional information is the #1 reason why attention works, so I agree with this.)
Let's see why.
What are Q and K? The idea in the original paper was: we have some learned embeddings for our tokens, and some "context" we'd like to use to enrich those embeddings. Stuff like the positional information, context accumulated through layers, other learned stuff. So we do this enrichment, and that produces the so-called "query" (Q) and a "key" (K) vectors. Attention is STILL a weighted average, where the weights are determined by how "similar" those vectors are (this is what we're talking about when we discuss the "distance" provided by the so-called "kernel function").
Anyway the reason it's hard for me to parse this reply is because I think a lot is getting lost in the transition from a second language to English. For example, "distance" in this context means something specific, and to be precise the exponential kernel provides a similarity measurement (dot product can encode distance but generally requires fixed-length vectors). Similarly, I am confused by their use of "Euclidean" distance (kernel functions do not require this, see p. 203 of Elements of Statistical Learning), I'm not sure how to interpret "what you feed to the kernel is Euclidean distance" (the kernel should compute the similarity, not be fed the similarity) and so on. None of this is the end of the world, I am just having a very hard time figuring out what they meant.
In this context, what does "closest" mean? Most similar token embeddings? Most similar Q embedding to prior K embeddings? Most similar positional embedding?
Are you, in essence, implying that the Key-Query attention mechanism is basically a neural function approximation of the Nadaraya-Watson kernel or something very like it?
No, kernel smoothing is based on a function of distance between points. Attention is based on a function of distance between their learned Q and K projections.
>> To be sure, the success of LLMs shows how much information about the world is encoded in the statistical patterns of language alone
How can we still have it so backwards. Language is just a medium to convey information on an underlying reality. LLM do not model the medium, they model the underlying reality (so far as the medium allows it).
Attention is not the key, if anything is the key it to intelligence it is scale. As long as all of the information is there for the model, with sufficient scale you will see intelligence emerge. This premise is core to the belief that the main blocker behind LLMs being "discovered" was not architecture, but in fact compute. Once sufficient compute is available eventually someone will be able to create an LLM that is intelligent.
The original paper title is catchy but it’s more like “positional encoding and attention and parallelism across attention heads are all you need…”
And it turns out you probably need massive compute, about 1/3 RLHF, all the worlds data and millions of people connecting their private data to the AI to learn from amongst other things :-/
what does it take to get some authorative flailing around here anyway!
it's not like we are asking for a tame god to do our bidding or anything, but well ok, we are asking for a tame god, but it will be great, and the world will be perfect, so hurry up, back to work, or the Boss™, who isn't tame might have to have a word with all you little pumpkins, so back to flailing, but harder, more decisive flailing, or hey! drugs
77 comments
[ 0.17 ms ] story [ 58.7 ms ] threadOf course, attention is almost certainly a heuristic readers use, but to even bring it up as key to intelligence begs so many questions about LLMs and the mind...
--
A computer can't read a file, it is just sending electric signal, sensing the voltage, probing the magnetic field, etc.
"read" a file is anthropomorphized terms for what the machine do when processing a file.
--
Maybe we should just stop worrying and embrace the new meaning of those words?
As a former perceptual neuroscientist, I can assure you that ML "attention" has very little to do with biological attention.
E.g., a high-level auditory neuron might primarily fire in response to high-pitched voices, but if you hear someone say your name in a high-pitched voice at a party, you can bias that neuron to try more selectively fire in response to just their voice, "tuning out" similarly-pitched voices.
When you think about it that way, you can bridge the two types of cognition more usefully. The how of intelligence, what attention consists of, can remain completely different, while the why can feel so universal that humans can easily forget they're talking to a machine.
How attention affects biological neurons has little in common with the ML mechanism of attention.
Salience explains the "why", but I'm talking about the "how".
People are doing what people do: they're desperately moving the goal post to keep pretending humanity is particularly special and that intelligence is particularly special. Unique snowflakes and all that, we're the center of the universe. It's nothing more than elaborate human weakness & fragility on display.
We're in the first inning of this, intelligence is not going to end up being special, it's going to be trivial and low cost. See: Chinese models ten years from now.
Astra, Opus, Fable, Argon - they're all radically beyond the top 1% of humanity intellectually already, and it's going to get a lot worse quickly. The sole aspect the top 1% beat them on, and increasingly rarely, is at hyper specialization, where a person is elite at what they do. For $20-$100-$200 per month I can get access to an omnimath genius intellect far beyond any human that has ever existed, and that AI will work for me like a dog. Humanity should be scared, and the reason HN hates AI so much, is because it will wipe out very large numbers of their lucrative jobs that they imagined would form long-term careers (six figures, stock options, path to wealth, elitism re competition).
https://www.quantamagazine.org/is-attention-the-essence-of-i...
"Sentences with pronouns such as “it” in them illustrate the challenge that attention overcomes. Consider: “The animal didn’t cross the street because it was too tired.” And: “The animal didn’t cross the street because it was too wide.” To translate these sentences correctly into French, an AI model must determine that, in the first one, “it” refers to the animal, which is a masculine word in French, so the translation should use that form of the pronoun, il. But in the second sentence, “it” refers to the street, a feminine word that requires a different pronoun, elle. How can the machine figure this out?"
"Bahdanau remembered doing translation exercises when he was learning English in school. To write each next word of his translated sentence, he’d have to glance all around the source text to figure out what should come next. “Literally you have your source sentence on the left and your writing on the right, and you’ll be, like, turning your head, left-right, left-right,” he said. “That process is what I expressed as a neural mechanism.”
That simplicity seems to me the real tell that this is the real insight. The author does a good job of walking the ready past a long list of distractors and conjuring a sense of a path to an insight, and then, there's the insight - a simple encoding of a simple task: "What is the next word? How do I figure that out? Left-right. Left-right"
Most of the semantics of a sentence are indicated by the precise order the tokens appear. This is easiest to see in a language like English, since it is visible in two very prominent ways.
First, the morphology is fairly derivational, meaning you can keep adding suffixes to things and English speakers will generally get what you mean. For example: derivation-al-ish-ly. You've probably never heard this specific derivation, but you probably have some idea of what it means anyway. The transformer can only capture these semantics if it has some representation of the fact that the suffixes inflect the meaning of the word's root (presumably something like `deriv`). English encodes this entirely by the position of the suffixes relative to the root. Without the positional information, there is no way to recover it, and sentences would lose much of their meaning, since English depends heavily on this construction.
Second, English has relatively few cases, and thus relies very heavily on the order of words to indicate what is the subject and what is the object. English specifically is Subject-Verb-Object, and the instinct of native English speakers to use SVO ordering is so strong, you will find them inserting nouns and pronouns into sentences that do not actually have subjects at all. "It's raining outside" is the most famous example. This sentence has no subject at all, and yet native English speakers feel a strong need to place "it's" at the beginning.
If the transformer did not have positional encodings it would be completely hopeless to recover what is the subject and what is the object from English sentences. There simply is no other way to reliably tell, other than inspecting the word positions.
It's worth noting we also do have the counterfactual. Generative language models that explicitly exclude positional information, like Latent Dirichlet Allocation, are only able to (essentially) cluster word co-occurrences. That is not as useless as it sounds—if I say "fire house burning ambulence" you kind of get the point even without a subject or an object—but it certainly does not generate the lucid, provoking sentences that modern LLMs do.
German: es regnet French: Il pluet
I don't see why English is special here.
I guess the author means in contrast to some cases like "Pluvas" in Esperanto, see https://www.youtube.com/watch?v=fOBkKcbJUAE for a nice employ of it.
Italian, Spanish and Latin (for example) can generally avoid to express explicitly the subject in a dedicated pronoun, as the verb already encode the it through specific flection. In contrast in French and German only in imperative tense one can formally make such an ellipsis (colloquially of course one can encounter more nuances).
So nothing exceptionnal
The rain is not magical, the environment is producing it. The environment is raining. It's raining outside means the environment is producing rain outside.
Who could possibly confuse that as lacking a subject.
https://www.cambridge.org/core/journals/journal-of-linguisti...
French is descended from Latin, which is "pro-drop" in the sense that you'd drop the "dummy subject" and simply write "pluit." Later, Old French, like English, lost most of its cases. Because of that, many word endings started sounding very similar (a problem known as "agreement syncretization"), and all of that seems to correspond in the record with the pro-drop "null" subject.
This is an entirely different path than English took, and it's also not a path any other Romance language took. Spanish isn't even subject-prominent. It fully allows the "null" construction, e.g., "llueve", which literally is just "rains."
In Ukrainian it’s «дощить» (from “дощ” — rain) — the form of the word tells you its function in the sentence. Same happens in other Slavic languages: Polish, Belarusian, Slovakian.
Latin languages rely on the order of words to explain their function. So much so, English speakers don’t say “blue big ball” in German, time before place is mandated.
I would rather say it’s derived from contextual element. And in linguistic context which encodes more interpretation cues, order of morphemes drop symmetrically in semantical importance, though it can still affect interpretation on pragmatic level. Latin for example already offers more flexibility regarding word order, though it’s used mostly in poetry/stylised-rhetoric. And as German show having the case encoded in the nouns don’t even prevent mandatory rigidity in sentence structure.
There isn't really a way to get around this. Natural language IS structured as a recursive tree. Our job is to extract this tree from a flat stream of tokens (or similar), and straightforwardly the only way to do that is to make the ordering of the tokens meaningful. There is no other option! This is how 100% of parsing works in any context whatsoever, both theoretically and practically.
And just because utterances can be more or less matched with recursive tree doesn’t mean there is an ontological absolute truth about language there. Even "language" refers to some vague abstraction. Just like the "it" in "it rains" mentioned in this discussion: no entity out there actually respond to "language".
At introspection level, I absolutely don’t interpret utterances like recursive tree. They are more like cues going through living graphs more or less already connected and being able to change topology on the go. Impermanence of interpretation results is the only constant.
Now it’s not to dismiss that some other people might feel like "Natural language IS structured as a recursive tree" is the only way. People have different perspective, different brains, different life experiences. One who dwell in aphantasia¹ don’t perceive the world the same way as people that have hyperphantasia². Different of course here doesn’t mean better/worst or superior/inferior.
¹ A word I discovered in some HN comment
² A word that I had never red before but just popped up in my mind as I was thinking about this sentence, but which is actually already is in use in literature apparently: https://en.wikipedia.org/wiki/Hyperphantasia
I’m also not sure why you are talking about why natural languages are not different from other ones. That is what I’m saying, it’s the reason behind the comment that all parsing works the same way.
And if you look at how transformers work you will immediately realize there's nothing a transformer can do that an RNN cannot do, and frankly that an RNN cannot do 1000x more efficiently at inference time. It's just that training even the best known RNNs is going to take a lot longer because it can't be parallelized. It cannot train on 100 GPUs, in fact it cannot really use a GPUs advantage.
Plus equivalence proofs will get you a lot further than RNNs too. Kernel methods will obviously work as well, for example.
> because the Vaswani et al transformer (as it is described here) simply does not work until you add the most important ingredient of all: the positional encoding.
This is factually correct. The Viswani et al transformer DOES NOT WORK without positional encodings. A transformer architecture that involves (e.g.) causal masking is a different architecture. If you want to separately discuss one of those architectures I am happy to do that too.
"Spreading activation" would be a better analogy.
----
It's mostly harmless, though it occasionally results in head-scratchers from the biological POV. I once saw a ML paper talking about "retinal attention", which is funny, because the retina is one of the few places in the central nervous system where afawk, attention is impossible.
There's simply no efferent (brain->retina) non-motor nerves capable of biasing photoreceptor output; the retina is largely ballistic. (Eyeball motion is not attention.)
Imagine you have a series of (x,y) coordinates. You receive a new x coordinate, and you'd like to predict the corresponding y. So, the absolute dumbest thing you can do is find the nearest x, and then report its corresponding y as your prediction. The second dumbest thing you can do is find the n closest x coordinates, and then average their y values together, and output that as your predicted value. And the third-dumbest thing you can do is that n-closest-average thing, but give the closest x coordinates the most "weight" when you average them.
That third-dumbest thing is Nadaraya-Watson kernel smoothing. And that's what attention does. You have a big list of vectors. You compute a weighted average of those vectors. You now have a new vector. You can do what you want with it, but LLMs will generally turn that into a distribution over token types, from which you'll generally want to select either the top one, or weighted-randomly one from the top k (known as "increasing the temperature").
With all that said, I'm not sure "Kernel smoothing is all you need" is actually materially better than "attention is all you need" for 99.999% of humans. :)
[1]: https://arxiv.org/abs/1908.11775
https://scikit-learn.org/stable/modules/generated/sklearn.im...
https://scikit-learn.org/stable/modules/generated/sklearn.me...
I don't have any specific experience in ML but I always liked the formalism presented in https://www.youtube.com/watch?v=J1YCdVogd14 as a way to unify everything
Now if you could please explain the rest of math in a similar manner?
btw, "kernel smoothing" etc. are still fairly complex terms that sound a bit like math magic: 1-nearest-neighbor (1-nn), k-nearest-neighbor (knn), and weighted knn would be even simpler and possibly more descriptive.
First of all, the trick relies on imputed 1NN/imputed kNN/imputed weighted kNN. And second of all, you still have to explain how imputed weighted kNN relates to the problem of taking a bunch of x/y pairs and predicting a novel y for some novel x. When you start to unpack all of that it does not look simpler to me than what I wrote.
For the weights, I'm not sure that's as mysterious as you make it seem. The most important thing they learn is, basically syntax. So you have 40k token types to choose from at each prediction, and the vast majority of them are statistically implausible because they are syntactically invalid. At some positions this restriction reduces entropy from ~15-16 bits (or 40k choices) to 2^8 (~200 choices) and in some places it reduces it "only" to 12 or 13 (~8k choices, common when the next token is "approximately a noun").
Once you winnow it down, it's much closer to a moderately big multiple choice problem. And much of that entropy lies just in the model's representation of word co-occurrences.
First and most importantly, attention is, formally, kernel smoothing with a particular exponential kernel function, and if you don't believe me, you can read the paper I linked, which proves it.
Second, and less importantly: kernel smoothing absolutely, definitely does NOT require a kernel function representing Euclidean distance, and in fact the entire point of kernel smoothing is that you can pick basically any function that fits your needs. Many use cases find it helpful to have a real-valued kernel function that is positive and semidefinite, or nonnegative and integrating to 1, or whatever. Different kernel functions in these spaces have different properties and trade-offs. The exponential kernel attention uses (if memory serves) is (1) finite and nonnegative, and (2) has a positive total weight so you can normalize.
Third, and least importantly, you definitely can have a kernel function that is "uniform" (i.e., "just" an average) although it's generally not very helpful (e.g., not positive semidefinite).
For those following along, when this user says that attention is kernel smoothing with a "Q-K based distance function", I think (THINK!) what they're trying to say is the second part does more work than it gets credit for when we say attention is "just" a weighted average.
Both of these things are actually true. Attention is indeed "just" a weighted average (there is a proof in the paper in my original reply), AND ALSO, the Q/K are learned, and the similarity between two learned vectors suggests there is more going on than might initially be obvious. (I actually argue in the top comment that the positional information is the #1 reason why attention works, so I agree with this.)
Let's see why.
What are Q and K? The idea in the original paper was: we have some learned embeddings for our tokens, and some "context" we'd like to use to enrich those embeddings. Stuff like the positional information, context accumulated through layers, other learned stuff. So we do this enrichment, and that produces the so-called "query" (Q) and a "key" (K) vectors. Attention is STILL a weighted average, where the weights are determined by how "similar" those vectors are (this is what we're talking about when we discuss the "distance" provided by the so-called "kernel function").
Anyway the reason it's hard for me to parse this reply is because I think a lot is getting lost in the transition from a second language to English. For example, "distance" in this context means something specific, and to be precise the exponential kernel provides a similarity measurement (dot product can encode distance but generally requires fixed-length vectors). Similarly, I am confused by their use of "Euclidean" distance (kernel functions do not require this, see p. 203 of Elements of Statistical Learning), I'm not sure how to interpret "what you feed to the kernel is Euclidean distance" (the kernel should compute the similarity, not be fed the similarity) and so on. None of this is the end of the world, I am just having a very hard time figuring out what they meant.
[1]: https://mybiostats.files.wordpress.com/2015/03/chapter5-kern...
Are you, in essence, implying that the Key-Query attention mechanism is basically a neural function approximation of the Nadaraya-Watson kernel or something very like it?
I thought both were useful in their own way, it's not like one replaced the other right?
How can we still have it so backwards. Language is just a medium to convey information on an underlying reality. LLM do not model the medium, they model the underlying reality (so far as the medium allows it).
But then dumbly concludes : oh is language intelligence ? ("Perhaps our form of intelligence is tantamount to language modeling, too?"). C'mon.
And it turns out you probably need massive compute, about 1/3 RLHF, all the worlds data and millions of people connecting their private data to the AI to learn from amongst other things :-/
I dunno, but this HN post and the chatter that follows is sure to help raise my stocks!
FLAIL HARDER! DAMIT!
what does it take to get some authorative flailing around here anyway!
it's not like we are asking for a tame god to do our bidding or anything, but well ok, we are asking for a tame god, but it will be great, and the world will be perfect, so hurry up, back to work, or the Boss™, who isn't tame might have to have a word with all you little pumpkins, so back to flailing, but harder, more decisive flailing, or hey! drugs