> I hope these rather unorthodox leaps between linguistics and mathematics helped make it almost obvious that some words and ideas are untranslatable in practice. I also hope you don't take the analogy too seriously, because it won't go much further than this.
Communication/language depends on shared context. The more context you share the shorter the trigger for evoking that thing and that context. And if you share no context communication becomes very difficult.
The article seems to think that a word is untranslateable if there is no single word in the target language. If I'm not misreading the article, then this is completely obvious -- just consider the number of words in English and the number of words in almost any other language, and you will find that there are more English words than the other language. It is now clear that there exist English words that don't correspond to a single word in the other language.
There's a real irony that the examples are coming from Japanese since it is an agglutinative language.
I think people don't realize how weird language is. Like you could look at Chinese and call each sentence a "word" as there are no spaces. What's the difference between that and a compound word like "nighttime" or the whole German language where you got words like Krankenwagen ("patient" + "car").
Now this doesn't mean there aren't words or phrases that aren't translatable. But the thing is we can always translate the words themselves. What we can't always translate is the meaning behind them. I think the best example of this comes from Star Trek and the Tamarian Language[0,1]. "Sokath, his eyes open!" The problem with communication is not that the words don't translate, it is that the meaning behind them doesn't. Just as people struggle with idioms when learning American English or why someone might be confused about why someone "shit in the milk" or "fucked the dog". Words are an embedding. A compression.
The thing people are constantly forgetting, but is more important than ever in a globally connected world, is that words are not perfect representations of thoughts. We compress our thoughts into them and hope the person on the other side can decompress them. It is why you can more easily communicate with your close friends who have better context than you can with another person that natively speaks your language and is why someone that learns a new language can speak perfectly well but still struggle to communicate. Language is not just words, it is culture[2]. So in a much more connected world today we have these disconnects in culture and thus interpretation of what people say. I know every one of you has been told to "speak to your audience" but how do you speak to your audience when your audience is everybody and when you don't know who your audience is? The new paradigm requires us to be much better interpreters than we were before. Least everyone is going to sound crazy, other than those you frequently talk to and have that shared understanding.
[2] This is, btw, why people argue for embodied AI being so critical. Not because LLMs can't appear to grasp the language, but because we as humans have embodied our language so deeply you probably didn't even realize that I used the word "grasp" to refer to an abstract concept and not something you can actually touch with your hand.
> It is now clear that there exist English words that don't correspond to a single word in the other language.
But that's true of any language. Not only that, but English uses loanwords heavily which are often Anglicisations of words from other languages, which may not in themselves be just one word.
"Ho ho ho", the flag-waving Little Englander types say, "Gaelic is such a stupid language, they don't even have a word for 'television', they just say 'television' in a stupid accent!"
But English also has no word for "television". Worse, the word "television" isn't even just a loanword, it's two words from two different languages, "tele" from Greek and "vision" from Latin. What a bodge job! Imagine letting something like that slip through to production use!
The hypothetical Catalan-Hungarian inventor of it in another leg of the trousers of time may have called it llunylátás, and then where would we be?
Well, most languages would have some variant of that word to mean "television", as they do now, I expect.
The English word "galore" (meaning "sufficient" shading towards "more than enough") comes from the Gaelic words "gu leòr", (goo lyaawr, the grave accent above the o makes the vowel sound longer). What a silly language English is, doesn't have a word that means "more than you're ever likely to need", has to steal one from Gaelic and then spell it wrong.
Oh, they use this word "whisky". You know what that means? It means "uisge beatha" but they only say the first word, in a silly accent because they can't pronounce it properly.
Quite often there's no single word for a thing you're trying to translate but that doesn't mean it's untranslateable. English has only one single word for rain, for example, but Gaelic has about half a dozen of which the only ones I can reproduce here are "uisge" (that word again) which just means "water", and "fras" which is more like a gentle shower. The rest of the words in the Gaelic of the North-West of Scotland that refer to rainy weather are, of course, profane in the extreme.
But perhaps all languages have a countably infinite number of words, in which case that proof doesn't work. (In English we have: legless, leglessness, leglessnessless, leglessnesslessness, ... It's not a great example, but it's good enough.)
Even if the number of words in a language were finite we wouldn't have a reasonable way of counting them. There are too many kinds of fuzziness involved in deciding what counts as a "word" and you can't ignore the borderline cases because the borderline cases vastly outnumber the straightforward cases.
"...there are more English words than the other language" There might be more words in some English dictionaries than in some dictionaries of other languages, but that may just be due to a lot more effort having gone into English lexicography than X language lexicography. I doubt that most native speakers of English know more English words than equally educated native speakers of some other language know words of their language.
> and you will find that there are more English words than the other language. It is now clear that there exist English words that don't correspond to a single word in the other language.
You're forgetting about synonyms. The common adage that English has the largest vocabulary stems from the fact that it often has multiple words for the same thing. Sofa, couch. Autumn, fall. Etc etc. Other languages generally don't do this. I've never heard anyone suggest that English has words for more concepts.
Tangent: I really like vornoi diagrams and part of me thinks there's a hidden, precious concept they represent. I didn't get their relation to the article but was wondering if they have applications in engineering/sciences.
A Voronoi diagram is created when you color every point on an image according to which discrete point it is closest to.
So in this case, I see the diagrams as representing the boundaries drawn when projecting / quantizing complex ideas into a set of central points that are insufficient for catching all of the nuance of the original. How well can you adapt a nuanced idea to a different space?
If Language A has an idea that exists at one point in space, which is the closest word in Language B that might be used to represent it? A Voronoi diagram is one possible way of illustrating it.
Tangent on your tangent: this GDC presentation from 2016 is probably my favorite real-world application of Voronoi Diagrams, and uses them for N-player split-screen camera control: https://www.youtube.com/watch?v=tu-Qe66AvtY&t=1594s
I have a lingering dream in the back of my mind to make a single-couch Liero-style casual game for N-players with good dynamic camera support using this technique.
It's a tenuous analogy, but if you along with it, you can take it further.
You could consider the "cost" of expressing a word as some kind of metric or norm on the vector. What in one language/basis is a simple Kronecker delta, in another is a very complex vector (of course if it were the same vector in two bases, it would have the same length, but we could rather think of translation as an affine transformation, say).
And finally, with two bases, they need not span the same vector space. You can have a three-coordinate vector space all you like, if you have only two basis vectors you ain't spanning it. At best you can hope for an orthogonal projection from one to the other, and lose some nuance.
Eventually, with bilinguality, you learn not to translate words. Concepts live in different languages and describe a reality. Usually you can describe that reality in two different languages, but sometimes not.
I think a succinct way to describe my thoughts on linear algebra/language is that language has high dimensionality (ie many different basis vectors that may not necessarily be orthogonal) and that individual languages use a unique coordinate system to express thought. Each language is a lossy approximation of all conceivable thought and some languages can more efficiently represent the “all thoughts” vector space because they have basis vectors that point in more uncommon directions (like the go to japan example). So while you can more or less point to any thought in any language, some thoughts are easier to express in certain languages, which the post (and me) agree to be untranslatable words.
I tried to find the really interesting article about language and color that describes how some cultures use different naming schemes for colors but couldn’t find it. It talked about how back in the day we don’t know orange as a color, we just thought it was red-yellow and only after the fruit was distributed did the word for the color catch on. Here’s the best article I can find that talks about this phenomena https://burnaway.org/magazine/blue-language-visual-perceptio...
Each language is a lossy approximation of all conceivable thought...
This ultimately boils down to the private language discussion started by Wittgenstein. If you admit public language is a lossy approximation of meaning, you're taking a position on the existence of private languages.
> If the mere sight of the above is like a punch in the face for you, don't worry. I'm not going to math you to death in what follows. I will only remind you of a tiny basic part of it that I think relates to languages.
Yes, that mathematical expression is like a punch in my face, but not for the reason you think. I am offended that the rank of the matrix does not match the dimension of the matrix, not that I'm seeing a matrix.
This article assumes that concepts are somehow precise coordinates within a single language; that's not the case, at best, speakers of a language mutually approximate a relatively consistent representation, but like, look at a word like yeet or whatever: we decided as a society on its meaning while it was being developed, as it were. Furthermore, it never rigorously defines what it means by translation. It claims 上京 is a single basis meaning moving to Tokyo, for example, but that isn't even an accurate translation: the individual components represent superior/greater/above and Tokyo and as an idiomatic phrase it represents the concept of moving to the capital for a better life. Something like "moving on up" or the like in some vernaculars of English, and idioms translating to idioms is a form of translation. It's disingenuous to represent the first concept as a single basis but not the second.
Similarly, it claims mono no aware (物の哀れ) is unable to be translated, but, again, more literally "translated" is saying "the sorrow within things" character by character, and, only as an idiom has the full contextual understanding. It's not really a single point even if it's rather accurately located in a hypothetical embedding space by Japanese speakers. Imo, an English translation of the concept is "everything is dust in the wind", only 2 more individual conceptual units than the original Japanese phrase, and 3 of them are mainly just connecting words, but it's understood as a similar idiom/concept, here.
Concepts are only usefully distinguished by context and use.
By the author's own argumentation: nothing is translatable (or, generally, even communicatable) unless it has a fixed relative configuration to all other concepts that is precisely equivalent. In practice, we handle the fuzziness as part of communication and its useless to try and define a concept as untranslatable unless you're also of the camp that nothing is ever communicated (in which case, this response to the author's post is completely useless as nobody could possibly understand it enough internally for it to be useful. If you've read this far, congrats on squaring the circle somehow)
This. Two speakers of the same language only have approximately the same understanding of the meanings of the words they both use. Communication succeeds because we are constantly seeking and correcting misunderstandings that arise due to no two people speaking exactly the same language.
The same process that allows two speakers of the same language to communicate adequately allows one to translate from one language to another. If it were truly impossible to translate from one language to another, we would be unable to perceive this and argue about it. The recognition and correction of errors is part of the process of translation just as it is part of the process of communication in a single language.
Well said! To add on: if meaning is largely not "in" the words themselves, but embedded in a shared cognitive space, then in order to have a truly singular (ie "untranslatable") basis point would require positing unique cognitive mechanisms or some experiential quality that is unknown to members of the target language. But as you pointed out, most concepts do have an analogous representation in most languages, even if the tokens in use appear superficially different. And this is merely because the context in this case is a shared cognitive substrate (the low-level operating system, if you will) consisting of sense data, emotions, and so on, which in its fundamental operations does not substantially differ between members of the human race - or so I would argue. In either case, what matters seems to me to be not so much the actual tokens but the experiences or cognitive context in which they are embedded.
Just read Wittgenstein (The Blue+Brown Books / Philosophical investigations), and this confusion will go away. The difference between translation, definition, and explanation needs to be understood.
There's one aspect that I think the article starts to hint at, but doesn't quite make the jump to is that words in a language just map to a subset of concepts that don't necessarily have the same subset boundaries in other languages.
If you think back to the meme from a decade or two ago about how men and women perceive colour [1], where e.g. "pink" to a man covers a whole range of colours to a woman, then that kind of hints at the idea.
One example back in the realm of vocabulary is the English word "happy". This embodies a range of meanings from joy, willingness, pleased, contentment, satiation, etc. There might be some overlap in some of these meanings with other words like "joy" or "excited", that don't have the same overlaps in other languages. E.g. "happy" might be translated to French as "heureuse" for the senses of pleased or content, but not for willingness sense.
Similarly, the French word "dommage" can be translated into a whole bunch of English words that aren't normally synonyms of each other - pity, damage, shame, harm.
This kind of nuance can lead to two opposite problems when translating - when the meaning is limited to a subset of possible meanings by context, and the wrong one is chosen in the foreign language, and when the author's meaning embodied multiple meanings and the chosen translation doesn't cover all of them.
Some of these features can lead to the humour in subtle jokes being lost in translation, e.g. "he'd be late to his own funeral".
My personal analogy, useful in my early days: Translating is like finding a vector in another space that points in the same direction or carries a similar magnitude of meaning.
In other words:
The source sentence is a vector in “language A space.”
The target sentence is a vector in “language B space.”
A good translation finds a vector that has the same direction (same meaning, intent, tone) even though it lies in a different coordinate system (the new language).
I know what you mean, but semantics is about relative positions of points in a given space. Comparing two points from two different spaces is apples and oranges. I feel like this analogy should be salvageable with a small tweak, however.
What a trendy article, in tune with our recently linear-algebraic turn in how we see language thanks to LLM's.
But I think this exposes an even greater problem, where words thought to be direct translations will always drift in vector value as they are weighted for attention within their respective corpora. Are we on the brink of translation-nihilism?
This isn't even limited to complex phenomena or shades of snow. Even "I like" is a different construction in many languages, in an unexpected way to new language learners.
Interestingly enough for this morning's walk I was musing over the tension between the hypotheses that: 'LLMs can map between languages in the vector space' (thus languages are ~equivalent); and 'Language affects thoughts' (as in German is good for Philosophy and English for getting things done).
If both these thoughts are true, then it would appear that languages have topological characteristics. We can (topologically) map from one to another, 'thoughts' (that is a complex of words) form 'paths on the language manifold' and certain paths may be more 'natural' in one topological form than the other.
There is the platonic representation hypothesis, which speculates that as LLMs get larger and more multimodal, they all end up learning isomorphic representations of reality. Maybe for humans something like this is true as well, since ultimately language must be rich enough to capture and communicate reality.
Reading any poem that makes use of extensive wordplay within a language shows why there will always be some untranslatable aspect. You can't create all the exact shades of a single pun if all those shades aren't in a different language.
Go translate an ee cummings poem and make sure to retain all its meanings.
Big claim but not much substance. They should try to really understand linear algebra first, and also linguistics a bit. Semantic domain (from linguistics) is a better way to describe it, where using sets (from math) might better convey what they want to say.
The word 'word' is polysemous, and also vague. Polysemous: it can refer to written words or spoken words (or signed words, in sign languages).
Vague: does it mean all the inflected forms of a word, or just the stem without inflection? Example: is (are?) 'walk', 'walked', 'walks' and 'walking' four words, or one? What about "stand/stood"? (And languages where the bare root or stem can't appear by itself, like verbs in Spanish.) Derived words, like 'push' and 'pushy'.
Do compound nouns count as a word, or do only the parts count? Example: 'doghouse' (or 'dog house'). What about idioms? Example: 'to crane his/her/my/your/our neck(s)'.
What about different pronunciations? Is 'roof' pronounced to rhyme with 'aloof' the same word as 'roof' pronounced with the vowel of 'put'? And different spellings but same pronunciation: 'bear' vs. 'bare'.
What about words with different grammatical categories, like 'push' as a noun ("I gave her a push") or a verb ("I pushed her"). Or the same word with virtually unrelated meanings, "I pushed her on the swing" vs. "I pushed my ideas."
I like this as an analogy but not as an explanation. In fact, if you’re unfamiliar with linear algebra, this might be a nice way to think about projection onto a different set of basis vectors. But even the best human translators can be deeply at odds over what translation is appropriate for a term in its context. There’s never a right translation, let alone a uniquely right translation.
45 comments
[ 3.8 ms ] story [ 95.7 ms ] threadPhew! Thanks for clarifying.
I wasn't aware that that idea was in dispute.
I think people don't realize how weird language is. Like you could look at Chinese and call each sentence a "word" as there are no spaces. What's the difference between that and a compound word like "nighttime" or the whole German language where you got words like Krankenwagen ("patient" + "car").
Now this doesn't mean there aren't words or phrases that aren't translatable. But the thing is we can always translate the words themselves. What we can't always translate is the meaning behind them. I think the best example of this comes from Star Trek and the Tamarian Language[0,1]. "Sokath, his eyes open!" The problem with communication is not that the words don't translate, it is that the meaning behind them doesn't. Just as people struggle with idioms when learning American English or why someone might be confused about why someone "shit in the milk" or "fucked the dog". Words are an embedding. A compression.
The thing people are constantly forgetting, but is more important than ever in a globally connected world, is that words are not perfect representations of thoughts. We compress our thoughts into them and hope the person on the other side can decompress them. It is why you can more easily communicate with your close friends who have better context than you can with another person that natively speaks your language and is why someone that learns a new language can speak perfectly well but still struggle to communicate. Language is not just words, it is culture[2]. So in a much more connected world today we have these disconnects in culture and thus interpretation of what people say. I know every one of you has been told to "speak to your audience" but how do you speak to your audience when your audience is everybody and when you don't know who your audience is? The new paradigm requires us to be much better interpreters than we were before. Least everyone is going to sound crazy, other than those you frequently talk to and have that shared understanding.
[0] https://memory-alpha.fandom.com/wiki/Tamarian_language
[1] https://www.youtube.com/watch?v=3-wzr74d7TI
[2] This is, btw, why people argue for embodied AI being so critical. Not because LLMs can't appear to grasp the language, but because we as humans have embodied our language so deeply you probably didn't even realize that I used the word "grasp" to refer to an abstract concept and not something you can actually touch with your hand.
But that's true of any language. Not only that, but English uses loanwords heavily which are often Anglicisations of words from other languages, which may not in themselves be just one word.
"Ho ho ho", the flag-waving Little Englander types say, "Gaelic is such a stupid language, they don't even have a word for 'television', they just say 'television' in a stupid accent!"
But English also has no word for "television". Worse, the word "television" isn't even just a loanword, it's two words from two different languages, "tele" from Greek and "vision" from Latin. What a bodge job! Imagine letting something like that slip through to production use!
The hypothetical Catalan-Hungarian inventor of it in another leg of the trousers of time may have called it llunylátás, and then where would we be?
Well, most languages would have some variant of that word to mean "television", as they do now, I expect.
The English word "galore" (meaning "sufficient" shading towards "more than enough") comes from the Gaelic words "gu leòr", (goo lyaawr, the grave accent above the o makes the vowel sound longer). What a silly language English is, doesn't have a word that means "more than you're ever likely to need", has to steal one from Gaelic and then spell it wrong.
Oh, they use this word "whisky". You know what that means? It means "uisge beatha" but they only say the first word, in a silly accent because they can't pronounce it properly.
Quite often there's no single word for a thing you're trying to translate but that doesn't mean it's untranslateable. English has only one single word for rain, for example, but Gaelic has about half a dozen of which the only ones I can reproduce here are "uisge" (that word again) which just means "water", and "fras" which is more like a gentle shower. The rest of the words in the Gaelic of the North-West of Scotland that refer to rainy weather are, of course, profane in the extreme.
Even if the number of words in a language were finite we wouldn't have a reasonable way of counting them. There are too many kinds of fuzziness involved in deciding what counts as a "word" and you can't ignore the borderline cases because the borderline cases vastly outnumber the straightforward cases.
You're forgetting about synonyms. The common adage that English has the largest vocabulary stems from the fact that it often has multiple words for the same thing. Sofa, couch. Autumn, fall. Etc etc. Other languages generally don't do this. I've never heard anyone suggest that English has words for more concepts.
So in this case, I see the diagrams as representing the boundaries drawn when projecting / quantizing complex ideas into a set of central points that are insufficient for catching all of the nuance of the original. How well can you adapt a nuanced idea to a different space?
If Language A has an idea that exists at one point in space, which is the closest word in Language B that might be used to represent it? A Voronoi diagram is one possible way of illustrating it.
Tangent on your tangent: this GDC presentation from 2016 is probably my favorite real-world application of Voronoi Diagrams, and uses them for N-player split-screen camera control: https://www.youtube.com/watch?v=tu-Qe66AvtY&t=1594s
I have a lingering dream in the back of my mind to make a single-couch Liero-style casual game for N-players with good dynamic camera support using this technique.
You could consider the "cost" of expressing a word as some kind of metric or norm on the vector. What in one language/basis is a simple Kronecker delta, in another is a very complex vector (of course if it were the same vector in two bases, it would have the same length, but we could rather think of translation as an affine transformation, say).
And finally, with two bases, they need not span the same vector space. You can have a three-coordinate vector space all you like, if you have only two basis vectors you ain't spanning it. At best you can hope for an orthogonal projection from one to the other, and lose some nuance.
Eventually, with bilinguality, you learn not to translate words. Concepts live in different languages and describe a reality. Usually you can describe that reality in two different languages, but sometimes not.
I tried to find the really interesting article about language and color that describes how some cultures use different naming schemes for colors but couldn’t find it. It talked about how back in the day we don’t know orange as a color, we just thought it was red-yellow and only after the fruit was distributed did the word for the color catch on. Here’s the best article I can find that talks about this phenomena https://burnaway.org/magazine/blue-language-visual-perceptio...
Yes, that mathematical expression is like a punch in my face, but not for the reason you think. I am offended that the rank of the matrix does not match the dimension of the matrix, not that I'm seeing a matrix.
Concepts are only usefully distinguished by context and use.
By the author's own argumentation: nothing is translatable (or, generally, even communicatable) unless it has a fixed relative configuration to all other concepts that is precisely equivalent. In practice, we handle the fuzziness as part of communication and its useless to try and define a concept as untranslatable unless you're also of the camp that nothing is ever communicated (in which case, this response to the author's post is completely useless as nobody could possibly understand it enough internally for it to be useful. If you've read this far, congrats on squaring the circle somehow)
The same process that allows two speakers of the same language to communicate adequately allows one to translate from one language to another. If it were truly impossible to translate from one language to another, we would be unable to perceive this and argue about it. The recognition and correction of errors is part of the process of translation just as it is part of the process of communication in a single language.
[1] https://plato.stanford.edu/entries/quine/#IndeTran
That indeterminacy of translation isn’t mentioned is a huge shortcoming of this article.
If you think back to the meme from a decade or two ago about how men and women perceive colour [1], where e.g. "pink" to a man covers a whole range of colours to a woman, then that kind of hints at the idea.
One example back in the realm of vocabulary is the English word "happy". This embodies a range of meanings from joy, willingness, pleased, contentment, satiation, etc. There might be some overlap in some of these meanings with other words like "joy" or "excited", that don't have the same overlaps in other languages. E.g. "happy" might be translated to French as "heureuse" for the senses of pleased or content, but not for willingness sense.
Similarly, the French word "dommage" can be translated into a whole bunch of English words that aren't normally synonyms of each other - pity, damage, shame, harm.
This kind of nuance can lead to two opposite problems when translating - when the meaning is limited to a subset of possible meanings by context, and the wrong one is chosen in the foreign language, and when the author's meaning embodied multiple meanings and the chosen translation doesn't cover all of them.
Some of these features can lead to the humour in subtle jokes being lost in translation, e.g. "he'd be late to his own funeral".
[1] e.g. https://www.psychologytoday.com/us/blog/brain-babble/201504/... or https://digitalsynopsis.com/design/male-vs-female-color-perc...
In other words:
The source sentence is a vector in “language A space.”
The target sentence is a vector in “language B space.”
A good translation finds a vector that has the same direction (same meaning, intent, tone) even though it lies in a different coordinate system (the new language).
But I think this exposes an even greater problem, where words thought to be direct translations will always drift in vector value as they are weighted for attention within their respective corpora. Are we on the brink of translation-nihilism?
This isn't even limited to complex phenomena or shades of snow. Even "I like" is a different construction in many languages, in an unexpected way to new language learners.
N^{any constant} is not bijective with a single R.
If both these thoughts are true, then it would appear that languages have topological characteristics. We can (topologically) map from one to another, 'thoughts' (that is a complex of words) form 'paths on the language manifold' and certain paths may be more 'natural' in one topological form than the other.
Go translate an ee cummings poem and make sure to retain all its meanings.
There's an xkcd devoted to this problem, even using computational linguistics as an example, IIRC.
Vague: does it mean all the inflected forms of a word, or just the stem without inflection? Example: is (are?) 'walk', 'walked', 'walks' and 'walking' four words, or one? What about "stand/stood"? (And languages where the bare root or stem can't appear by itself, like verbs in Spanish.) Derived words, like 'push' and 'pushy'.
Do compound nouns count as a word, or do only the parts count? Example: 'doghouse' (or 'dog house'). What about idioms? Example: 'to crane his/her/my/your/our neck(s)'.
What about different pronunciations? Is 'roof' pronounced to rhyme with 'aloof' the same word as 'roof' pronounced with the vowel of 'put'? And different spellings but same pronunciation: 'bear' vs. 'bare'.
What about words with different grammatical categories, like 'push' as a noun ("I gave her a push") or a verb ("I pushed her"). Or the same word with virtually unrelated meanings, "I pushed her on the swing" vs. "I pushed my ideas."