Sharing this in hopes it helps other founders who are scared l10n is expensive/bad quality when using Claude Code and other coding agents, it doesn't have to be.
It is absolutely revolting to see LLM translation being sold as "better than professionals". Yeah, yeah, I saw you attached a caveat to it. No. You don't need to be dishonest. LLM translation is shit but it's borderline free, and you want the benefit of translation without having to pay for it. Who can argue with the economics of "free", after all?
I would leave it at that, but then I consider what you're actually using it for...
> two-person founding team running a language-learning app in dozens of languages, so the copy is read by people who are unusually sensitive to it.
They aren't "unusually sensitive to it". Language learners don't fucking know what native speech in the target language sounds like, they're learning it! Nor does your two person team. How could you possibly claim to objectively evaluate how good translations are in dozens of languages you don't speak? 58, according to your site! This is beyond scummy even for the usual AI boosters, and will deceive people trying to learn, who don't know better, with low-grade slop. Disgusting.
Language learners need to be able to use the site in their native language. The whole world doesn’t feel comfortable using English as a second language to learn another language. None of my students use English to learn Spanish or Mandarin Chinese. They use their mother tongues so they don’t have to add to their cognitive load by processing another language on top of the one they're already learning.
The localized sites are exactly what they can judge as native speakers. They’re judging copy in their own language, not the language they’re learning.
We noticed that a good share of our learners were already using Google Translate built into their browsers to access the site in their native languages. The quality is atrocious, so what we’re offering is a step change from that.
We offer 58 languages to learners. Only 16 have been localized. That means the chrome of the website and apps -- the learner’s native language -- not the video content they watch in the target language they’re learning.
Between my co-founder and me plus people who have kindly offered to review the site we can judge seven of those localizations well enough to see that they’re orders of magnitude better than what Google Translate would concoct for them.
Between the two of us, we do speak 7 languages well enough to judge a localized site, and we have people around us for review on some more.
We run our workflow for a new localized version of say, German, then judge the quality and add rules to our prompt, guidance and voice guide for issues we spot, never hand-editing the actual strings.
We then re-run it until it's in a place where we feel this feels fairly native (feels transcreated, not calqued) and serves its purpose well).
Once we had that done in a good amount of locales, we started running it in languages we can't judge ourselves. Here we do depend on people on the app/website to spot and help us if we got it wrong somewhere, so it's a living process that evolves.
English can use "upgrade" as an intentionally underspecified one-word label. On, say, a paywall, Spanish would naturally use a construction like "pasar a [otro plan]," and French “passer à [une autre offre].” The preposition a/à is how that particular meaning is expressed in those languages.
So it isn’t randomly translating “upgrade" as "upgrade to." The context tells you that "upgrade” means "move to a different plan," rather than "update the software."
You're right about the random capitalization though - it wasn't intentional and we'll edit that bit to avoid confusion.
Huh. This article is about the issue of picking which translation of a word to use according to what the word means in context... a task that LLMs should ace.
I was hoping this was about using an LLM to produce text in the house style. That's also a task they should be good at, but that clearly doesn't happen by default.
Here's a two-page public announcement from Ironhide Games, who make Kingdom Rush. They're apologizing over a direction they chose for the upcoming 6th game in the series:
- "People complained, and we're going in a different direction."
- "We had this translated from Spanish by an LLM."
The 11 paragraphs of text are superfluous, because... (1) anything you could learn from them you could also learn from the two-bullet-point summary on the second page; and (2) the text style clearly doesn't reflect anything that the spokesperson for Ironhide Games wrote. This is a document written specifically to prevent any information from coming through other than the two bullet points in the summary.
And that's a style that companies often invoke on purpose. PR statements are vetted for this kind of thing, which is why everyone hates them. But it appears to be the only style that LLMs want to produce, whether they're making a PR statement or not, and I feel sure that almost none of the people using them for translations actually want this.
I also feel sure that the "LLM style" is specifically intended by the LLM providers, and that it wouldn't exist if they didn't spend enormous effort causing it to.
This is interesting! Exactly our experience, you do have to guide LLMs on author the source text in the way we would write, with few-shot examples in the prompt. And then prompt them to transcreate so it doesn't calque (read as a literal translation rather than something a native speaker would have written for other native speakers). Both of this can be solved for reasonably well with the right prompt guidance IMHO.
In my experience, there's two key incredients to get to a good results (with humans OR LLMs doing the work), I've worked on plenty of sites that had human l10n before LLMs came about.
9 comments
[ 5.7 ms ] story [ 30.1 ms ] threadI would leave it at that, but then I consider what you're actually using it for...
> two-person founding team running a language-learning app in dozens of languages, so the copy is read by people who are unusually sensitive to it.
They aren't "unusually sensitive to it". Language learners don't fucking know what native speech in the target language sounds like, they're learning it! Nor does your two person team. How could you possibly claim to objectively evaluate how good translations are in dozens of languages you don't speak? 58, according to your site! This is beyond scummy even for the usual AI boosters, and will deceive people trying to learn, who don't know better, with low-grade slop. Disgusting.
The localized sites are exactly what they can judge as native speakers. They’re judging copy in their own language, not the language they’re learning.
We noticed that a good share of our learners were already using Google Translate built into their browsers to access the site in their native languages. The quality is atrocious, so what we’re offering is a step change from that.
We offer 58 languages to learners. Only 16 have been localized. That means the chrome of the website and apps -- the learner’s native language -- not the video content they watch in the target language they’re learning.
Between my co-founder and me plus people who have kindly offered to review the site we can judge seven of those localizations well enough to see that they’re orders of magnitude better than what Google Translate would concoct for them.
Between the two of us, we do speak 7 languages well enough to judge a localized site, and we have people around us for review on some more.
We run our workflow for a new localized version of say, German, then judge the quality and add rules to our prompt, guidance and voice guide for issues we spot, never hand-editing the actual strings.
We then re-run it until it's in a place where we feel this feels fairly native (feels transcreated, not calqued) and serves its purpose well).
Once we had that done in a good amount of locales, we started running it in languages we can't judge ourselves. Here we do depend on people on the app/website to spot and help us if we got it wrong somewhere, so it's a living process that evolves.
Here's an example of how that guidance and prompt evolved over time, in case you're interested: https://gist.github.com/tobyurff/2b461c259c34dfc6758a3932982...
So it isn’t randomly translating “upgrade" as "upgrade to." The context tells you that "upgrade” means "move to a different plan," rather than "update the software."
You're right about the random capitalization though - it wasn't intentional and we'll edit that bit to avoid confusion.
I was hoping this was about using an LLM to produce text in the house style. That's also a task they should be good at, but that clearly doesn't happen by default.
Here's a two-page public announcement from Ironhide Games, who make Kingdom Rush. They're apologizing over a direction they chose for the upcoming 6th game in the series:
https://cdn.discordapp.com/attachments/524968371494846484/15...
https://cdn.discordapp.com/attachments/524968371494846484/15...
To me the main message is twofold:
- "People complained, and we're going in a different direction."
- "We had this translated from Spanish by an LLM."
The 11 paragraphs of text are superfluous, because... (1) anything you could learn from them you could also learn from the two-bullet-point summary on the second page; and (2) the text style clearly doesn't reflect anything that the spokesperson for Ironhide Games wrote. This is a document written specifically to prevent any information from coming through other than the two bullet points in the summary.
And that's a style that companies often invoke on purpose. PR statements are vetted for this kind of thing, which is why everyone hates them. But it appears to be the only style that LLMs want to produce, whether they're making a PR statement or not, and I feel sure that almost none of the people using them for translations actually want this.
I also feel sure that the "LLM style" is specifically intended by the LLM providers, and that it wouldn't exist if they didn't spend enormous effort causing it to.
In my experience, there's two key incredients to get to a good results (with humans OR LLMs doing the work), I've worked on plenty of sites that had human l10n before LLMs came about.
If you're interested in the actual prompt we're using for our voice that includes instructions on how to avoid sounding like bad LLM copy, I've popped that here in a Github Gist: https://gist.github.com/tobyurff/2b461c259c34dfc6758a3932982...
Hope it helps, I might turn that into a blog post one day as well, seems like this could be useful to others!