I agree with the take, but it's a temporary one, the sad reality is that we will be literally inferior soon, there will be a point where we will not trust human input without counter check by AI, we need to remember that we are kinda at the beginning of the AI era, in 5 to 10 years it's very unlikely that a human translator or software engineers will do better than the tooling we will have.
There is already a tipping point now in software engineering where we prefer to ask AI instead of humans because we believe accuracy will be better, see SO death as an example or just see the current state of online dev communities, it's getting deserted and between team members at work, we can also notice that people speak less and less.
I think it's an interesting perspective, because translation is one of the jobs that I (a) hear is the first to lose work due to AI, and (b) often used as an example of "acceptable" AI by people who are skeptics of LLMs and AI-generated art.
I think the only people that genuinely think AI translation is viable are those that never read in the first place.
There are a number of games I've seen and played on Steam which use MTL to get the game playable in English and universally the translation is absolute shit. Sentences that don't transition into another, wordplay that becomes nonsense and a completely flat affect to everything.
I have no doubt that the writer is better at translating than AI, but I have to say that AI translation has gotten so good that I'm not sure how much longer translation work will be there, or rather it might end up being more about auditing.
For example, I just read the Lawrence Ellsworth translation of The Three Musketeers, which I very thoroughly enjoyed. I don't speak or read French, but from my understanding Ellsworth's translation is considered one of the more accurate translations of the work.
Out of curiosity, I sic'd Claude Fable on the original French version of The Three Musketeers and told it to translate accurately, but also try and keep the same jovial tone as the original and do not censor anything. After it was done, I didn't read the entire output, but I did compare a few individual chapters between the Ellsworth translation and the Fable translation.
They were honestly remarkably similar. As far as I could tell, nothing was substantially different from the Ellsworth translation and the Fable translation. I do think that the prose for the Ellsworth translation was a bit better, but the prose for the Fable one was actually perfectly readable. Again, I don't speak French so I cannot say for sure, but I do not believe that I would have gotten a significantly different experience had I read the Fable version instead of the Ellsworth version.
Now, it's possible (and likely) that this is somewhat self-fulfilling; Fable might have been trained using Ellsworth's translation and as such it's very directly able to crib from it; sadly since I do not speak any language outside of English, there's sort of a catch-22: the only way I can compare the accuracy of a translation is to compare against other translations, but if other translations exist then that will likely influence the results, and if a translation doesn't already exist then I have no way of auditing it.
I'm still going to continue reading through Ellsworth's translations for the subsequent stories simply because that feels more canonical, and as I said I do think the prose was a bit better.
> but I have to say that AI translation has gotten so good that I'm not sure how much longer translation work will be there, or rather it might end up being more about auditing
It's functional? I wouldn't say it's poetic, I wouldn't want any AI translator translating art, like say a book or poem, I'd be so uncertain that it would correctly bridge the concepts
A good translator can make stylistic choices that elevate the work and make it fit in their language
(Having read lots of well translated manga and anime, also from what I understand there's a few books I've been told by my bilingual friend's are just chef's kiss quality translations)
Considering translating meaningful art is of some value, on that score I don't think we're there yet
> I have to say that AI translation has gotten so good
> I do not speak any language outside of English
So you are entirely unqualified to judge this, and you acknowledge yourself that your test is flawed to the point of being completely useless. Yet you make grandiose statements about the quality.
I learned last year that "translation" can be a very tricky thing. Because there's never a one-to-one correlation between one language's words, phrases, structures and metaphors, and another language's equivalent stuff. And LLM translations may not be the actual translation you want, or need.
I wrote up my experiences of translating Lorca and Cavafy poems here[1]. tl;dr: I have developed a massive new respect for translators; however much they're being paid, they probably need to be paid more!
An honest to god article full of em dashes that's not because it was AI but because it was a human using them as a crutch to get around crafting sentences that flow naturally. Almost brings a tear to my eye.
Presumably the people paying the author for translation services are aware of AI, but for whatever reason are choosing a humans services instead. IMO it would be a form fraud to heavily rely on AI and not disclose that to the customer.
wrt. the end of the story, it will be interesting to see if people start noticing their Dunning-Kruger bias as a result of LLMs.
Specifically: LLMs make it really easy to misunderestimate the complexity of fields other than your own. (You can see this with a lot of vibecoded projects, for example – once they hit the wall of complexity, they stall out or start finding ugly patches for fundamental design issues, etc.)
I don't think this sort of cultural change will happen short-term, though.
All I got out of this article is that he should have went home and dumped it into chatgpt just to see what happened; then if it did as good a job as him, he should start looking for other places he can add value that AI can't.
I worked at large Japanese bank in New York and happened to sit near Chief US Economist next to his Japanese translator. She would occasionally ask about certain idioms. I remember explaining what a wildcat strike was for instance. But it must have been pretty tough because the guy was prolific in his commentary.
Translating is one thing that artificial intelligence undeniably excels at, and the value of this alone is enough to underpin the trillion dollar valuations of the gigantic AI companies.
Translation is a gigantic boon for business, but just as important for human connection, for culture, science, art, and entertainment. The value of automatic and cheap translation between all languages, this tower of Babylon, is immeasurable.
Human translators will always be better than any AI at their job. But they don't have unlimited time and energy, and they aren't cheap. AI makes good to great translations available to everybody.
The ending is a really powerful point. Most people apparently agree on two things:
1. AI is a great boon for all tasks and specialties we don’t have the skills to do ourselves. Understandable, since (A) we’re ill equipped to see the flaws in its output because it isn’t our area of expertise, and (B) it often can unlock great gains because if we trust it, we then don’t have to pay and wait for humans to do that thing.
2. AI is a terrible replacement for me - my skills are at such a high level that it’s almost theoretical that it’ll ever be good enough to replace me for 90% of what I get paid to do. It’s a tool at best.
This is why I use AI for all my medical questions and doctors use AI to write software, and we both smirk at the quality the other person is getting from it.
Honestly, we're at a point where AI can write better software than some devs and answer medical questions with more knowledge than some doctors.
Likewise, AI is oblivious to it's own mistakes, much like said professionals can be at times.
Not that AI is actually thinking, but rather the collective corpus of text yields greater insights (knowledge of the crowd, not wisdom of the crowd) than a lower-average person in that same industry.
We should create a generalized version of "Gell-Mann Amnesia". This applies not just to fields of study. But also to time and space. We read history as if the person who wrote the history book has perfect knowledge.
The fundamental issue imo is that LLMs are trained to make believable outputs. They can be complete horseshit, but because they look plausible they get treated like quality.
I swear that the intensity and time I've had to take with code reviews has gone up because LLMs are so good at making flawed code look good. I assume the same goes for everything else we use LLMs for.
my skills are at such a high level that it’s almost theoretical that it’ll ever be good enough to replace me for 90% of what I get paid to do.
Is it really true for most people that they are using their core advanced skills 90% of the time? I'm curious about how people feel about this.
I'm a professor, which is supposed to be an intellectually demanding job. I do research in NLP/AI, and I don't think AI will replace my core intellectual tasks in the near future, but I don't think my core intellectual tasks represent even 10% of my time. Most of the time is taken by various things like writing bureaucratic reports, writing and polishing grant applications, grading exams and exercises, designing a poster, planning a course's calendar for a given year, creating a figure for slides, writing assignments and exams, attending teaching coordination meetings... which definitely are or should be automatable. Probably even teaching the same lesson for the umpteenth time also is from an objective point of view, we'll probably be kept doing it due to the human factors driving motivation but not because a lecture given by a human is intellectually superior.
This reminds me of an exchange I had (this would have been 2018 or so) with a colleague I was visiting when I worked for a large retailer. She had a ton on her plate, and was in charge of running an important department. She lamented privately to me that the company had specifically banned the hiring of anyone as an "assistant." She was spending hours of her highly-paid time doing tasks similar to what you listed here, which didn't require any of her important skills, things like filling out expense reports or scheduling meetings. I could see how we were getting a lot less productivity from her because of this dogmatic idea that was no doubt done more for optics than anything else.
It seems like that company's policy is pretty much pervasive - outside the C-suite, no one has an assistant no matter how well paid they are, but we give them a ton of silly busywork to waste their time on.
To bring this back to the subject at hand, I do hope that "AI" can crack the literal "assistant" duties. If it did, it really could be a net good for society and business, given that it wouldn't even be displacing human assistants, since they just fell out of fashion in 1995 or whatever.
AI isn’t replacing me. Like a toddler, it
needs to be constantly coached.
Like a toddler, it will grow up.
Humans are really bad at noticing trajectories. They see the current situation. They know what the situation was 5 years ago. But for some reason they do not believe that there is a trajectory. They view the present state as the final destination.
This is just about the worst career you could be in right now. Of course people are just going to upload it to ChatGPT. Processing text is its forte.
This person is in the first stage of grief (denial); artists are several stages ahead. Most customers are not going to care about the difference in translation quality unless it's in a regulated sector.
Out of curiosity, I pasted an article in French I was reading a few minutes before coming across this thread into ChatGPT and asked for a translation into English. It was certainly passable from a functional perspective, and I wouldn't hesitate to use it to translate an article from a language I don't understand. But it was not professional-quality work. There were a couple instances where the French grammar was mistranslated, and the writing was perfunctory, not going into any effort to have the article flow like it was originally written in English instead of simply translating each sentence literally. Would I read an article written like this? A short one. A novel? Definitely not.
> Should you pay your roofer less because he uses a hammer instead of his bare hands?
Yes. Effective tools increase the supply of roofs made. More supply means lower prices per roof. But because the same number of roofs need to get worked on, the increase in roofs per roofer means less roofers will be needed.
AI should be used for all the bullshit tasks that no one wants to do. There are garbage dumps full of stuff that can be reused and recycled. But it's not high enough ROI to pay someone $25/h to sort trash, so it isn't happening.
95 comments
[ 2.6 ms ] story [ 79.3 ms ] threadThere is already a tipping point now in software engineering where we prefer to ask AI instead of humans because we believe accuracy will be better, see SO death as an example or just see the current state of online dev communities, it's getting deserted and between team members at work, we can also notice that people speak less and less.
Sad but I believe it.
There are a number of games I've seen and played on Steam which use MTL to get the game playable in English and universally the translation is absolute shit. Sentences that don't transition into another, wordplay that becomes nonsense and a completely flat affect to everything.
> Ah, you can’t fire me, I’m self-employed!
I don't understand thinking like this. I think companies can certainly fire their contractors.
For example, I just read the Lawrence Ellsworth translation of The Three Musketeers, which I very thoroughly enjoyed. I don't speak or read French, but from my understanding Ellsworth's translation is considered one of the more accurate translations of the work.
Out of curiosity, I sic'd Claude Fable on the original French version of The Three Musketeers and told it to translate accurately, but also try and keep the same jovial tone as the original and do not censor anything. After it was done, I didn't read the entire output, but I did compare a few individual chapters between the Ellsworth translation and the Fable translation.
They were honestly remarkably similar. As far as I could tell, nothing was substantially different from the Ellsworth translation and the Fable translation. I do think that the prose for the Ellsworth translation was a bit better, but the prose for the Fable one was actually perfectly readable. Again, I don't speak French so I cannot say for sure, but I do not believe that I would have gotten a significantly different experience had I read the Fable version instead of the Ellsworth version.
Now, it's possible (and likely) that this is somewhat self-fulfilling; Fable might have been trained using Ellsworth's translation and as such it's very directly able to crib from it; sadly since I do not speak any language outside of English, there's sort of a catch-22: the only way I can compare the accuracy of a translation is to compare against other translations, but if other translations exist then that will likely influence the results, and if a translation doesn't already exist then I have no way of auditing it.
I'm still going to continue reading through Ellsworth's translations for the subsequent stories simply because that feels more canonical, and as I said I do think the prose was a bit better.
It's functional? I wouldn't say it's poetic, I wouldn't want any AI translator translating art, like say a book or poem, I'd be so uncertain that it would correctly bridge the concepts
A good translator can make stylistic choices that elevate the work and make it fit in their language
(Having read lots of well translated manga and anime, also from what I understand there's a few books I've been told by my bilingual friend's are just chef's kiss quality translations)
Considering translating meaningful art is of some value, on that score I don't think we're there yet
> I do not speak any language outside of English
So you are entirely unqualified to judge this, and you acknowledge yourself that your test is flawed to the point of being completely useless. Yet you make grandiose statements about the quality.
I wrote up my experiences of translating Lorca and Cavafy poems here[1]. tl;dr: I have developed a massive new respect for translators; however much they're being paid, they probably need to be paid more!
[1] - https://rikverse2020.rikweb.org.uk/blog/adventures-in-poetry...
Also, I like em dashes.
And if this is my worst sin, so be it.
> “Oh, I can’t! It’s really not reliable enough.”
Gell-Mann Amnesia strikes again.
Specifically: LLMs make it really easy to misunderestimate the complexity of fields other than your own. (You can see this with a lot of vibecoded projects, for example – once they hit the wall of complexity, they stall out or start finding ugly patches for fundamental design issues, etc.)
I don't think this sort of cultural change will happen short-term, though.
Every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon.
Lemma: any statement about AI which uses the word "never" to preclude some feature from future realization is false.
Lemma: contemporary implementations have already improved; they're just unevenly distributed.
Translation is a gigantic boon for business, but just as important for human connection, for culture, science, art, and entertainment. The value of automatic and cheap translation between all languages, this tower of Babylon, is immeasurable.
Human translators will always be better than any AI at their job. But they don't have unlimited time and energy, and they aren't cheap. AI makes good to great translations available to everybody.
1. AI is a great boon for all tasks and specialties we don’t have the skills to do ourselves. Understandable, since (A) we’re ill equipped to see the flaws in its output because it isn’t our area of expertise, and (B) it often can unlock great gains because if we trust it, we then don’t have to pay and wait for humans to do that thing.
2. AI is a terrible replacement for me - my skills are at such a high level that it’s almost theoretical that it’ll ever be good enough to replace me for 90% of what I get paid to do. It’s a tool at best.
This is why I use AI for all my medical questions and doctors use AI to write software, and we both smirk at the quality the other person is getting from it.
Likewise, AI is oblivious to it's own mistakes, much like said professionals can be at times.
Not that AI is actually thinking, but rather the collective corpus of text yields greater insights (knowledge of the crowd, not wisdom of the crowd) than a lower-average person in that same industry.
I swear that the intensity and time I've had to take with code reviews has gone up because LLMs are so good at making flawed code look good. I assume the same goes for everything else we use LLMs for.
Is it really true for most people that they are using their core advanced skills 90% of the time? I'm curious about how people feel about this.
I'm a professor, which is supposed to be an intellectually demanding job. I do research in NLP/AI, and I don't think AI will replace my core intellectual tasks in the near future, but I don't think my core intellectual tasks represent even 10% of my time. Most of the time is taken by various things like writing bureaucratic reports, writing and polishing grant applications, grading exams and exercises, designing a poster, planning a course's calendar for a given year, creating a figure for slides, writing assignments and exams, attending teaching coordination meetings... which definitely are or should be automatable. Probably even teaching the same lesson for the umpteenth time also is from an objective point of view, we'll probably be kept doing it due to the human factors driving motivation but not because a lecture given by a human is intellectually superior.
This reminds me of an exchange I had (this would have been 2018 or so) with a colleague I was visiting when I worked for a large retailer. She had a ton on her plate, and was in charge of running an important department. She lamented privately to me that the company had specifically banned the hiring of anyone as an "assistant." She was spending hours of her highly-paid time doing tasks similar to what you listed here, which didn't require any of her important skills, things like filling out expense reports or scheduling meetings. I could see how we were getting a lot less productivity from her because of this dogmatic idea that was no doubt done more for optics than anything else.
It seems like that company's policy is pretty much pervasive - outside the C-suite, no one has an assistant no matter how well paid they are, but we give them a ton of silly busywork to waste their time on.
To bring this back to the subject at hand, I do hope that "AI" can crack the literal "assistant" duties. If it did, it really could be a net good for society and business, given that it wouldn't even be displacing human assistants, since they just fell out of fashion in 1995 or whatever.
This is more "humans are special" hubris imo. Not saying it's gonna happen tomorrow but look at the advancements from just 2019 to now.
It's unwise to say it'll never happen.
A list of "Examples AI will silently fail at" would be a lot more interesting, and might just convince your next potential client to _not_ use AI.
Humans are really bad at noticing trajectories. They see the current situation. They know what the situation was 5 years ago. But for some reason they do not believe that there is a trajectory. They view the present state as the final destination.
This person is in the first stage of grief (denial); artists are several stages ahead. Most customers are not going to care about the difference in translation quality unless it's in a regulated sector.
Yes. Effective tools increase the supply of roofs made. More supply means lower prices per roof. But because the same number of roofs need to get worked on, the increase in roofs per roofer means less roofers will be needed.