This is very neat. I’d love to see Hebrew and Mandarin added, though (each is likely to be challenging in its own way given the way sentences are matched).
The most popular tokenizer for Japanese is probably MeCab. It can also give you readings, but in cases where the reading depends on the meaning of the sentence, it's about as likely to get it wrong as to get right. My current approach is to also use Kuromoji and compare the output so that I can at least notice which parts might be inaccurate.
Really cool concept, love the split view. The voiceover feature is nice, but I think an inline dictionary would be even more useful - maybe something along the lines of macOS' built-in lookup (https://i.imgur.com/qGTQ80a.png)
> I think an inline dictionary would be even more useful
Yes this. It would make a huge difference for me at my level. I'm finding that I lose my focus on specific words. I can gather the general meaning from the sentence and/or paragraph but...
Translating the entire paragraph is too much of a crutch and forces me back into thinking in English. If I can get past the single word(s) with a click, I can keep my thoughts entirely in the second language. I suppose an extension could probably fix this...
Seems like the paragraph matching is still pretty buggy.
For example I chose French as the language to get better at and selected "Tour du monde en 80 jours" as the text. The very first paragraph of the French text is
> En l'année 1872, la maison portant le numéro 7 de Saville-row, Burlington Gardens -- maison dans laquelle Sheridan mourut en 1814 --, était habitée par Phileas Fogg, esq., l'un des membres les plus singuliers et les plus remarqués du Reform-Club de Londres, bien qu'il semblât prendre à tâche de ne rien faire qui pût attirer l'attention.
and the English translation is
> Mr. Phileas Fogg lived, in 1872, at No. 7, Saville Row, Burlington Gardens, the house in which Sheridan died in 1814. He was one of the most noticeable members of the Reform Club, though he seemed always to avoid attracting attention; an enigmatical personage, about whom little was known, except that he was a polished man of the world.
Notice how there's an extra sentence at the end about him having an enigmatic personage not present in the French version. Indeed that's in the next paragraph. And the matching goes basically out of whack. The second paragraph of the English is
> People said that he resembled Byron--at least that his head was Byronic; but he was a bearded, tranquil Byron, who might live on a thousand years without growing old.
But the third paragraph of the French is
> On disait qu'il ressemblait à Byron -- par la tête, car il était irréprochable quant aux pieds --, mais un Byron à moustaches et à favoris, un Byron impassible, qui aurait vécu mille ans sans vieillir. Anglais, à coup sûr, Phileas Fogg n'était peut-être pas Londonner.
They don't tell where they took the translations from and who created these translations. Given your examples, I assume the whole thing is based on Gutenberg or some similar, not on machine translations. If it were humans who made these translations, deviations as you describe them are to be expected. Literary translations ofter are rather some kind of re-creation or interpretation of the original work.
Yeah, we took those translations off of a pre matched corpora of books. We did, however develop later a system for automatic matching of translations, but unfortunately didn't get to use it
Yes, the problem is most translations aren’t literal one-to-one translations. Literal translations are usually hard to read so some translators use “dynamic equivalency” while others paraphrase heavily. Unfortunately this can make machine linguistic matching difficult and unreliable. For instance, there are two translations of the Nordic classic Kristin Lavransdatter which both read differently and there are people who will argue passionately about which translation is best.
Statistical machine translation uses curated parallel texts for training, but they tend to match with multiple corpora so the translation is some sort of average I believe. I wonder if matching with just one translation might produce less reliable results?
We do have a tool that does real paragraph matching based on the Gale–Church alignment algorithm and offline dictionaries. On top of that we have an additional manual process to make sure that the alignment is correct. However many of the books were pre matched and we didn't align them using our tool.
Like the concept as I am actively learning Dutch right now but I find it hard to copy a word without accidentally clicking on the sentence to play it out.
I want to copy words to keep my own word list and to translate the word.
I looked at Robinson Crusoe, the Dutch looks really old. Spelling is incorrect (according to modern standards), you'll talk a bit like a Dutch Shakespeare if you'll learn it this way.
Well as someone else commented, the source material is likely from Project Gutenberg, so a long out of copyright translation is perhaps expected.
On the other hand, Cervantes and Shakespeare were contemporaries, so I think it enhances reading Don Quixote for the translation to use early modern English - e.g. "Idle reader: thou mayest believe me without any oath".
Also, in the german text the chapter markers ("Erstes Kapitel", "Zweites Kapitel") have been pulled into the paragraph following them, but they're missing completely in the english text.
On Firefox (locale is English): I speak Whatever, I want read Whatever else. Choose any book. Voice is English. The only exception is if I choose I want to read Spanish, but this is because I have installed a particular Spanish voice in my system, and that's the one that's used.
On Vivaldi (browser's locale is Spanish): Anything is read with a Spanish voice, unless I choose I want to read English, then an English voice is used.
Yeah I had the same issue. If my device locale is English the voice reads it like an apathetic 10th grader who refuses to try the accentuate properly while read French aloud in class.
On the bright side it was a trip down memory lane to high school.
It's not even backwards. The voice doesn't match the languages chosen. On a machine with Danish locale it will read the french text with a danish voice. (I chose "learn french/know english")
I like the idea. Only the (very few) available books require even in my own language a pretty advanced level. I would use this for sure if the reading level of the books available would be more for children so I can learn words and sentences that I can actually use and practice in daily life.
Because of legal reasons we can only have books from the public domain. If you know some that are free, just point me to them and I'll match them for you. I need them to be in .epub format for both languages.
Lingq.com is a mature platform to learn languages by reading and listening. The only app I'm not abandoning after few days, because I can read stuff that's really interesting to me (because I can import articles and entire ebooks). This video explains how it works: https://www.youtube.com/watch?v=QeqFrO1sbTM
I took a quick look at English / Swedish, and noticed that the Swedish version of Three Men in a Boat is badly encoded: the (vital) Swedish letter å has been consistently replaced with a. On the other hand, ä and ö seem to be present as expected.
So (based on a sample size of one), I'm afraid it doesn't look like the translations are necessarily reliable.
I've previously struggled with the encoding of books on Project Gutenberg. In theory each text file has a header that specifies the encoding and other data about the book like author and title. In practice the header format varies unpredictably (likely it was written manually and not meant for machine consumption) and even if you can parse the encoding value, you might have one that says ASCII but is actually some Windows code page, or UTF-8, or something completely different. And that encoding is also used in other parts of the header, so it can happen that you fail to parse the header because you don't know which encoding to use. One book actually used different encodings in different parts.
After collecting a messy tangle of special cases over several days, I threw in the towel and just used BeautifulSoup's UnicodeDammit to brute-force a working encoding.
Maybe the Swedish book didn't indicate a vs å in the first place, though.
Not sure this one came from Gutenberg; at least, I can't seem to find a Swedish version of it there. But anyhow, either the source is worthless, or some conversion process somewhere was bad. A vs Å is not a minor or optional issue in Swedish; å, ä and ö are separate letters of the alphabet in their own right, and "omitting accents" isn't a thing. It's just wrong.
I had the same thought. Some text is hard to read as well. For example:
-- Harris sade, att han drabbades av sa utomordentligt starka yrselanfall emellanat, att han knappt visste vad han gjorde; och da sade George att han led av starka yrselanfall, och knappt visste vad han gjorde.
I'm reminded of a book on topology I read which suggested learning how to read mathematical text in French with a particular book by -- I think it was Bourbaki, "using the English translation as a pony". I've never tried it, but supposedly you could get pretty far this way.
Oh the poor people who try to learn German by reading Kafka.
I'm not sure literature in general is suitable for learning a language that you don't already have a tight grip on. Maybe children's books, but even then it's too far removed from spoken language for some languages.
I've found the concept of extensive reading with a 98% threshold (that is, you must be understand 98% of words on a page) to be pretty effective in picking up foreign languages. Less than that and I give up. You're right, starting to learn German through Kafka is going to lead to bad times. I tried that with Mandarin and a translation of an Orhan Pamuk novel and gave up some twenty pages in after a brutal slog.
Having some simpler and easier selections ala Lingua Latina (https://lingualatina.dk/wp/) would definitely help. If you haven't heard of it, that is a book for learning Latin Wooten entirely in Latin. It starts off with very simple grammar and vocabulary and adds more each chapter. You can find digital copies pretty easily.
I liked Readlang's approach where you click words or expressions that you don't understand and you get the translation. It's also added to your personal dictionary so you can create cards for spaced repetition training.
I tried out the english->hungarian and it's pretty hilarious, because hungarian version sound so german that even I can't understand what it says without the text. :D
I'd completely disable the audio -- for Spanish, at least, it's horrible. I like the concept though and think being able to switch to the language is nice.
Nice idea. But instead of paragraphs it would be better if there was an option to select lines instead. A beginner has only the processing power of few words and paragraphs sometimes can be very long.
89 comments
[ 6.6 ms ] story [ 160 ms ] threadYes this. It would make a huge difference for me at my level. I'm finding that I lose my focus on specific words. I can gather the general meaning from the sentence and/or paragraph but...
Translating the entire paragraph is too much of a crutch and forces me back into thinking in English. If I can get past the single word(s) with a click, I can keep my thoughts entirely in the second language. I suppose an extension could probably fix this...
I've given it a quick try, and it seems to work fine with this site
For example I chose French as the language to get better at and selected "Tour du monde en 80 jours" as the text. The very first paragraph of the French text is
> En l'année 1872, la maison portant le numéro 7 de Saville-row, Burlington Gardens -- maison dans laquelle Sheridan mourut en 1814 --, était habitée par Phileas Fogg, esq., l'un des membres les plus singuliers et les plus remarqués du Reform-Club de Londres, bien qu'il semblât prendre à tâche de ne rien faire qui pût attirer l'attention.
and the English translation is
> Mr. Phileas Fogg lived, in 1872, at No. 7, Saville Row, Burlington Gardens, the house in which Sheridan died in 1814. He was one of the most noticeable members of the Reform Club, though he seemed always to avoid attracting attention; an enigmatical personage, about whom little was known, except that he was a polished man of the world.
Notice how there's an extra sentence at the end about him having an enigmatic personage not present in the French version. Indeed that's in the next paragraph. And the matching goes basically out of whack. The second paragraph of the English is
> People said that he resembled Byron--at least that his head was Byronic; but he was a bearded, tranquil Byron, who might live on a thousand years without growing old.
But the third paragraph of the French is
> On disait qu'il ressemblait à Byron -- par la tête, car il était irréprochable quant aux pieds --, mais un Byron à moustaches et à favoris, un Byron impassible, qui aurait vécu mille ans sans vieillir. Anglais, à coup sûr, Phileas Fogg n'était peut-être pas Londonner.
Statistical machine translation uses curated parallel texts for training, but they tend to match with multiple corpora so the translation is some sort of average I believe. I wonder if matching with just one translation might produce less reliable results?
There's no real "paragraph matching". They just took old texts translated in 2 languages and assumed they match paragraph for paragraph.
http://paralleltext.io/read/?b=0c8b4c11-bd2a-4ded-8d3e-96058...
Really interesting idea but a bit disappointing first experience.
I want to copy words to keep my own word list and to translate the word.
On the other hand, Cervantes and Shakespeare were contemporaries, so I think it enhances reading Don Quixote for the translation to use early modern English - e.g. "Idle reader: thou mayest believe me without any oath".
The use case here might be exceptional, in contrast to contextual ad overlays.
Also, the "I speak" list needs to be checkboxes, not radio (and remember the choices).
The book is the basis of the film Stalker, which is a must-see
Great job!
Maybe it's individual metadata for Sherlock Holmes - when I tried Spanish to English worked on a different book it worked ok.
On Vivaldi (browser's locale is Spanish): Anything is read with a Spanish voice, unless I choose I want to read English, then an English voice is used.
Unintelligible German spoken by an English voice. My German-speaking friend thought it was trying to speak Chinese!
https://pingtype.github.io
Sorry that the user interface is so bad, please just try clicking some items in the heading row to see what's there.
But being native Spanish speaker... No Spanish as source language but many texts with Spanish as language to learn?
Shouldn't be the same texts but viceversa?
So (based on a sample size of one), I'm afraid it doesn't look like the translations are necessarily reliable.
After collecting a messy tangle of special cases over several days, I threw in the towel and just used BeautifulSoup's UnicodeDammit to brute-force a working encoding.
Maybe the Swedish book didn't indicate a vs å in the first place, though.
(E.g. see the Swedish alphabet at http://omniglot.com/writing/swedish.htm)
> I threw in the towel and just used BeautifulSoup's UnicodeDammit to brute-force a working encoding.
For a limited definition of "working", at times! "A working encoding" that shows the wrong letters isn't "working" in a very useful sense.
-- Harris sade, att han drabbades av sa utomordentligt starka yrselanfall emellanat, att han knappt visste vad han gjorde; och da sade George att han led av starka yrselanfall, och knappt visste vad han gjorde.
This made very little sense to me, until I googled it and found another source where the second sentence italicized some words in the second half, making it at least make some sense. (https://sv.wikisource.org/wiki/Tre_m%C3%A4n_i_en_b%C3%A5t._K...)
[0] http://opus.nlpl.eu/
Sample: http://opus.nlpl.eu/EUbookshop/de-en_sample.html
Kindly please fuck off and die, thx.
Maybe the audio can be initiated with a quick highlight on the text or sentence?