As another poster said, yes : you can find good word vectors for most languages.
Furthermore, given a copy of the text from wikipedia, it only takes 2 days of computation on a laptop with a modern GPU to regenerate them from scratch for any language having a wikipedia (I did that using two R packages a few years ago, I expect that it would be even simpler nowadays).
Yep, like most projects that use embeddings it doesn't seem that smart when you look closely. It does "better than chance" but that's not a high standard!
btw what did you like most about using the magnitude format? lazy loading seems good, depending on if it takes time to load the sqlite DB at each startup. That's one of the major pains of working on anything with word embeddings is that server reloads / hot restarts take forever.
The fastText vectors (see below) also use subword embeddings so I think they had potentially better results. I used FT for some Chinese stuff and I think it worked better for that since chinese characters are so much more important than latin scripts.
Seems magnitude python API also has some other features like POS tagging, I wonder how that compares to say spaCy.
Player 0 it is your turn!
The word you are trying to match in meaning is:
antepenultimate
Your char list is:
['i', 'g', 'j', 'u', 'b']
Please input a valid word made from some or all of the provided characters
But how does "bug" match the meaning of "antepenultimate" in any way? I must be missing the point of the game.
Is it just "words that vaguely, extremely distantly, have some relation, as determined by the neural net"? And your job is to work out which extremely-distant word is slightly closer? (I noticed "gubi" scored very, very slightly more points than "bug" in the demo.
Essentially, all words are embedded in an N-dimentional space. Then you can simply measure the distance between words in the word-cloud. The exact method of how it decided what coordinates to assign to each word in the cloud isn't important to the game.
so you didn't have a word near penultimate and then pick some letters from it? It's really just open-ended what the users should choose? That does make it a lot harder...
It's interesting. There has been lots of ways to explore Word2Vec, but maybe this helps people get an intuitive understanding of the strengths and weaknesses better. (hint, the farther the words are from the target, the more meaningless the distance comparison becomes).
21 comments
[ 2.2 ms ] story [ 26.9 ms ] threadSame with any other language models and NLP related stuff. It's hard to find anything valuable.
Actually facebook research are pretty good at gathering things, their ParlAI project also gathers together other sources for conversational stuff.
Furthermore, given a copy of the text from wikipedia, it only takes 2 days of computation on a laptop with a modern GPU to regenerate them from scratch for any language having a wikipedia (I did that using two R packages a few years ago, I expect that it would be even simpler nowadays).
I don't think there's been much happening around word vectors specifically, but it's still worth knowing about.
(And it's "pedant", not "pendant").
The fastText vectors (see below) also use subword embeddings so I think they had potentially better results. I used FT for some Chinese stuff and I think it worked better for that since chinese characters are so much more important than latin scripts.
Seems magnitude python API also has some other features like POS tagging, I wonder how that compares to say spaCy.
I did something similar when I was still at school. Scripts which emulated Codenames [1], and providing hints for given set of words [2].
I believe that is possible extension to your project - emulate Codenames games:)
[1] https://boardgamegeek.com/boardgame/178900/codenames
[2] https://github.com/johnnyheineken/codenames-AI
I’ve also played around with word vector games — Robot Mind Meld [1] has you and a robot working together to converge on the same word.
[1] http://robotmindmeld.com
Is it just "words that vaguely, extremely distantly, have some relation, as determined by the neural net"? And your job is to work out which extremely-distant word is slightly closer? (I noticed "gubi" scored very, very slightly more points than "bug" in the demo.
Essentially, all words are embedded in an N-dimentional space. Then you can simply measure the distance between words in the word-cloud. The exact method of how it decided what coordinates to assign to each word in the cloud isn't important to the game.
It calculates the distance in vector space from your word to the given word.
I think it helps if you understand what Word2Vec is, as then it becomes clear what is going on.