Twitter: @misterkardas.
Blog: https://blog.kardas.org.
Looking at your list you may like VisiData (https://www.visidata.org/). See the demo from 2018: https://www.youtube.com/watch?v=N1CBDTgGtOU
Have a look at https://beta.openai.com/tokenizer which uses javascript reimplementation of the GPT-2 / GPT-3 BPE tokenizer. In this case it's [31373, 995].
I wrote a blog post [1] with an interactive widget where you can provide an encoding for a random decimal digit and see how close you can get to the theoretical log₂(10) ≈ 3.32 bits. [1]:…
Glad you like Papers with Code. Please check [1] for the list of scientific domains we currently support and [2] for CS in particular. [1]: https://portal.paperswithcode.com/ [2]: https://cs.paperswithcode.com/
It's not about the number of letters in the compounds, but about the number of morphemes. Your "powychodziłybyście" example could be translated as "you (feminine, plural) would have been going out". With the word…
Looking at your list you may like VisiData (https://www.visidata.org/). See the demo from 2018: https://www.youtube.com/watch?v=N1CBDTgGtOU
Have a look at https://beta.openai.com/tokenizer which uses javascript reimplementation of the GPT-2 / GPT-3 BPE tokenizer. In this case it's [31373, 995].
I wrote a blog post [1] with an interactive widget where you can provide an encoding for a random decimal digit and see how close you can get to the theoretical log₂(10) ≈ 3.32 bits. [1]:…
Glad you like Papers with Code. Please check [1] for the list of scientific domains we currently support and [2] for CS in particular. [1]: https://portal.paperswithcode.com/ [2]: https://cs.paperswithcode.com/
It's not about the number of letters in the compounds, but about the number of morphemes. Your "powychodziłybyście" example could be translated as "you (feminine, plural) would have been going out". With the word…