2 comments

[ 2.9 ms ] story [ 15.8 ms ] thread
Love this kind of experiment. Would the model perform better with word tokens?
A friend of mine forked the repo and tried it with BPE (Byte-Pair Encodings), and it did noticeably improve performance.