Stay tuned, this is just the beginning. Merging with Cohere will help in scaling, too.
Hey, I worked on pre-training data Kolibri. We spent considerable time and effort to go beyond just translating. For example by building a pipeline that processes Common Crawl dumps specifically for German. You might be…
By various forms of deduplication (exact, fuzzy, substring), heuristic filters and distilling quality classifiers that annotate our data. Synthetic rephrases can also be considered a form of cleaning/getting more out of…
I worked on Kolibri, in particular pre-training data and mid-training. We strive to be as open as possible. Glad you like it.
Hey there, I worked on Kolibri pre-training data. Agree with the point that non-US/non-Chinese labs need to pool effort which I'm also personally excited about. The merger is public, nothing to hide.
Stay tuned, this is just the beginning. Merging with Cohere will help in scaling, too.
Hey, I worked on pre-training data Kolibri. We spent considerable time and effort to go beyond just translating. For example by building a pipeline that processes Common Crawl dumps specifically for German. You might be…
By various forms of deduplication (exact, fuzzy, substring), heuristic filters and distilling quality classifiers that annotate our data. Synthetic rephrases can also be considered a form of cleaning/getting more out of…
I worked on Kolibri, in particular pre-training data and mid-training. We strive to be as open as possible. Glad you like it.
Hey there, I worked on Kolibri pre-training data. Agree with the point that non-US/non-Chinese labs need to pool effort which I'm also personally excited about. The merger is public, nothing to hide.