anas-awadalla
No user record in our sample, but anas-awadalla has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
No user record in our sample, but anas-awadalla has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
Hello! Totally agree that tokens will be model dependent. We chose to calculate tokens using the GPT-2 tokenizer as that is a common metric used by other datasets like fineweb. So this should roughly give you a sense of…
Hello! Creator of MINT here. We do a lot of pre-processing of commoncrawl (which in its raw form isn’t all that useful for training models). This includes heuristics to remove low quality text and images and…