Making a vintage LLM from scratch; Take #2 (crlf.link)

I built a pile of new datasets, 3 new vintage models, a benchmark, an evaluation pipeline, experiments and a very good vintage tokenizer.

Demo play with the vintage-LLM: https://huggingface.co/spaces/croqaz/Piston-and-Prose