HN link is not the paper’s title. Also doesn’t seem to be peer reviewed and isn’t even uploaded to a pre-print server like arXiv—literally hosted on GitHub.
I think their thought process is unconvincing, although I think they're probably correct.
A much better paper is, "Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models". They took it much further and trained an LLM such that for every inference, they could see exactly which document from the training dataset it referenced to answer a particular question. ngl This paper AGI-pilled me.
4 comments
[ 3.2 ms ] story [ 18.2 ms ] threadA much better paper is, "Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models". They took it much further and trained an LLM such that for every inference, they could see exactly which document from the training dataset it referenced to answer a particular question. ngl This paper AGI-pilled me.
https://arxiv.org/abs/2411.12580
https://twitter.com/flowersslop/status/1873115669568311727