Nice presentation of the list! I'd recommend watching a few of his talks/podcasts before during reading these to get the overview and how all the bits in these works tie together.…
https://lwn.net/Articles/1065620/
https://xcancel.com/tdietterich/status/2055000956144935055
The author discussed this here four days ago https://news.ycombinator.com/item?id=48077663
Agreed. LLMs have helped me achieve much deeper reading, _when directed to do so_. Asking an LLM to “Teach me Socratically about this paper/code. One question at a time”, usually allows me to get a much deeper reading…
Each of these models has a thinking/reasoning variant and a default non-thinking variant. I would expect the reasoning variants (o3 or “GPT5 Thinking”, Gemini DeepThink, Claude with Extended Thinking, etc) to do better…
As far as I can tell they don’t say which LLM they used which is kind of a shame as there is a huge range of capabilities even in newly released LLMs (e.g. reasoning vs not).
They fix the temperature at T=0.6 for all k for all models, even though their own Figure 10 shows that RL model benefits from higher temperatures. I would buy the overall claim much more if they swept of temperature…
Nice presentation of the list! I'd recommend watching a few of his talks/podcasts before during reading these to get the overview and how all the bits in these works tie together.…
https://lwn.net/Articles/1065620/
https://xcancel.com/tdietterich/status/2055000956144935055
The author discussed this here four days ago https://news.ycombinator.com/item?id=48077663
Agreed. LLMs have helped me achieve much deeper reading, _when directed to do so_. Asking an LLM to “Teach me Socratically about this paper/code. One question at a time”, usually allows me to get a much deeper reading…
Each of these models has a thinking/reasoning variant and a default non-thinking variant. I would expect the reasoning variants (o3 or “GPT5 Thinking”, Gemini DeepThink, Claude with Extended Thinking, etc) to do better…
As far as I can tell they don’t say which LLM they used which is kind of a shame as there is a huge range of capabilities even in newly released LLMs (e.g. reasoning vs not).
They fix the temperature at T=0.6 for all k for all models, even though their own Figure 10 shows that RL model benefits from higher temperatures. I would buy the overall claim much more if they swept of temperature…