Show HN: NanoRL – RL training for LLMs in ~1,800 lines (github.com)
The smallest async RL trainer I could write: one loop that runs REINFORCE on CartPole on a laptop and async GRPO on a cluster
(e.g. 8xH100 trainer, 8 vLLM workers, ran as a [SkyPilot job group](https://docs.skypilot.ai/en/latest/examples/job-groups.html) on k8s ).
All without Ray or TRL or DeepSpeed etc., workers talk to the trainer over stdlib HTTP.
1 comment
[ 1.9 ms ] story [ 9.6 ms ] thread