Hey all, I spent some time digging into GRPO over the weekend and kicked off a bunch of fine-tuning experiments. When I saw there was already an easy to use implementation of GRPO in the trl library, I was off to the races. I broke out my little Nvidia GeForce RTX 3080 powered laptop with 16GB of VRAM and quickly started training. Overall I was pretty impressed with it's ability to shape smol models with the reward functions you provide. But my biggest takeaway was how much freaking VRAM you need with different configurations. So I spun up an H100 in the cloud and made table to help save future fine-tuners the pains of OOM errors. Hope you enjoy!
2 comments
[ 3.3 ms ] story [ 10.9 ms ] thread