Unsloth creators fix universal error with gradient accumulation (unsloth.ai)

4 points by ZQ-Dev8 1y ago ↗ HN

2 comments

[ 3.1 ms ] story [ 14.5 ms ] thread

seems like something pytorch maintainers would want to know about and fix asap...

Article title: Bugs in LLM Training - Gradient Accumulation Fix