Unsloth creators fix universal error with gradient accumulation (unsloth.ai)

4 points by ZQ-Dev8 2y ago ↗ HN

2 comments

[ 2.3 ms ] story [ 18.5 ms ] thread

seems like something pytorch maintainers would want to know about and fix asap...

Article title: Bugs in LLM Training - Gradient Accumulation Fix