HN Explorer

Ask HN: Case Study on Training LLMs on Apple M2 Ultra Neural Engine?

1 points by pritambarhate ↗ HN
Has someone here come across a case study on training LLMs on Apple M2 Ultra Neural Engine? I wanted to know how it would compare to training LLMs on GPUs like H100.

Considering the cost and shortage of H100, can Mac Pro Ultras be used for training LLMs? I mean people are trying to do it on SuperComputers using CPUs (https://news.ycombinator.com/item?id=40348371) surely someone must have tried using Apple Silicon Neural Engine.

I tried searching for it, but didn't find anything proper.

1 comment

[ 3.4 ms ] story [ 15.7 ms ] thread
talldayo
The Neural Engine itself on those is pretty much non-comparable to something like an H100. The GPU is much more suitable for training, or even the CPU probably.