Run Llama 3 405B on home cluster (Distributed Llama) (github.com)

2 points by b4rtazz 2y ago ↗ HN

1 comment

[ 1.7 ms ] story [ 20.2 ms ] thread

Technically, should be enought 2 machines with 128 GB RAM or 4 machines with 64 GB RAM (Q40 weights).