Run Llama 3 405B on home cluster (Distributed Llama) (github.com)

2 points by b4rtazz 1y ago ↗ HN

1 comment

[ 3.0 ms ] story [ 10.0 ms ] thread

Technically, should be enought 2 machines with 128 GB RAM or 4 machines with 64 GB RAM (Q40 weights).