slimesli
No user record in our sample, but slimesli has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
No user record in our sample, but slimesli has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
for llama 3-8b (smaller model) you need 8GB/16GB of memory: minimum/recommended || for llama 3-70b (larger model) you need 64GB/96GB of memory what kind of memory? there are 3 types of memory: GPU memory: medium…
It's not a fundamental limit. Google palm had 540B parameters as dense model. But it's a practical limit because models with over 1T would be extremely slow even on newest gpus. Even now, OpenAI has limit of 25…
Because of memory bandwidth. H100 has 3350gB/s of bandwidth, more gpus will give you more memory but not bandwidth. If you load 175b parameters in 8bit then you can get theoretically 3350/175=19 tokens/second. In MoE…