Tracing tokens through Llama 3.1 8B inference on H100s (krithik.xyz) 3 points by krithik_7 2mo ago ↗ HN
[–] gglavas 2mo ago ↗ yeah, architecture and serving efficiency matter just as much as model quality once you leave the demo phase
1 comment
[ 0.21 ms ] story [ 11.9 ms ] thread