Tracing tokens through Llama 3.1 8B inference on H100s (krithik.xyz) 3 points by krithik_7 4mo ago ↗ HN
[–] gglavas 4mo ago ↗ yeah, architecture and serving efficiency matter just as much as model quality once you leave the demo phase
1 comment
[ 5.2 ms ] story [ 16.1 ms ] thread