vLLM v0.6.0: 2.7x Throughput Improvement and 5x Latency Reduction (blog.vllm.ai) 3 points by xmo 2y ago ↗ HN
1 comment
[ 2.8 ms ] story [ 19.5 ms ] thread