Sarathi-Serve: A low-latency and high-throughput serving engine for LLMs (github.com) 2 points by tonyabracadabra 2y ago ↗ HN
0 comments
[ 1.9 ms ] story [ 11.1 ms ] threadNo comments yet.