Sarathi-Serve: A low-latency and high-throughput serving engine for LLMs (github.com) 2 points by tonyabracadabra 2y ago ↗ HN
0 comments
[ 3.2 ms ] story [ 6.8 ms ] threadNo comments yet.