Sarathi-Serve: A low-latency and high-throughput serving engine for LLMs (github.com) 2 points by tonyabracadabra 2y ago ↗ HN
0 comments
[ 2.9 ms ] story [ 6.3 ms ] threadNo comments yet.