How continuous batching improves LLM inference throughput 23x (twitter.com)

1 points by george_123 3y ago ↗ HN

0 comments

[ 2.5 ms ] story [ 9.6 ms ] thread

No comments yet.

A showcase of shard-db. Data from the Hacker News API. Refreshed every 15 minutes.

Source · Live DB stats