Efficient LLM Scheduling by Learning to Rank (hao-ai-lab.github.io)

2 points by zhisbug 1y ago ↗ HN

1 comment

[ 3.7 ms ] story [ 15.0 ms ] thread

We study how to approximate the famous shortest-job-first scheduling in LLM inference!