I find it odd that sglang, vLLM, TRTLLM don't seem to want to publish benchmarks comparing each other. They used to, but now there seems to be some unspoken rule against it.
At least we get comparison against "other OSS engine" this time, but that could be HF's Transformers as well :)
> I find it odd that sglang, vLLM, TRTLLM don't seem to want to publish benchmarks comparing each other. They used to, but now there seems to be some unspoken rule against it.
Someone always cries foul with "you're using the wrong version/patch" or similar, and they get hopelessly outdated so fast too, especially since the labs tend to release when everyone else is releasing too, hassle to keep them up to date, and why would you when sometimes the alternatives manages to get ahead of you? :)
Yet another website where I don't know what they do, so I go to the homepage that has a marketing sentence explaining what they do, and I still don't understand.
6 comments
[ 4.8 ms ] story [ 22.1 ms ] threadBechmarks from InferenceX (they do not have apples-to-apples setups to compare the different engines for whatever reason): https://inferencex.semianalysis.com/inference?i_hc=1&g_model...
I find it odd that sglang, vLLM, TRTLLM don't seem to want to publish benchmarks comparing each other. They used to, but now there seems to be some unspoken rule against it.
At least we get comparison against "other OSS engine" this time, but that could be HF's Transformers as well :)
Someone always cries foul with "you're using the wrong version/patch" or similar, and they get hopelessly outdated so fast too, especially since the labs tend to release when everyone else is releasing too, hassle to keep them up to date, and why would you when sometimes the alternatives manages to get ahead of you? :)
Something with LLMs, obviously.
I think that is pretty self-explanatory. Do you understand the word benchmark?