[–] 7e 2y ago ↗ And this is why nobody submits MLPerf against NVIDIA. [–] greenknight 2y ago ↗ Its weird, i looked up whether AMD has any benchmarks on the 405B for the MI300x, and came across this one -- https://dstack.ai/blog/amd-mi300x-inference-benchmark/#token...From my understanding, it can get up to around 2500 tokens/s? Both are 8x units (h200 and MI300x)
[–] greenknight 2y ago ↗ Its weird, i looked up whether AMD has any benchmarks on the 405B for the MI300x, and came across this one -- https://dstack.ai/blog/amd-mi300x-inference-benchmark/#token...From my understanding, it can get up to around 2500 tokens/s? Both are 8x units (h200 and MI300x)
[–] EgoIncarnate 2y ago ↗ not "an H200", "In the table above, tensor parallelism is compared to pipeline parallelism with each across eight GPUs" [–] FanaHOVA 2y ago ↗ Title on HN is wrong. The article says GPUs and it's referring to one of their 8xH200 boxes.
[–] FanaHOVA 2y ago ↗ Title on HN is wrong. The article says GPUs and it's referring to one of their 8xH200 boxes.
5 comments
[ 3.4 ms ] story [ 27.5 ms ] threadFrom my understanding, it can get up to around 2500 tokens/s? Both are 8x units (h200 and MI300x)