Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses (quesma.com) 1 points by stared 15d ago ↗ HN
[–] mrrrcs 15d ago ↗ One run per cell is more noise than the gap you are measuring [–] stared 15d ago ↗ These was some code budget there.But as from seeing various runs, errors bars are gross overestimation (as not "the same test", but "if we have different tasks from the same sample").
[–] stared 15d ago ↗ These was some code budget there.But as from seeing various runs, errors bars are gross overestimation (as not "the same test", but "if we have different tasks from the same sample").
2 comments
[ 0.30 ms ] story [ 5.9 ms ] threadBut as from seeing various runs, errors bars are gross overestimation (as not "the same test", but "if we have different tasks from the same sample").