[dead]
I haven't tried a 3080, but I rented a 16GB 4060 Ti to see how these MoEs behave. gpt-oss 20B actually fits, and I got around 100 tok/s. Qwen3.6 35B doesn't fit, and with part of it in system RAM I was still around 40.
[flagged]
[dead]
I haven't tried a 3080, but I rented a 16GB 4060 Ti to see how these MoEs behave. gpt-oss 20B actually fits, and I got around 100 tok/s. Qwen3.6 35B doesn't fit, and with part of it in system RAM I was still around 40.
[flagged]