I have 2 HP DL380s with 2 Xeon Gold 6252 CPU @ 2.10GHz and 256gb of DDR4 memory that I bought for $700 a piece off ebay. I run 4 gpt-oss:120b models on CPU using 120GB memory for each VM. I get about 10-15 tokens/second…
I have 2 HP DL380s with 2 Xeon Gold 6252 CPU @ 2.10GHz and 256gb of DDR4 memory that I bought for $700 a piece off ebay. I run 4 gpt-oss:120b models on CPU using 120GB memory for each VM. I get about 10-15 tokens/second…