[flagged]
[dead]
Did a side project on Workers and the self host test for me was whether I could run it off Cloudflare at all. Durable Objects and KV bindings meant no, so it was really just deploy it yourself.
Nice, though generation speed is the easy half for MoE offload, what's your prompt processing look like at say 16k context?
[flagged]
[flagged]
[flagged]
[dead]
Did a side project on Workers and the self host test for me was whether I could run it off Cloudflare at all. Durable Objects and KV bindings meant no, so it was really just deploy it yourself.
[flagged]
Nice, though generation speed is the easy half for MoE offload, what's your prompt processing look like at say 16k context?