Write up here: https://meshllm.cloud/docs/pages/exo-comparison/
Yes! I’ve worked on the settings interface between our runtime and llamacpp, these are documented and available via our config.toml file
This is actually how I develop and use the mesh at home. Rather than splitting models, I aggregate disparate compute behind one endpoint, without having separate inference providers on each host and a gateway like…
There’s a breakdown on our docs page you can check out! https://meshllm.cloud/docs/pages/exo-comparison/
Yeah, this is one area we’re struggling with due to the sheer volume of variations and conditions, but I’ve been thinking of collecting some real-time statistics around latency, prefill/decode, and model distribution……
We didnt post it, we use a library (iroh) who featured us - so we are here answering any Q’s instead :)
It depends on the size / structure of the model… but generally you can check out the MeshLLM huggingface org for pre-computed model splits. We have a job that will take popular models, and dice them up and post on HF.…
Great question! We’ve had a lot of discussions about the direction we want to take this, and how to best generate some kind of incentive / fairness reward. And we’ve found ourselves hesitating on a direction because, at…
[dead]
Write up here: https://meshllm.cloud/docs/pages/exo-comparison/
Yes! I’ve worked on the settings interface between our runtime and llamacpp, these are documented and available via our config.toml file
This is actually how I develop and use the mesh at home. Rather than splitting models, I aggregate disparate compute behind one endpoint, without having separate inference providers on each host and a gateway like…
There’s a breakdown on our docs page you can check out! https://meshllm.cloud/docs/pages/exo-comparison/
Yeah, this is one area we’re struggling with due to the sheer volume of variations and conditions, but I’ve been thinking of collecting some real-time statistics around latency, prefill/decode, and model distribution……
We didnt post it, we use a library (iroh) who featured us - so we are here answering any Q’s instead :)
It depends on the size / structure of the model… but generally you can check out the MeshLLM huggingface org for pre-computed model splits. We have a job that will take popular models, and dice them up and post on HF.…
Great question! We’ve had a lot of discussions about the direction we want to take this, and how to best generate some kind of incentive / fairness reward. And we’ve found ourselves hesitating on a direction because, at…
[dead]
[dead]