I presume this works (will work) also for JupyterLite that is based on Pyodide ? Would be great if it helped getting the latest OpenCV-python version [0] and it's dnn goodies being available on a zero-install client…
OpenCV being in the list of Pyodide modules [0] was the biggest boon for my online teaching experience because remotely dealing with install woes (corporate proxies & cie) was a show stopper for regular Python. I'm…
Nice ! My most pressing request for VSS would be efficient binary vectors : is this on the table ?
Only those who don't care/know about prompt processing speed are buying Macs for LLM inference.
For LLM inference, I don't think the PCIe bandwidth matters much and a GPU could improve greatly the prompt processing speed.
Indeed, recent Flash Attention is a pain point for non CUDA.
The idea is presumably that you would "sell" at an artificially low price.
> The Soviets had the […]first woman,[…] That is quite the claim !
You forgot the "/s", or do you actually believe that it's capitalism's fault is a mother taking care of her children is "unpaid labor" ?
"is hard" ≠ "sucks"
Most interesting ! Would you mind sharing the prompt and the resulting CLAUDE.md file ? Thx !
IMO, it would be more interesting to have a 3-way comparison of price/performance between DeepSeek 671b running on : 1. M3 Ultra 512 2. AMD Epyc (which Gen ? AVX512 and DDR5 might make a difference in both performance…
DeepSeek is not a model.Which model did you use (v3 ? R1 ? a distillation ?) at which quantization ?
Nice ! Is it possible to connect to an in browser DB like WASM DuckDB https://duckdb.org/docs/api/wasm/overview.html or https://github.com/babycommando/entity-db ? That would be most useful imho !
It seems that this aims to refutes claims for inaction with facts about spending money. However, the high speed rail project or homelessness management seem to show that in California, $$$ spent doesn't always imply…
How do we know that this extension can be trusted ?
«Unfortunately, I have only seen 3 models, 3B or over, handle RAG.» I would love to know which are these 3 models, especially if they can perform grounded RAG. If you have models (and their grounded RAG prompt formats)…
Just played a bit with it. Were you working with ASCII ? This example didn't work for you ? https://github.com/jfalcou/eve/blob/a141ba93048bb2916c2157a9...
Counterpoint : your message is not synthetic data and will contribute to lots of LLMs saying the same. Many such cases ? (It seems to me obvious that a fgrep would sanitize synthetic data obtained from competitors.)
DeepSeek v3 can run on CPU & RAM : https://www.reddit.com/r/LocalLLaMA/comments/1hqidbs/deepsee... Epyc Gen4 and 12 memory channels of DDR5 @4800 should give you 7 to 9 t/s.
I don't think that Python would be the right language for such low-level performance maxxing endeavor. I would have picked C++ but t was eye opening for me to see how rust enabled such low level optimization, so I'm…
Amazingly thorough ! I love how the author leaves no stone unturned. I had no idea you could do the kind of low level efficiency shaving in Rust. I wonder how a C++ implementation with https://github.com/jfalcou/eve…
Most interesting ! Amazing job at optimizing various parts of the task. It seems that being an MoE with 'only' 37B active params per token would put it within the reach of CPU & RAM inference for the lucky hobbyist with…
For training, doesn't checkpoint saving make high reliability a moot point ? Why pay for 99.99999? uptime when you can restart your training from last/best model ?
About words embeddings, the №1 example is the famous King - Man + Women = Queen This works nicely in the vector space but fails to make a visual impression when projected on 2 dimensions. Neither with ACP, nor MDS ot…
I presume this works (will work) also for JupyterLite that is based on Pyodide ? Would be great if it helped getting the latest OpenCV-python version [0] and it's dnn goodies being available on a zero-install client…
OpenCV being in the list of Pyodide modules [0] was the biggest boon for my online teaching experience because remotely dealing with install woes (corporate proxies & cie) was a show stopper for regular Python. I'm…
Nice ! My most pressing request for VSS would be efficient binary vectors : is this on the table ?
Only those who don't care/know about prompt processing speed are buying Macs for LLM inference.
For LLM inference, I don't think the PCIe bandwidth matters much and a GPU could improve greatly the prompt processing speed.
Indeed, recent Flash Attention is a pain point for non CUDA.
The idea is presumably that you would "sell" at an artificially low price.
> The Soviets had the […]first woman,[…] That is quite the claim !
You forgot the "/s", or do you actually believe that it's capitalism's fault is a mother taking care of her children is "unpaid labor" ?
"is hard" ≠ "sucks"
Most interesting ! Would you mind sharing the prompt and the resulting CLAUDE.md file ? Thx !
IMO, it would be more interesting to have a 3-way comparison of price/performance between DeepSeek 671b running on : 1. M3 Ultra 512 2. AMD Epyc (which Gen ? AVX512 and DDR5 might make a difference in both performance…
DeepSeek is not a model.Which model did you use (v3 ? R1 ? a distillation ?) at which quantization ?
Nice ! Is it possible to connect to an in browser DB like WASM DuckDB https://duckdb.org/docs/api/wasm/overview.html or https://github.com/babycommando/entity-db ? That would be most useful imho !
It seems that this aims to refutes claims for inaction with facts about spending money. However, the high speed rail project or homelessness management seem to show that in California, $$$ spent doesn't always imply…
How do we know that this extension can be trusted ?
«Unfortunately, I have only seen 3 models, 3B or over, handle RAG.» I would love to know which are these 3 models, especially if they can perform grounded RAG. If you have models (and their grounded RAG prompt formats)…
Just played a bit with it. Were you working with ASCII ? This example didn't work for you ? https://github.com/jfalcou/eve/blob/a141ba93048bb2916c2157a9...
Counterpoint : your message is not synthetic data and will contribute to lots of LLMs saying the same. Many such cases ? (It seems to me obvious that a fgrep would sanitize synthetic data obtained from competitors.)
DeepSeek v3 can run on CPU & RAM : https://www.reddit.com/r/LocalLLaMA/comments/1hqidbs/deepsee... Epyc Gen4 and 12 memory channels of DDR5 @4800 should give you 7 to 9 t/s.
I don't think that Python would be the right language for such low-level performance maxxing endeavor. I would have picked C++ but t was eye opening for me to see how rust enabled such low level optimization, so I'm…
Amazingly thorough ! I love how the author leaves no stone unturned. I had no idea you could do the kind of low level efficiency shaving in Rust. I wonder how a C++ implementation with https://github.com/jfalcou/eve…
Most interesting ! Amazing job at optimizing various parts of the task. It seems that being an MoE with 'only' 37B active params per token would put it within the reach of CPU & RAM inference for the lucky hobbyist with…
For training, doesn't checkpoint saving make high reliability a moot point ? Why pay for 99.99999? uptime when you can restart your training from last/best model ?
About words embeddings, the №1 example is the famous King - Man + Women = Queen This works nicely in the vector space but fails to make a visual impression when projected on 2 dimensions. Neither with ACP, nor MDS ot…