It is absolutely possible and even not so hard. If you use Redis Vector Sets you will easily see 20k - 50k (depending on hardware) queries per second, with tens of millions of entries, but the results don't get much worse if you scale more. Of course all that serving data from memory like Vector Sets do. Note: not talking about RedisSearch vector store, but the new "vector set" data type I introduced a few months ago. The HNSW implementation of vector sets (AGPL) is quite self contained and easy to read if you want to check how to achieve similar results.
Author here. Thanks for the interest! On the performance side: we've applied optimizations like prefetching, SIMD, and a novel batch distance computation (similar to a GEMV operation) that alone gives ~20% speedup. We're working on a detailed blog post after the Lunar New Year that dives into all the techniques—stay tuned!
And we always welcome independent verification—if you have any questions or want to discuss the results, feel free to reach out via GitHub Issues or our Discord.
You're right that we didn't include sqlite-vec in our initial benchmarks—apples-to-apples comparisons are always better. I've actually added basic zvec tests to my fork of sqlite-vec (https://github.com/luoxiaojian/sqlite-vec), so feel free to give it a try. We'll also be publishing a more complete performance comparison in an upcoming blog post—stay tuned!
I haven't been following the vector db space closely for a couple years now, but I find it strange that they didn't compare their performance to the newest generation serverless vector dbs: Pinecone Serverless, turbopuffer, Chroma (distributed, not the original single-node implementation). I understand that those are (mostly) hosted products so there's not a true apples-to-apples comparison with the same hardware, but surely the most interesting numbers are cost vs performance.
I recently discovered https://www.cozodb.org/ which also vector search built-in. I just started some experiments with it but so far I'm quite impressed. It's not in active development atm but it seems already well rounded for what it is so depending on the use-case it does not matter or may even be an advantage. Also with today's coding agent it shouldn't be too hard to scratch your own itch if needed.
Author here. Thanks everyone for the interest and thoughtful questions! I've noticed many of you are curious about how we achieved the performance numbers and how we compare to other solutions. We're currently working on a detailed blog post that walks through our optimization journey—expect it after the Lunar New Year. We'll also be adding more benchmark comparisons to the repo and blog soon. Stay tuned!
Why no benchmarks against pg_vector, DuckDB with extension?
For benchmarks you may just prepare Phoronix Test Suite module (https://www.phoronix-test-suite.com/) to facilitate replication on a variety of machines.
Hi, since this is in process, I wonder what is the memory requirement to run this for e.g. serving 1TB vector data (I have this in pgvector for now) data for example?
Just put Zvec vs LanceDB vs Qdrant through the paces on a 3 collection (text only) 10k per collection dataset.
Average latency across ~500 queries per collection per database:
Qdrant: 21.1ms
LanceDB: 5.9ms
Zvec: 0.8ms
Both Qdrant and LanceDB are running with Inverse Document Frequency enabled so that is a slight performance hit, Zvec running with HNSW.
Overlap of answers between the 3 is virtually identical with same default ranking.
So yes, Zvec is incredible, but the gotcha is that the reason zvec is fast is because it is primarily constrained by local disk performance and the data must be local disk, meaning you may have a central repository storing the data, but every instance running zvec needs to have a local (high perf) disk attached. I mounted blobfuse2 object storage to test and zvec numbers went to over 100ms, so disk is almost all that matters.
My take? Right now the way zvec behaves, it will be amazing for on-device vector lookups, not as helpful for cloud vectors.
Author here. Thanks for putting Zvec through its paces and sharing such detailed results—really appreciate the hands-on testing!
Just a bit of context on the storage behavior: Zvec currently uses memory-mapped files (mmap) by default, so once the relevant data is warmed up in the page cache, performance should be nearly identical regardless of whether the underlying storage is local disk or object storage—it's essentially in-memory at that point. The 100ms latency you observed with blobfuse2 likely reflects cold reads (data not yet cached), which can be slower than local disk in practice. Our published benchmarks are all conducted with sufficient RAM and full warmup, so the storage layer's latency isn't a factor in those numbers.
If you're interested in query performance on object storage, we're working on a buffer pool–based I/O mode that will leverage io_uring and object storage SDKs to improve cold-read performance. The trade-off is that in fully warmed‑up, memory‑rich scenarios, this new mode may be slightly slower than mmap, but it should offer more predictable latency when working with remote storage. Stay tuned—this is still under development!
23 comments
[ 2.3 ms ] story [ 58.5 ms ] threadSee e.g., https://scikit-learn.org/stable/auto_examples/neighbors/plot...
I'd love to see those results independently verified, and I'd also love a good explanation of how they're getting such great performance.
A better comparison would be with Meta's FAISS
And we always welcome independent verification—if you have any questions or want to discuss the results, feel free to reach out via GitHub Issues or our Discord.
https://zvec.org/en/docs/benchmarks/
It would be great to see how it compares to Faiss / HNSWLib etc. I'd will consider integrating it into txtai as an ANN backend.
For benchmarks you may just prepare Phoronix Test Suite module (https://www.phoronix-test-suite.com/) to facilitate replication on a variety of machines.
Average latency across ~500 queries per collection per database:
Qdrant: 21.1ms LanceDB: 5.9ms Zvec: 0.8ms
Both Qdrant and LanceDB are running with Inverse Document Frequency enabled so that is a slight performance hit, Zvec running with HNSW.
Overlap of answers between the 3 is virtually identical with same default ranking.
So yes, Zvec is incredible, but the gotcha is that the reason zvec is fast is because it is primarily constrained by local disk performance and the data must be local disk, meaning you may have a central repository storing the data, but every instance running zvec needs to have a local (high perf) disk attached. I mounted blobfuse2 object storage to test and zvec numbers went to over 100ms, so disk is almost all that matters.
My take? Right now the way zvec behaves, it will be amazing for on-device vector lookups, not as helpful for cloud vectors.
Just a bit of context on the storage behavior: Zvec currently uses memory-mapped files (mmap) by default, so once the relevant data is warmed up in the page cache, performance should be nearly identical regardless of whether the underlying storage is local disk or object storage—it's essentially in-memory at that point. The 100ms latency you observed with blobfuse2 likely reflects cold reads (data not yet cached), which can be slower than local disk in practice. Our published benchmarks are all conducted with sufficient RAM and full warmup, so the storage layer's latency isn't a factor in those numbers.