CEO of Neon here. After we built an in memory HNSW index for Postgres that allowed us to establish a baseline in performance and prove that it's the right approach to support vector search we now built it "the right way" and now it support restarts of Postgres, replication and the rest of the Postgres machinery.
This means that the index is persisted in Postgres pages which are durable against restarting Postgres. Our previous version what fully in-memory and wasn't "durable".
Does the the initial construction (creation) of the index need to fit into RAM?
Also, should one expect the reading via Postgres's buffer cache to have better access patterns / need less memory / have other benefit vs having a non-Postgres HNSW index that's mmapped from disk?
Different vector indexing algorithms under the hood, with different tradeoffs.
In this case "hnsw" is the name of the algorithm. If you want to know more, just search "hnsw" and "ivfflat" to understand the differences.
tsvector is something else; it's not a numeric vector index, it's a different kind of data structure that stores the actual lexemes (keywords) with positions and weights to facilitate full text search (rather than using a text embedding model.)
pg_embedding is based on a different approximate nearest neighbors algorithm - HNSW, which is generally considered - and studied - to be faster as well as more accurate than pgvectors IVF (ignoring a lot of nuance).
However pg_embedding serves the index out of disk, whereas most vector databases opt to serve the index out of memory, or delegate to mmap'ed files. HNSW is a graph-based algorithm, thus access patterns are random and this does not lend itself to disk based access. I'd expect pg_embedding to be slower than memory-resident indices due to this fact. Also in general with a postgres index, my concern would be scalability and resource isolation. It's convenient to colocate these things but ANN indices have very different CPU/Memory/Disk usage patterns than what you may need for just your relational data.
For example
Chroma -> Serves HNSW out of memory, persists to disk with a WAL.
Weviate -> Serves HNSW out of memory, writes HNSW graph search to WAL and uses that for durability.
Milvus -> Serves HNSW out of memory, supports partial mmap, also supports another algorithm called DiskANN which is optimized for SSD. It uses cloud storage with a shared-everything architecture for durability (a lot of nuance here.)
QDrant -> Has a WAL, supports memory and mmap'ed indices.
As I mentioned elsewhere, there's more to vector database selection than raw performance: Developers leveraging their existing experience with Postgres, existing infrastructure investment (often in managed Postgres on AWS/GCP etc), and a single API into the vector store shared with other parts of their app (their ORM / DB layer).
Many teams can also get away with good performance vs the fastest performance, given smaller index sizes and the other tradeoffs I mentioned.
That said, I can imagine the pgvector folks precaching the new HNSW index support they're working on, as they do with their IVFFLAT index.
Sure, I don't disagree that there is more to vector database selection than raw performance. Any technical decision is filled with many considerations.
The commenter - thewataccount - asked about performance and I shared my intuition.
Maybe I am misunderstanding but the postgres buffer cache is LRU and will evict pages right? At times data may go to disk and then during serving it will have to be loaded into memory? So it is quite dependent on the size of your buffer cache as well as contention for that buffer cache.
Also the cache access patterns will vary between these implementations and is worth considering.
This is comparing apples and oranges. You've pointed to some technologies from vector database vendors. Neon isn't a vector database vendor. They're a Postgres vendor. The objective of pg_embedding, pg_vector, and others is to offer teams who deploy Postgres the opportunity to use their existing infrastructure for vector search. The added and important benefit here is hybrid search: using existing DB data to filter semantic search results.
What the work done by Neon, the pgvector team, Supabase and others points to is that "speed" isn't the only factor in vector database selection. Developer experience and existing infrastructure investment are too.
No one is discrediting (or even discussing really) the validity of the point that if you use postgres pgvector or pgembedding may makes sense for the reasons you mention. The question was about how the algorithms compare, I compared the algorithms. Comparing ANN algorithms is relatively apples-to-apples, but is still complicated and filled with nuance.
DiskANN construction is much much slower than HNSW, and the support around updates has been discussed in literature but has not been open sourced yet (They call this FreshDiskANN in the paper - DiskANN is from microsoft research btw).
HNSW-IF is an excellent extension to HNSW (that the vespa team has made easy to implement) that takes advantage of the speed/recall of HNSW in combination with the disk scalability of inverted indices - it is a hybrid approach.
I've done just a little reading on ANN indexes and data structures, but considering you're at Chroma, could you explain why closeness (inverted distance) would be used in HNSW-IF instead of distance?
DiskANN has some great ideas that will likely make it into pg_embedding. The most important implication is how many vectors it can support. DiskANN claims it can manage 1Bln vectors and it's a huge deal. Index construction does take a long time, but we are researching how we can scale it out.
I'm interested in how many vectors are indexed/how large the index is that corresponds to the latency chart? If we have an in-memory HNSW index of 10M vectors at ~20GB (512 dim), say, what are the RAM requirements when using the disk-based version?
28 comments
[ 3.9 ms ] story [ 72.8 ms ] threadDoes the the initial construction (creation) of the index need to fit into RAM?
Also, should one expect the reading via Postgres's buffer cache to have better access patterns / need less memory / have other benefit vs having a non-Postgres HNSW index that's mmapped from disk?
What's the difference between pg_embedding, pg_vector, and tsvector? Are they compariable/interchangable? And how do you know which one to pick?
My understanding is pg_vector has poorer performance compared to some dedicated vector databases, does pg_embedding perform better?
Sorry if these are silly questions.
In this case "hnsw" is the name of the algorithm. If you want to know more, just search "hnsw" and "ivfflat" to understand the differences.
tsvector is something else; it's not a numeric vector index, it's a different kind of data structure that stores the actual lexemes (keywords) with positions and weights to facilitate full text search (rather than using a text embedding model.)
However pg_embedding serves the index out of disk, whereas most vector databases opt to serve the index out of memory, or delegate to mmap'ed files. HNSW is a graph-based algorithm, thus access patterns are random and this does not lend itself to disk based access. I'd expect pg_embedding to be slower than memory-resident indices due to this fact. Also in general with a postgres index, my concern would be scalability and resource isolation. It's convenient to colocate these things but ANN indices have very different CPU/Memory/Disk usage patterns than what you may need for just your relational data.
For example
Chroma -> Serves HNSW out of memory, persists to disk with a WAL.
Weviate -> Serves HNSW out of memory, writes HNSW graph search to WAL and uses that for durability.
Milvus -> Serves HNSW out of memory, supports partial mmap, also supports another algorithm called DiskANN which is optimized for SSD. It uses cloud storage with a shared-everything architecture for durability (a lot of nuance here.)
QDrant -> Has a WAL, supports memory and mmap'ed indices.
Many teams can also get away with good performance vs the fastest performance, given smaller index sizes and the other tradeoffs I mentioned.
That said, I can imagine the pgvector folks precaching the new HNSW index support they're working on, as they do with their IVFFLAT index.
* edited for the grammar gremlins
The commenter - thewataccount - asked about performance and I shared my intuition.
Also the cache access patterns will vary between these implementations and is worth considering.
What the work done by Neon, the pgvector team, Supabase and others points to is that "speed" isn't the only factor in vector database selection. Developer experience and existing infrastructure investment are too.
HNSW-IF is an excellent extension to HNSW (that the vespa team has made easy to implement) that takes advantage of the speed/recall of HNSW in combination with the disk scalability of inverted indices - it is a hybrid approach.
https://unum-cloud.github.io/usearch/
Which was also recently on HN: https://news.ycombinator.com/item?id=36942993
I'm interested in how many vectors are indexed/how large the index is that corresponds to the latency chart? If we have an in-memory HNSW index of 10M vectors at ~20GB (512 dim), say, what are the RAM requirements when using the disk-based version?