Show HN: EmbedFlow –> Upgrade embedding models without re-embedding your corpus (github.com)

4 points by coolArnav ↗ HN
I was playing around with embedding models and I realized the moving between embedding models on a large corpus can cause a huge backfill, as all of the previous documents would have to be re embedded.

embedflow tries to forgo that; it takes candidate documents from the previous index, and reranks them in realtime with the new model. It supports faiss, qdrant, and pgvector

people who have migrated large production vector indexes, would this be helpful for you?

2 comments

[ 2.9 ms ] story [ 19.7 ms ] thread
The github is live, and I would love for you guys to try it out
this is interesting, but at what scale, or model pair have you validated for