Note that this is not just running a lib like HNSWlib or FAISS on top of database persistence. It's deeply integrated with database internals, which is also why in runs nicely with low-memory devices. The entire binary is like 3 MB, so leaves lots of resources to actual vector data and LLMs.
1 comment
[ 3.2 ms ] story [ 9.1 ms ] threadNote that this is not just running a lib like HNSWlib or FAISS on top of database persistence. It's deeply integrated with database internals, which is also why in runs nicely with low-memory devices. The entire binary is like 3 MB, so leaves lots of resources to actual vector data and LLMs.
AMA...