This is such a basic thing nowadays, and ElasticSearch is massive overkill for it. Something like SQLite or LanceDB or basically any vector database is much more appropriate.
This seems to be coming from the “we must make ElasticSearch AI-compatible” department more than anything.
Summary of the article (https://pastebin.com/aawJfrF6) since the original one is like reading an academic paper filtered through an LLM that hates human readers.
It seems like a cool approach. Don't know if it's novel but it's much smarter than "shove markdown files into directories".
so the 11% miss rate - do users actually notice when the agent drops a memory? like if someone already said they tried X and the agent suggests it again.
I'm using Typesense to power my take on a md kb, highly recommend this option which positions itself against Elasticsearch and Algolia. Combines vector with bm25 and all the extras you get from a trad search tool like Algolia.
21 comments
[ 4.6 ms ] story [ 49.5 ms ] threadThis seems to be coming from the “we must make ElasticSearch AI-compatible” department more than anything.
It seems like a cool approach. Don't know if it's novel but it's much smarter than "shove markdown files into directories".
- Hybrid recall + reranker: Two searches merged, then re-scored for best matches
- Supersession: Old facts get hidden, new ones take their place
- Decay: Recent or often‑used memories get a score boost
- DLS: Each user only sees their own documents