Show HN: I built Haystack – your own google for scattered workplace knowledge (haystack.it)
A few weeks ago I was scrolling through confluence pages trying to find ssh connection details to our integration machine for 40 minutes straight, later I discovered my co-worker slack'ed me the ssh connection string two months ago.
So the same weekend I started working on haystack - a search engine for workplace apps. that enables you to search slack, confluence, jira, teams, sharepoint, github, and email in one place.
I wanted it to support natural language queries so a query like: "how to connect to integ2 machine?" yields:
ssh -i private.pem ubuntu@ec2-integration2.eu-est-1.compute.amazonaws.com
I rolled it out to my co-workers a week ago and they thought it's a hit, so I'm planning on releasing it publicly on March 2023. But if you want to try it out before then it's available here: https://haystack.it. Thanks!
68 comments
[ 3.2 ms ] story [ 127 ms ] thread- https://haystacksearch.org/
- https://haystack.deepset.app/
- https://www.haystackapp.io/
- https://www.haystackteam.com/
- https://thehaystackapp.com/
- https://www.usehaystack.io/
I would argue that every known noun are the first domains to get registered. My goal is to associate workplace search engines with haystack.
[1] https://www.spglobal.com/engineering/en/products/haystack-go...
[2] https://www.plume.com/serviceproviders/platform/haystack/
Let the best product be remembered for the name.
But I think there's value in at least recognizing that the namespace is quite crowded given the collisions that two interweb randoms were able to identify in short order.
Same idea :)
Haystack runs entirely client sided in the browser, so it has a unique tech-stack:
Storage
using IndexDB, haystack stores user indexes locally, + a compressed 90mb NLP model (t5-small) is stored.
Indexing
Locally in the browser, using a t5-small bi-encoder, and some parsing of documents happens in wasm.
Search
Query converted to embedding, then searched over index, atlast results are reranked with a t5-small based cross encoder, and top results go through a seq-to-seq transformer to produce a nice consise textual answer.
The problem that I have is that I can't remember days/months later if something I read is in an open tab or closed a while ago resulting in some frantic searches.
This would enable querying my little private search engine like "the HN story a few weeks ago that talked about ancient greek mining techniques" or "the reddit comment that had an analysis comparing Orwell's 1984 to the bible".
For those not familiar, semantic embeddings take a chunk of text and embed it in a high dimensional vector space (~hundreds of dimensions) where semantically similar texts are closer together.
hundred of millis is what I experienced.
Just a thought on the design: I really feel like the gold gradients make the whole site feel 'cheap', not trustworthy and not very 'professional'. Actually makes me not want to use it. Replacing them with a simple warm yellow improves this a lot. That might be just me, but maybe it's something for you to consider.
Good luck!
Which under the hood is based of a t5 encoder
But yeah
Slack, teams, confluence or notion? airtable? jira?
Not affiliated but just a happy user of their product it searches slack, confluence, jira, gmail, gdrive, github and source code all at once. With extras like Go links, verification, and some knowledge base features.
> Show HN is for something you've made that other people can play with. HN users can try it out, give you feedback, and ask questions in the thread.
> Off topic: Those can't be tried out, so can't be Show HNs. Make a regular submission instead.
I'd suggest you change the title asap.
(1) https://news.ycombinator.com/showhn.html
And most first posts don’t get the intended traction for various reasons (too long, bad wording, unclear).
reposting and changing the post is totally allowed :)
Good stuff!
But also have paid features that managers would like to use.
some rust code that compiles to WASM loads LLM from memory, and uses custom transformer.py like rust alternative we wrote.
Minutes, not days. Very big data sets might take 30+ minutes (or even a couple of hours), but usefulness starts in the first few minutes (because of the priority algorithm)
Very cool - any more info on that?
I am wondering how the search latentcy will be with your approach, especially for cases with more than a few hundreds documents. Do you have any insights about that?
I'm testing this on a large knowledge base.
I'll keep it haha
I almost built a potential solution to this problem years ago but backed out. I'd love to see a solution that sticks, and to be wrong about this, but it feels very much like a Tarpit problem to me:
https://www.youtube.com/watch?v=GMIawSAygO4