Show HN: Danswer – Open-source question answering across all your docs (github.com)
It is an open source, self-hosted search tool that allows you to ask questions and get answers across common workspace apps AND your personal documents (via file upload / web scraping)! Full demo here: https://www.youtube.com/watch?v=geNzY1nbCnU&t=2s.
The code (https://github.com/danswer-ai/danswer) is open source and permissively licensed (MIT). If you want to try it out, you can set it up locally with just a couple of commands (more details in our docs - https://docs.danswer.dev/introduction). We hope that someone out there finds this useful
We’d love to hear from you in our Slack (https://join.slack.com/t/danswer/shared_invite/zt-1u3h3ke3b-...) or Discord (https://discord.gg/TDJ59cGV2X). Let us know what other features would be useful for you!
68 comments
[ 4.2 ms ] story [ 145 ms ] threadLooks nice! Curious about the business model or is it just a hobby project?
I came across Danswer a few days ago as an option for this, so I spent a day building a connector [1]. I was pleasantly surprised how accurate the output was for something like this. I have a few pages detailing my servers and I could ask things like "Where is x server hosted"? and get a correct response accompanied with a link to the right source page.
Some things to be aware of specifically about Danswer: It only works with OpenAI right now, although the team said that open model support is important as a future focus. Additionally it felt fairly heavy to run and required a 30 minute docker build process but I think they've improved on this now with pre-built images, and I'm not familiar with the usual requirements/weight of this kind of tech. Otherwise, things were easy to start up and play around with, even for an AI noob like me. Both their web and text-upload source connectors worked without issue in my testing.
[1]: https://github.com/danswer-ai/danswer/pull/139
[0]: https://github.com/ggerganov/llama.cpp/issues/1602
It comes with its own variant of the GGML sub format "ggcv1" but there's quants available on HF [2]
Although if you have a GPU I'd go with the newly released AWQ quantization instead [3] the performance is better.
(I may or may not have a mild local LLM addiction - and video cards cost more then drugs)
[1] https://github.com/cmp-nct/ggllm.cpp
[2] https://huggingface.co/TheBloke/falcon-7b-instruct-GGML
[3] https://huggingface.co/abhinavkulkarni/tiiuae-falcon-7b-inst...
For now I will stick to PrivateGPT and LocalGPT.
And, to share with you something: I saw somewhere a tool (maybe it was GPT4ALL itself) that had the ability to expose a OpenAI-compatible local API on localhost:8080... Ah, yes. Here it is. Actually, there are two. They are described as possible backends for Bavarder (that's a free access to multiple online models, API key is not required): https://bavarder.codeberg.page/help/local/
[^1]: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-backen...
Ex: If a connected gdrive document gets indexed, but then someone fixes the share settings in google docs for some item to be more restrictive.. How does Danswer avoid leaking that data? Dynamic check before returning any doc that the live federated auth settings safelist the requesting user reading that doc?
The immediate plan is to extend our current poll / push based connectors to also grab access information (+ add IdP integrations for cross-app identity). There will be some delay to grab access updates, which will be combatted by the dynamic check with the app / IdP itself at query time that you mentioned (still investigating exactly how this will work).
We are also considering adding support for group based access defined within Danswer itself for sources that don't provide APIs to get access information (default being all-public if not specified). Of course, for these, we will not be able to sync permissions.
No company at a decent size (those who actually reach some complexity of documentation) will be okay with exfiltrating confidential information to an external service we have no deal or NDA with. Sure, OpenAI is easy to integrate, but it's also an absolute showstopper for a company.
We don't need state-of-the-art LLMs with 800k context, we need confidentiality.
Google Search itself is a somewhat special case - it gets a free pass because of its utility and because you're unlikely to paste anything confidential into a search box. But there are many places where even Google Search is banned on the data security grounds.
OpenAI offerings - ChatGPT, the playground, and the API - all very much encourage pasting large amounts of confidential information into them, which is why any organization with minimum of legal sense is banning or curtailing their use.
The TOS for the (paid) enterprise products such as Google workspace are totally different from the (free) consumer versions. For example Google can't use the data for AI training.
> Any data sent through the API will be retained for abuse and misuse monitoring purposes for a maximum of 30 days, after which it will be deleted (unless otherwise required by law). [0]
The word "review" implies humans reading your data, but this wording only says it's retained for "monitoring". That could mean other things.
Or are you seeing the "review" wording somewhere else?
[0] https://openai.com/policies/api-data-usage-policies
>OpenAI retains API data for 30 days for abuse and misuse monitoring purposes. A limited number of authorized OpenAI employees, as well as specialized third-party contractors that are subject to confidentiality and security obligations, can access this data solely to investigate and verify suspected abuse.
The Terms of Service? They're worth less than it costs to print them out.
Case in point: right now, Microsoft is repackaging OpenAI models on their Azure platform and raking it in - the main value proposition here is literally just that it's "OpenAI, but with proper contract and an SLA". But companies happily pay up, because that's what makes the difference between "reliable and safe to use at work" vs. "violating internal and external data safety standards, and in some cases plain up illegal".
It's been quite clear for some time that, between OAI and MS, they very neatly split their market: OAI handles the early development and direct customers, and MS handles enterprises. It would require OAI to be a much bigger company than it is right now to properly handle enterprises, and MS already has all that infrastructure (legal, support, etc.). Seems like a sensible setup to me, and I don't see the need for enterprises to run open source models themselves (in this context - of course I see the value in all the other respects about lock-in and specialization), especially if they are already on Azure.
If you're interested in something similar, send me an email.
Ticketing platforms like ServiceNow fall under a similar category, although a bit lower priority in my mind.
Additionally, the indexing process is setup as a composable pipeline under the hood. It would be fairly trivial to plug in different chunkers for different sources as needed in the future.
Would you mind saying a few words on how Danswer approaches this?
As far as using other data connector frameworks, we found that we either (1) didn't think they were very good or/and (2) they didn't support automatic syncing effectively. For larger enterprises, it's not feasible to do a complete sync every X minutes. We need to be able to get a time-bounded subset (or have them push updates to us), which is something LangChain, LlamaIndex, etc. don't support natively.
Danswer is:
(1) itself an end-to-end application which allows you to connect to all your workplace tools via a UI, and then ask questions and get answers based on these documents. The goal is to be a permissively licensed, open source solution to the enterprise knowledge retrieval problem.
(2) an ingestion framework specifically targeted for enterprise applications. We provide a UI for admins to manage connections to other common workplace apps, automatically sync them into a vector DB + a keyword search engine, and expose APIs that allow access to these underlying data stores (more to come in this direction). We take care of access control (more in the pipeline here as well), only grabbing updates so we don't have to pull thousands (or millions) of documents every X minutes, etc. TL;DR: we're focused on a specific ingestion use case.
Crawling sites to index the FAQ's and knowledge bases, into the vector search, isn't as intimidating as it sounds, at least on linux systems. Sometimes a thin wrapper function over plain old wget will get you 99% of the way
Thank you for your work, it looks great
https://osschat.io/
> What are the latest features introduced in ClickHouse?
and it answered:
> :sparkles::crystal_ball: Oh dear! A mystical internal server error has occurred. Please weave your magic and try again after a few seconds. :male_mage::star2:
> Danswer provides Docker containers that you can run anywhere, so data never has to leave your PVC. The one exception is using GPT for inference but we are working on allowing for locally hosted generative models as well.
Look, you can plug this hole trivially for many companies, by adding support for Azure OpenAI API. It's almost identical to OpenAI API - the main difference is how you pass keys and specify the model to use. But that alone will make it possible to use Danswer with company data in places that signed a relevant contract with Microsoft.
I would be interested in adding Azure OpenAI support and OneNote support.
As I understand it, you have a pre-trained LLLM model which you tune LOCALLY, and you can use it from the ui provided.
From what I am reading, there is ambiguity about whether data is sent elsewhere.
Obviously it doesn't matter if you host in EKS or EC or locally as far as I understand.