Show HN: I made a better Perplexity for developers (devv.ai)
I am Jiayuan, and I'm here to introduce a tool we've been building over the past few months: Devv (https://devv.ai). In simple terms, it is an AI-powered search engine specifically designed for developers.
Now, you might ask, with so many AI search engines already available—Perplexity, You.com, Phind, and several open-source projects—why do we need another one?
We all know that Generative Search Engines are built on RAG (Retrieval-Augmented Generation)[1] combined with Large Language Models (LLMs). Most of the products mentioned above use indexes from general search engines (like Google/Bing APIs), but we've taken a different approach.
We've created a vertical search index focused on the development domain, which includes:
- Documents: These are essentially the single source of truth for programming languages or libraries; I believe many of you are users of Dash (https://kapeli.com/dash) or devdocs (https://devdocs.io/).
- Code: While not natural language, code contains rich contextual information. If you have a question related to the Django framework, nothing is more convincing than code snippets from Django's repository.
- Web Search: We still use data from search engines because these results contain additional contextual information.
Our reasons for doing this include:
- The quality of the index is crucial to the RAG system; its effectiveness determines the output quality of the entire system.
- We focus more on the Index (RAG) rather than LLMs because LLMs evolve rapidly; even models performing well today may be superseded by better ones in a few months, and fine-tuning an LLM now has relatively low costs.
- All players are currently exploring what kind of LLM product works best; we hope to contribute some different insights ourselves (and plan to open source parts of our underlying infrastructure in return for contributions back into open source communities).
Some brief product features:
- Three modes: - Fast mode: Offers quick answers within seconds. - Agent mode: For complex queries where Devv Agent infers your question before selecting appropriate solutions. - GitHub mode(currently in beta): Links directly with your own GitHub repositories allowing inquiries about specific codebases.
- Clean & intuitive UI/UX design.
- Currently only available as web version but Chrome extension & VSCode plugin planned soon!
Technical details regarding how we build our Index:
- Documents section involves crawling most documentation sources using scripts inspired by devdocs project’s crawler logic then slicing them up according function/symbol dimensions before embedding into vector databases;
- Codes require special treatment beyond just embeddings alone hence why custom parsers were developed per language type extracting logical structures within repos such as architectural layouts calling relationships between functions definitions etc., semantically processed via LMM;
- Web searches combine both selfmade indices targeting developer niches alongside traditional API based methods. We crawled relevant sites including blogs forums tech news outlets etc..
For the Agent Mode, we have actually developed a multi-agent framework. It first categorizes the user's query and then selects different agents based on these categories to address the issues. These various agents employ different models and solution steps.
Future Plans:
- Build a more comprehensive index that includes internal context (The Devv for Teams version will support indexing team repositories, documents, issue trackers for Q&A)
- Fully localized: All of the above technologies can be executed locally, ensuring privacy and security through complete localization.
Devv is still in its very early stages and can be used without loggi...
75 comments
[ 5.4 ms ] story [ 179 ms ] thread"For complex queries where Devv Agent infers your question before selecting appropriate solutions."
Could you expand on this a bit? What does "infers your question" mean?
It's not all that clear to me from the site or your post when Fast Mode vs. Agent Mode should be used. Is Fast Mode for answering conversational questions and Agent Mode for answers that involve writing code?
Looks like there's an opportunity to improve the fast mode by caching the results for simple searches.
I've outlined some initial ideas in this post and may develop a more detailed article later on. Stay tuned!
Ie if the quality RAG index is your primary offering, then as a user i imagine my experience will depend on how well you have indexed things i care about. Maybe my language of choice (Rust) has decent indexes, but some random Crate i try to use might not.
I'd love to be able to queue up index ingests of standard API sources like docs.rs/crates.io and be notified when that ingest completes.
Will give it a try today, congrats on the launch!
Thank you for your valuable feedback; it's an excellent suggestion! In fact, we've already begun implementing this feature with our initial step being the introduction of GitHub Mode. This new functionality will enable seamless integration with your personal GitHub repositories. We've developed a bespoke indexer tailored to various programming languages to enhance this experience.
Furthermore, we can expand this capability to include documentation and other resources as well. The architecture is designed to be extensible, so all that's needed is the creation of additional indexers to support these materials.
I just tested it by typing "llama cpp gpu support" that's it.
Flawless instructions for Python, but when I followed up with
"in node"
It didn't know about node-llama-cpp. Is there a general knowledge cutoff, and/or is loading developer-specific stuff a manual process?
The results: https://devv.ai/search?threadId=dl3vwbdu52ww
P.S. Agent Mode is a superior option to Fast Mode. It meticulously examines your questions and assigns an appropriate agent to provide answers, leveraging GPT-4 technology in its operations.
So agent mode is better for more recent stuff that you might find in a search engine?
https://devv.ai/search?threadId=dl3rtxmcsruo
EDIT: The syntax came from a language proposal in a github issue from 8 years ago, so I guess it's not fully hallucinated. But still not the best choice of what source to use.
I utilized Agent Mode to rephrase the query, and here are the results: https://devv.ai/search?threadId=dl3rtxmcsruo
P.S. Agent Mode is a superior option to Fast Mode. It meticulously examines your questions and assigns an appropriate agent to provide answers, leveraging GPT-4 technology in its operations.
---
> The syntax came from a language proposal in a github issue from 8 years ago, so I guess it's not fully hallucinated. But still not the best choice of what source to use.
Yes, if the source contains errors, the response may generate inaccurate information. We are continually refining our re-ranking algorithm within the Retrieval-Augmented Generation (RAG) system to select the most reliable sources.
Outside of popular languages it seems like they always hallucinate.
I am also pleasantly surprised it is not suffering a "hug of death" following the presentation here. I am curious about the need in resources for your engine? What kind of hardware is it running on?
backend: go/rust/python + gin + mysql + pinecone + es + redis + aws
llm: openai/azure + aws gpu + aws bedrock
Granted I started with a hard one, but I asked it how to create a GTK3 interface with PHP, and it gave me instructions to download and use an abandoned project for GTK2, but described it as GTK3 in the steps.
I tried asking it some other questions about languages and applications specific to version numbers - it seems to provide incredibly ambiguous and version agnostic responses, or tells me essentially "you may or may not be able to do this, and you should check if you can" when the answer is clearly that it is not possible. Or it just ignores the version entirely and provides instructions that don't match up - hallucinating UI elements or commands that don't (or didn't yet) exist.
For something targeted at developers, this is a gaping hole and is what I would consider a major oversight - the responses I'm getting are very similar in content to what I get from GPT and Ollama's generic models.
I also just queried something with "perplexing" results in fact, but I tried the "generic" "knowledge" instead of the "specific" about coding: in the reply the engine included good pointers, but clearly without knowing why they were especially relevant - relevance which instead appeared in the linked references.
It is an LLM+RAG based search engine: the value is only partly in the summary, which could even be misleading - as expected from lack of actual intelligence -, the value is in the linked resources.
In other words, it "understands" your query better that a search engine of the past - and that is valuable. But for the actual solution you are querying for, the "summary" part could be good or could be defective: it is probably best to consult the linked material... Material that you could have not found immediately otherwise - it could have been tricky with past technology to express your need in a way that makes you obtain good search results.
Or perhaps a more brief summary for each result explaining the relation?
More generally, do text AI's not support weighting terms like the image AI's do? Over in Stable Diffusion that sounds like something where I'd add a weight like "How do I create a <GTK3:1.2> interface in <PHP:1.1>?"
P.S. Agent Mode is a superior option to Fast Mode. It meticulously examines your questions and assigns an appropriate agent to provide answers, leveraging GPT-4 technology in its operations.
Feedback: I tried to click one of the links under "source" but it kept jumping down as the LLM-generated content was added.
Does this mean you intend to let people self-host?
Yes, this is on our roadmap. We will launch "Devv for Teams" in the upcoming quarter. This new feature will enable seamless integration of internal team knowledge, including codebases, wikis, issue trackers, and logs.
I really distrust putting my API keys into brand new and unknown websites, just seems like credentials harvesting to me.
You might want to check out this project.
That sounds interesting. Could you provide further details? By the way, integrating an API is part of our future plans. We plan to enable Devv integration with Slack, Linear, and websites in the future.
Also, if you want to discuss more, feel free to email me at jiayuan@devv.ai
It would be really cool if, when starting a coding task with Plandex, relevant docs/context from a web search could be automatically included in context via this kind of API. Currently urls can be loaded into context with `plandex load [url]` but you have to figure out which urls would be helpful to load yourself.
1 - https://github.com/plandex-ai/plandex
I dont see a link to it on the github/website. DO i have to self host right now ?
Good luck with this
Gonna add some free models with search in future
Try searching for "Weather in [your city]" and compare it to Google or any weather app. It's consistently wrong.
I had an issue with shopify and was able to work through the fix using perplexity which I wasn't able to get with chatgpt on its own.
I love that you can change the models, I mostly use Claude opus though.
I do wish the image generator was better but they frame perplexity as a search engine rather than chat so I have firefly if I really need an image.
The 3 engines you mention (Perplexity, You.com and Phind) all do that. So do Google, Bing and DuckDuckGo. It makes it easier to link to results and build custom links.
Also, I could add you to Gnod Search then:
https://www.gnod.com/search/ai
https://www.gnod.com/search/ai?q=Python%3A%20How%20do%20I%20...
In case anyone knows, I'd be curious: does that mean no terms apply to my usage if I can't view them by reasonable means? Just whatever local law defaults apply? Earlier today I noticed the terms of the local zoo 404'd (while buying tickets online) and I wondered the same
You can view the terms here: https://indexlabs.notion.site/Term-of-Service-6ca77cbc49504c...
I'm running the code it gave me to try it out on a small list, it's been 10 minutes and it's still running. Might be something worth looking into.
Granted, the way I asked for this function was not the most natural.
[1] https://devv.ai/search?threadId=dl4c8if11c00
You're just hitting a limit of the LLMs, they won't give you bug-free code, specially not from the first time, specially not complex ones like galloping timsort.