Show HN: OpenCopilot – Build and embed open-source AI copilots into your product (github.com)
OpenCopilot is an OSS framework which helps devs to build open-source AI Copilots that actually work and embed into their product with ease.
Why another LLM framework?
Twitter is full of impressive LLM applications but once you peel off the curtains it’s clear that they are just demos. The reason being because building an AI Copilot that goes beyond a Twitter demo can be complex, time-consuming and unreliable.
Our team has been in the AI space since 2018 and built numerous LLM apps & copilots. While doing that, we got approached by many startups saying they’d also like to build a copilot for their product but they haven’t been able to get it reliable, fast or cost-effective enough for production use. Thus we built OpenCopilot framework, so devs can intuitively get AI Copilots running in less than 10 minutes and iterate towards a useful Copilot in a single day.
We believe every product, company and individual will have their Copilot in the future. Thus, we’d love your feedback, questions and constructive criticism.
37 comments
[ 0.27 ms ] story [ 77.7 ms ] threadhttps://github.com/opencopilotdev/opencopilot/blob/d7aa8270d...
Generally, we leave the LLM up to the user -- if OpenAI or Google is a no-go, then you probably are anyway in the territory of self-hosting or even self-training your LLM, which means you're fine setting up your own inference endpoints as well.
I have a somewhat related project that wraps the gpt API and let's you interact in a codebase, where the LLM can request access to certain files, propose new files and edits
https://github.com/breeko/j-dev
We were hoping to make dynamic RAG part of the very first release but didn't - we'll be adding that soon.
could you please elaborate?
The most recent LLM app we've built has been the Ready Player Me copilot (which you can see linked in the main repo as well, and which was an inspiration for actually making something open-source based on what we had learned into). The problem it solves is onboarding developers onto their platform, and is currently live for a subset of their users.
We've built a ton of copilots that we haven't published. Some have been for work like founder copilots for ourselves and a few friends, sales outreach copilots, etc. Some have been for fun: I wanted to see if I could create a copilot that coaches me to do things faster a la https://patrickcollison.com/fast - so I did. As a PoC I created a copilot that explains the rules of an obscure sport I like (floorball).
In addition, with the same team, since the beginning of this year we've made a bunch of experiments that I'm sure many others have tried: * "chat with your knowledge base" style bots: for tech support, for developer docs, for internal HR use, and probably some I can't remember * an LLM-based quiz app that dynamically makes a quiz based on your Spotify history * an automated engineer (a much more basic version of smol-ai/developer) * a website builder where you mostly uses a chat UI for making the website * making AutoGPT better (higher quality, faster, more straightforward to use) * we also briefly considered making an OSS LLM before Llama 1 came out and kickstarted the OSS LLM wild ride we've seen since then
We killed almost all of these experiments within a few weeks after release, because none found early traction; we'd probably been wrong about the problem these were solving.
About my AI background: I built a 25-person computer vision team from zero at an identity verification unicorn. After that tried and failed to build a computer vision labelling tool startup. Before that, built robot perception software at Starship robots and unsuccessfully tried to make a contribution to AI safety with my thesis on active reinforcement learning in 2017.
About Johannes (OP) & the rest of the team: in 2018 him and part of the current team tried to make Sidekik: something like character.ai, but training models from scratch - which eventually didn't work out. From 2020 (after a pivot) the same team was building Sentinel, a deepfake-detection startup - which again did not work out.
FYI: Sidekik's original website: https://web.archive.org/web/20181008082142/https://sidekik.a...
It was way before all of it was cool and back then we didn't train LLMs but just LMs with LSTMs lol. Got to around 10K users but the technology was too early so we pivoted to Sentinel: https://thesentinel.ai/ Scaled that up to the largest deepfake detection provider in the world but eventually it didn't work out which is a longer story.
It's built with OpenCopilot; the bulk of the work is in the prompt (you could probably copy-paste it into ChatGPT and get not-too-bad resultS). The RAG part is not great right now since the documents are not chunked in a reasonable way. But it has been somewhat valuable for me as well.
Let me know if you tried it out and whether it helps you do anything faster ;)
Does this exist?
Current foundation AI models have safeguards in place to avoid generating content that could be used for disinformation, violence, and terrorism.
While not perfect, these safeguards have meant that we've been able to use ChatGPT over the last couple months, without a major backlash due to the technology being used for destructive purposes and it potentially being closed (as has happened many times before).
However, if you look at the most sensitive technologies, whether it's nuclear weapons or until recently cryptography, they're never open. The risk of them falling into the wrong hands is too high.
AI seems to us like any other software, as people who may understand it and its limitations. However, it's also a tool for unfathomable destruction -- not now, but in the coming years.
And so restrictions are almost certain to happen, and according to many, justified. You're not going to have access to "un-censored AI" in the coming years. That's unfortunately reality.
To put it another way, we're all concerned (at least in 1st would countries) about terrorists and hackers being the greatest threat to people's lives. So we're comfortable giving the government sole control of weapons - even though statistically that's not an accurate representation of the danger graph.
I don't know if it's good marketing or propaganda, but I find it ironic that "only government can be trusted".
Why would you like to train it on your own code?
Rest of the stack is fully open-source and it's much more than a wrapper. Maybe the impression comes from the aspect that we've made getting started dead simple with the minimal Copilot example. The framework covers a lot of ground so you can build your copilot easily: backend, adding a knowledge base & dynamic data, front-end, monitoring & evaluation.
This is pretty much v0.1 release. What else would you love to see?
With making it possible to bring any LLM, we're considering adding support for bringing anything that implements the Langchain LLM interface.
In general, we're still trying to figure out how much customizability to give to the user and how (Langchain is a great option here) -- power should not come at the expense of ease of use.
Anyway, congrats, it will be really useful for building POCs and mvps quickly. I'll definetely give it a try and feedback once I test it!
Would love to hear your feedback on what's good, what sucks and what to improve.