Show HN: Continue – Open-source coding autopilot (github.com)
Our GitHub is https://github.com/continuedev/continue. You can watch a demo of Continue and download the extension at https://continue.dev
— — —
A growing number of developers are replacing Google + Stack Overflow with Large Language Models (LLMs) as their primary approach to get help, similar to how developers previously replaced reference manuals with Google + Stack Overflow.
However, existing LLM developer tools are cumbersome black boxes. Developers are stuck copy/pasting from ChatGPT and guessing what context Copilot uses to make a suggestion. As we use these products, we expose how we build software and give implicit feedback that is used to improve their LLMs, yet we don’t benefit from this data nor get to keep it.
The solution is to give developers what they need: transparency, hackability, and control. Every one of us should be able to reason about what’s going on, tinker, and have control over our own development data. This is why we created Continue.
— — —
At its most basic, Continue removes the need for copy/pasting from ChatGPT—instead, you collect context by highlighting and then ask questions in the sidebar or have an edit streamed directly to your editor.
But Continue also provides powerful tools for managing context. For example, type ‘@issue’ to quickly reference a GitHub issue as you are prompting the LLM, ‘@README.md’ to reference such a file, or ‘@google’ to include the results of a Google search.
And there’s a ton of room for further customization. Today, you can write your own
- slash commands (e.g. ‘/commit’ to write a summary and commit message for staged changes, ‘/docs’ to grab the contents of a file and update documentation pages that depend on it, ‘/ticket’ to generate a full-featured ticket with relevant files and high-level instructions from a short description)
- context sources (e.g. GitHub issues, Jira, local files, StackOverflow, documentation pages)
- templated system message (e.g. “Always give maximally concise answers. Adhere to the following style guide whenever writing code: {{ /Users/nate/repo/styleguide.md }}”)
- tools (e.g. add a file, run unit tests, build and watch for errors)
- policies (e.g. define a goal-oriented agent that works in a write code, run code, read errors, fix code, repeat loop)
Continue works with any LLM, including local models using ggml or open-source models hosted on your own cloud infrastructure, allowing you to remain 100% private. While OpenAI and Anthropic perform best today, we are excited to support the progress of open-source as it catches up (https://continue.dev/docs/customization#change-the-default-l...).
When you use Continue, you automatically collect data on how you build software. By default, this development data is saved to `.continue/dev_data` on your local machine. When combined with the code that you ultimately commit, it can be used to improve the LLM that you or your team use (if you allow).
You can read more about how development data is generated as a byproduct of LLM-aided development and why we believe that you should start collecting it now: https://medium.com/@continuedev/its-time-to-collect-data-on-...
Continue has an Apache 2.0 license. We plan to make money by offering organizations a paid ...
103 comments
[ 962 ms ] story [ 2603 ms ] threadAre there any others apart from gpt4 suitable for programming copilot tasks ?
> If files get too large, it can be difficult for Continue to fit them into the limited LLM context windows. Try to highlight the section of code that include the relevant context. It's rare that you need the entire file.
Most of the value and real world use case benefits come from usage in a brownfield development where a legacy code isn’t well understood and is large (exceed current LLM context ?)
> telemetry through posthog Can organisations setup their own telemetry and development data collection to further analyse how and where the Copilot is being used ?
> Finops How does one get visibility of token / api usage and track api spends ?
> We've found claude-2 very capable, especially for chat functionality, and especially in situations where you're looking for the equivalent of a faster Google search, even smaller models will do. For inline edits, gpt4 well outperforms others, but we've only optimized the prompt for gpt4. There's a LOT of tinkering to be done here, and it seems clear that OSS models will be capable soon.
> Definitely value there. We have an embeddings search plugin heading out the door soon, but we very consciously avoided this for a while - it obstructs understanding of what code enters the context window, and we think transparency is underrated.
> Yes! You could have your own PostHog telemetry by simply switching out the key, but we also deposit higher quality development data on your machine (we never see it). Benefits being both 1) understanding ROI of the tool, and 2) being able to train custom models.
> This is a reasonable request! We'll add a feature for this. Right now, you can use the usage dashboard of whichever provider's key you use.
It's clear the team is shipping a ton too since almost every day I see VSCode popup about restarting my editor for the new version of Continue.
Excited to see where things go with this!
> You are using an out-of-date version of the Continue extension. Please update to the latest version.
[0] https://github.blog/2023-07-20-github-copilot-chat-beta-now-...
And yes, by default Continue sees your open file, but you can also highlight multiple code snippets or type '@' to include context from outside your codebase, like GitHub issues.
Sure, it's M$, sure it's not open source (IIRC), but all you get with these alternatives is a wrapper around the ChatGPT API, or in some cases a lesser model. No one's solved the whole directory context problem yet, no one seems to be doing much the Copilot suite can't.
Copilot is currently reasonably priced, and pretty much guaranteed support and development going forward. There's pull requests and cli in the pipeline [0].
I'd need directory context inference on a quality open model to be convinced to use anything else, and we're just not there yet.
[0] https://github.com/features/preview/copilot-x
[0] https://docs.github.com/en/copilot/github-copilot-chat/using...
Neither were doing anything, we simply forgot to `npm uninstall` after (a while ago) playing around to decide which service to use. Thanks for pointing it out.
That said, I'm not sure what's the difference between providing my own openapi key or not. The "Customization" doc is not entirely clear on what using my own key enables me to do.
For instance, is this required for gpt4? What are the limits of the free trial key?
I don't want to evaluate this not knowing which model is really using, and it's not clear what difference the key makes.
Edit: Also when I asked the Continue chat how to change my key and the model being used, it said that this is not possible since the key is yours, instead of pointing me to the "Extension Settings" inside the extension tab using the cog wheel.
We'll update the /help messaging so it knows this, and you can read more about choosing a model here: https://continue.dev/docs/customization
In addition to the selected lines, does Continue support getting related definitions from the language server and inject them in the prompt? That would be huge.
As of now, if there are collections of definitions that you frequently reference, you could save them in the system message, or write custom slash commands that let you prefix your prompt with these definitions.
It's not like you have to be coding all the time.
Things will of course change as tools evolve.
It's almost like .. coding it yourself!
And that's why it is a "pain point". These all can be done automatically.
Actually writing code is a minority of my time spent developing software, I don't need to trade that for need to spend time to know what and more importantly why something is where it is.
I guess it does work if you just want it to generate a webpage and you never want to then add any functionality. Or a glorified boilerplate generator for stuff not on the prod path.
I had two options: 1. Learn everything first and then start prototyping. 2. Start prototyping and learn along the way. I chose the latter. But instead of searching stackoverflow and putting the pieces together, I used GPT and learned from the generated outputs in the context of my own problem.
It may not save a lot of time in larger projects because as you said, one should ultimately learn the "what and why", but it definitely provides a more pleasing experience. And I guess the time-saving part becomes more relevant with better tooling, like the suggestion I had.
But then, I don't think this really go beyond entry level prototyping, any complexity and chatGPT doesn't have enough tokens.
https://marketplace.visualstudio.com/items?itemName=TomJenni...
A few observations:
- As soon as I sent it the first prompt, it tried to connect to windows.net. Why is that? Is this call safe to block?
- When opening a new window, macOS asks me to give access to VS Code for a lot of folders: Desktop, Downloads, Documents... The only new thing is the Continue plugin. Why would it ask that?
- It looks like it needs to redownload "Python packages" every time I open a new window. I wonder if this could be optimised for a quicker startup.
- It tries to connect to meilisearch.com . What information is being sent over? What is this used for?
Continue will cache Python packages in ~/.continue/server/env after the first download, but there might be something else causing slower startup. Will look into this as well!
Meilisearch is an open-source search library. We connect to meilisearch.com only in order to download the software, which then runs completely locally to power search in the dropdown as you type. The line of code where we do this is here: https://github.com/continuedev/continue/blob/ce76e391775034c...
You are sending full laptop specs of your users to meilisearch. Which in turn sends them to Twilio apparently.
If that connection is denied, the following error occurs:
> I would be surprised if VS Code did not already have access to Desktop / Documents / etc.Why would I give VS Code access to these directories? All my code resides elsewhere, it has no reason to access those, and it hasn't in the last couple years of this particular installation. Not only it hadn't accessed them, but it hadn't even asked me for access until I installed your extension.
Which of the local models have you seen the best results with?
UPDATE: Found a note here https://github.com/continuedev/continue/blob/ce76e391775034c... that says "While these models don't yet perform as well, they are free, entirely private, and run offline." and points to the documentation here for how to run one, but doesn't really recommend a specific model: https://github.com/continuedev/ggml-server-example
And good point! I've just added this as the recommendation in the README.
Assuming that may be I missed something, I deleted ~/.continue and trying to start from scratch (vs code message: Starting Continue Server ...). I do see FastAPI is up on http://localhost:65432/ still vscode dialog shows same message. Where I can see the logs what its trying to do? What am I missing?
Anyone else able to use local ggml?
The additional thing you need to do is set "default_model" to "ggml" in ~/.continue/config.py.
That’s been a major focus of my open source AI coding assistant project [0]. It seems super promising to exploit the semantic structure of code to help LLMs understand complex codebases [1].
Also, there’s a small discord [2] where a few of us are sharing learnings about building these coding assistants. Please join if you’d like to compare notes.
[0] https://github.com/paul-gauthier/aider
[1] https://aider.chat/docs/ctags.html
[2] https://discord.gg/FTYDTRKZ
Looks like cool work you're doing, would love to learn more—thanks for the invite!
I have tried almost every co-pilot solution, including GitHub Co-Pilot with Chat and Labs, Cody, and a few random extensions.
And for some reason, I still default to using ChatGPT, even with the massive drawback of copying and pasting.
I haven’t seen any breakthroughs with these developer experiences. Every still feels suboptimal.
Actually, it looks like this project is fairly similar in some ways. A little more full featured.
[1] https://marketplace.visualstudio.com/items?itemName=Rubberdu...
My one suggestion: I hate to say it but the name RubberDuck and the icon you've used kind of undersell the seriousness of the project. I think the whole "hacker" branding thing is kind of lame but it could be worth some tweaks to make it seem more "cool" :/. Also your website seems to be down!
Thanks for the feedback, good points!
On another note: we purposefully refer to ourselves as an "autopilot" rather than "co-pilot". While at first glance pedantic, we think there is real value behind the idea that developers should be in control. We want to get out of their way, rather than injecting AI everywhere. An autopilot should feel like less of a separate entity (a pair-programmer) and more of an extension of yourself, a part of your own brain that you can call on to more easily complete particular tasks.
I like this idea but I gotta say when I think "autopilot" I think "does it for you" and imagine ceding control rather than retaining it. I definitely assumed after reading the name that it was going to be a tool to minimize developer involvement as much as possible, similar to how AutoGPT is supposed to require less supervision.
But I do really like that functionality of attaching context to the query! Love that. I replied to the founder of Sourcegraph in this thread and I think that should answer your question as well.
I'm excited about your approach and even more about the fact that you made it open source! Thanks for that. If this sticks for me I can see myself contributing to it.
Let's say I want to build a complex React component that does a lot of stuff under the hood. Perhaps it needs to handle multiple inputs, it needs to show certain children UI conditionally based on the state and it also needs to push the state to a global redux store.
Perhaps the component also depends on some utils and other root components that pass props to it.
None of these solutions seem to acknowledge that what would improve my productivity is to have a proper way to model the problem for the AI the same way I'm understanding it. Allowing me to build the blast radius of the problem, instead of expecting the AI to infer it.
In the case of the React Component what I want is:
1) bootstrap the component
2) modify it to address requirements that emerge as I explore the needs.
3) look at dependencies such as schema files or root components and suggest modifications that align with the desired functionality or allow me to point out at required changes in dependencies.
When I say that these solutions are suboptimal, what I mean is that there's no straight-forward way for me to engage in these code generation tasks in a way that doesn't feel fragmented.
What I literally want is go to a file, tell the AI to consume all the context of that file and its dependencies and then modify it to add features, fix bugs, improve code or extend functionality. I then want to have the ability to accept those changes as a reviewer or suggest changes. And I want to do without forking away from my current context.
Nobody has gotten this experience right because what most players in this space are doing is building these restrictive form factors or atomized features like text brushes, global chats or in-line prompt to code generation.
I appreciate having these, but ultimately what I want is to have an AI system that accepts a context (a file or set of files) and a prompt and then generates the requested code modifications.
None of these extension allow me to do this effectively and if they do then it seems that ability is being diminished by a poor user-facing abstraction.
In my opinion, this is a design problem.
The problem is that at the moment it's slightly buggy and the UI is not so polished. I would recommend to submit your email to the newsletter on the website. Once I feel it's good enough, I'll send an email to everyone. Don't worry that I'll spam your inbox, I haven't sent any email yet to people who subscribed. I'll just send it when I feel the tool is good enough.
With the LSP you have access to to whole source code and can do edits in the whole workspace.
A couple of thoughts.
1. It would be nice if we could ingest external resources like web pages.
2. Deep integration with the debugger (Limitation mentioned in your docs). Code pilot seems to be very superficial here.
I know the issues above are limited by the token context. However, there's not a lot of innovation in these two aspects for code generation.
The second sounds quite interesting. I'm curious whether you envision as far as the language model stepping through a program on its own? We had considered working on this at one point in time.
Honestly I could see it both ways depending on the needs of the developer. The language model could step through autonomously or with the developer.
Thinking of how this could look like abstractly through a UI. Picture sequential timeline which consists each step the developer or the model has made through the code. A timeline that the developer could fast forward or rewind which has some implications for the LLM.
There are three main areas the developer could give LLM inputs.
1. Before debugging
The user could give input (Trace back, a question, ingested context (URL) before initializing debugging, instruction steps, autonomously or guided.
Autonomously given the context of before debugging, it could even optionally set its own breakpoints.
2. At any step during debugging
LLM might influence how the developers steps through the code.
3. Post debugging
Now imagine each step through the debugger being logged to which the user could ask questions within the context of the history of the debugger. LLM could have different scoped contexts relative to the timeline of the steps of interest.
Imagine the developer steps through 15 points in the code. Developer could ask about step 7 or a range of steps post debugging. LLM could be limited scope to only the previous steps including 7 or a range. Alternatively it could have knowledge of the entire step history. The underline assumption there is limiting context in some cases improves output of LLM.
Definitely not intending this type of behavior. We want to do everything to keep Continue completely private.
We chose Meilisearch because it works entirely on your machine (now that analytics are turned off ;) )
As for the issue of being cut out from the OpenAI API, we would survive such a situation because we work with any model, including open source.
Here are the docs:
https://github.com/paul-gauthier/aider#in-chat-commands
Here is a chat transcript that shows gpt-4 fixing code after being shown failing test output using the /run command:
https://aider.chat/examples/add-test.html
With multiple ai providers, customizable prompts, select to ask, comment to ask, history saving, SO search, command line exec, etc.