Launch HN: Glide (YC W24) – AI-assisted technical design docs
Why use design docs? Mainly because on complex software projects, if you don’t have a careful strategy, a few lines of code can lead to a ripple effect across the whole codebase. You're working on a small new feature and suddenly, your PR is touching dozens of different files in a handful of directories, and you may be questioning if it still makes sense at all.
Laying out your approach in a technical design document helps to avoid these situations, so tech companies often embed this practice in their engineering culture. Unfortunately, producing these write-ups is a manual and time-consuming process that keeps you switching between typing in Confluence, searching on GitHub, and drawing on a whiteboard.
For example, one of our early users is building a RAG (Retrieval Augmented Generation) system in Rust, and needs to rewrite their original chunking algorithm that was created in Python. A change like that involves not only porting the chunking logic but also defining new structs, choosing the Rust libraries for text parsing, re-integrating with the Rust server, and changing the endpoint on the front end. Simple-sounding changes like these can end up with many "tentacles" devs have to work through. Our product helps teams like this spec out their approach more quickly.
The need for this hit us when we were working on a product using LLMs to turn tickets into PRs. You can just assign a ticket to GPT-4 and review the PR, right? Turns out that most people (including us) don’t write very complete or specific tickets on the first pass, and when they do, that ends up being more effort than just doing the whole thing yourself.
What was missing was a fast way to iterate on goals, requirements, and implementation plans for code that could then be useful to developers (whether AI assisted or not) to implement with high confidence. Our editor, Glide, splits this process into the following stages and lets you spend as much or as little time on each stage as makes sense.
(1) We index your project code with Tree-Sitter and make it searchable, so you can add relevant parts of the codebase to the task context.
(2) AI writes the first pass for a high-level solution outline given the code and the goal. It will try to point out missing details or ambiguities about the task so far.
(3) As a developer you can easily edit the generated content—tweaks like “change step 3” or “remove the part about caching” don’t require additional back-and-forth conversation.
(4) The next stage is a step-by-step plan—a pseudocode-level guide for how to complete the change. AI generates this, and again you can make adjustments in-line as needed.
(5) There is a chat mode for deeper discussions about the content—just highlight any text and click chat. This works well for exploring alternatives, getting more specific on a particular point, or understanding why the AI wrote what it did.
(6) Once the plan looks good, you can have AI generate code edits for each step. You’re free to make additional changes here or just open the markdown doc with code edits in your local VSCode via an extension, so you have it as a reference while you implement.
This looks a lot like collaborative Chain of Thought, explicitly writing out each step in a line of reasoning, for both the develo...
73 comments
[ 3.3 ms ] story [ 137 ms ] threadThere's an incredible amount of promise in this UX. Excited to see it continue developing alongside the foundation models and new technique approaches.
Is this similar to Devin that just launched today, except that Glide is less autonomous and gives you more control? How do you compare to them? https://twitter.com/cognition_labs/status/176754876313496400...
It looks like Devin chooses a plan and then writes and runs the code or uses the internet to get feedback on how to change the plan/code accordingly.
We focus on having the engineer work with the AI on the plan before writing the code, and the engineer is responsible for the final implementation.
We found that doing everything autonomously was awesome when it worked but frustrating when it didn't; so we wanted to make the UX interactive enough to be useful even if the AI gives some unexpected results. Does that help?
From Kapathy on Twitter:
> In my mind, automating software engineering will look similar to automating driving. E.g. in self-driving the progression of increasing autonomy and higher abstraction looks something like:
1. first the human performs all driving actions manually
2. then the AI helps keep the lane
3. then it slows for the car ahead
4. then it also does lane changes and takes forks
5. then it also stops at signs/lights and takes turns
6. eventually you take a feature complete solution and grind on the quality until you achieve full self-driving.
There is a progression of the AI doing more and the human doing less, but still providing oversight. In Software engineering, the progression is shaping up similar:
1. first the human writes the code manually
2. then GitHub Copilot autocompletes a few lines
3. then ChatGPT writes chunks of code
4. then you move to larger and larger code diffs (e.g. Cursor copilot++ style, nice demo here https://youtube.com/watch?v=Smklr44N8QU)
5.... Devin is an impressive demo of what perhaps follows next: coordinating a number of tools that a developer needs to string together to write code: a Terminal, a Browser, a Code editor, etc., and human oversight that moves to increasingly higher level of abstraction.
It's very different from others like Waymo, who are going for more of an all-or-nothing approach.
Similarly, we hope to be useful early with fairly simple features so that we can get it into developers' hands to learn how to incrementally make the product better.
We're thinking about this a lot, but our goal is to be very careful about adding complexity to the product (we failed at this in our other endeavors!)
Echoing Rob's sentiment on autonomy, we're more focused on tit-for-tat collaboration between the developer and AI as opposed to "set it and forget it" agent execution. I think both modes will have a role in the dev workflow of the future
> P.S. We only keep the indexed code in an in-memory cache while you’re using it. We don’t store your code or use it as training data!
Personally before using this on a work repo I'd like to get some more info on this to make sure nothing from the private repo will leak out.
Also I'm a bit worried about giving Glide the permission "Act on your behalf". Is there no more granular and less powerful permission you could ask for, i.e. read access only? It seems that only "pull request (coming soon...)" which hasn't been implemented needs "write access".
Here are the steps we go through:
- Checkout the code to a temporary directory (which is deleted when it goes out of scope in python, so at the end of the indexing API call)
- Use Tree Sitter to create an index of just your chosen "Directory to Index" and then store that in a Redis cache
- We clear the index from the Redis cache if you haven't used it in 2 hours.
I don't know why exactly it says the message to "Act on your behalf". I just double-checked in our app, and confirmed that these are the only permissions we get:
- Contents - Read-only
- Metadata - Read-only
- Email addresses - Read-only
We agree that we shouldn't take write access if we are not writing PRs. Hopefully that helps!
It does say which sounds really ominous: https://imgur.com/a/w8M0wcE
The link on the page points to https://docs.github.com/en/apps/using-github-apps/authorizin... which basically says the app can do whatever whenever.
I think if the wording was different I would be able to successfully pitch this to my work.
On-prem is on our roadmap, we can definitely make that happen for you. We just went this route for now to make the app easily accessible to everyone
Despite the scary warning, you're only granting the listed permissions.
> Despite the scary warning, you're only granting the listed permissions.
Is there a way for me to verify that that's the case? At least after granting the permissions?
If you have it installed for an organization its slightly different. Go to the organization, then Settings->Github Apps->Configure.
For me at least, it lists the permissions the app has. Additionally, if we ever want to request new permissions in the future, you will have to grant them.
I’ve seen several company’s receive the same complaint on HN…. GitHub should reword it or make it more clear in some way
You have to “act on their behalf “ in order to read private repos (potentially) only they have access to…
I wish this ran locally or was self-hosted.
Question for you: would the app running locally be sufficient, or would the LLM also need to run on-premise?
Stay tuned!
We tried generating code diffs with our initial product. The issue we had was that when that diff started to grow, parsing it to determine whether it did the right thing became increasingly difficult.
We will generate code diffs in the future, but we're happy with generating the instructions for two main reasons:
-The LLMs seem to produce better output when you write out the intermediate steps
-The developer understands the thinking behind the code that was written. We feel that this is an important factor as the team size grows and developers have to interact more with each other's code
- It would be significantly more helpful if it could ingest my database schema.
- Why make me manually pick files instead of indexing them all and doing some RAG magic?
- Having used ChatGPT for similar tasks, I'm skeptical that step 2 you have, where it generates a high-level solution, will work all that well. Right now, LLMs work best for me when I'm giving them much more specific tasks and keeping the high-level plan in my head.
- This isn't really a technical design doc in the sense that I use the term? This might just be me using language incorrectly but I typically call a tech design doc a document that explains the inner workings of a system, normally with something like a UML or similar diagram. This just seems like todo list or subtask generation for writing a feature.
- Have you considered feeding the AST directly into the LLM? Honestly just curious how well it would understand the code if you did that vs. giving it the raw source.
Either way, congrats on the launch!
On RAG, I have a lot of thoughts - it's my pet project to make it work really well. We started out down this road but in practice getting good results out of RAG has been shockingly hard on bigger, more complicated codebases. In many cases when we put this in front of devs that know their code, the RAG part has been a lossy approximation of what they already knew the relevant code to be, so they'd sit there and watch "is this thing going to find the code that I already know I need for this task"
I think the key here will be a robust, language-agnostic symbol graph that maps out high-resolution dependencies, so when you are retrieving `foo`, you can also efficiently retrieve the references to and from of `foo` to some depth (often greater than 1). Some languages have better tools for this kind of symbol resolution than others, but in many cases they're highly bespoke. Github's stack-graphs and Sourcegraph's SCIP are the best attempts I've seen at solving this in a generic way, but neither of them have gotten all the way there yet.
I've been half-seriously thinking about running neovim in headless mode and using it as an interface to all the LSP's for each language, and resolving all the edges of the symbol graph that way - I might still try it one of these days
On design doc convention - I think you're right, it's not completely 1-1 with the term today, and there's not a single definition that everyone agrees on. We're using it as the initial mental model of where we want to plug in, but there might be a better succinct way to describe it
Haven't tried the raw ast yet, curious how that goes
-Schema: We agree. After seeing how many times it has asked to see the database schema when working on a task, that is definitely something we will be adding.
-Manual / RAG: We did some RAG magic in the first two iterations of our product and found it was really hard to get right when the task was not well defined and as the size of the codebase grew. We found that our users were watching it do the search for the files and basically saying, "Let's see if it finds the code file I already know we need." So, we decided to go manual so at least the user isn't unpleasantly surprised by the results while we determine the right path forward with RAG. Or maybe even combine both where the user gives an "entry point," and the AI figures out the rest of the files
-I think you're right to be skeptical. In our experience, the LLMs are really best at breaking down a task to be just one notch more specific. So, I think the output here wouldn't be the best for a more high-level task. Our plan to handle this is to allow plans to be solved recursively, where you continually create a tree of subtasks, and only when you get to a specific enough "leaf" do you actually go through with generating code.
-You're right; I think we used the term "technical design doc" more generally to steer people's thinking toward software design in English rather than code. I like that you mention diagrams, though, because we would love to incorporate diagrams into our documents as our tool and LLM's capabilities increase!
-This is a very interesting idea! We used a version of this (a select set of elements of the AST that we tried to rank in importance) in our initial RAG system to give the AI a "starting point" for the RAG code search. It worked pretty well. My concern would be that the quality of the search might then depend on the naming/structure of the code (I'm thinking some 1000-line mega functions or something)
Thanks so much for the thoughtful questions!
EDIT: I see Maks also responded and I think we both offer similar messaging with slightly different delivery :)
Iterating back and forth with GPT is cool at first, then it turns into a pain This is a lot better!
From seeing the demo, one thing I'd prefer, is if Glide would immediately generate everything, the thoughts, the plan and the code, together, then I can get a better overview and tweak where necessary
Keep up the great work!
We've been thinking about ways to make the user experience more seamless, and I think what you suggest could be a good option. We just need OpenAI to give us more tokens/sec! So you're not watching code generate for 10 minutes before being able to do anything.
Pros: it was kinda easy to get started, and Glide was able to figure out something that ChatGPT hadn't, which made a big difference, because I was about to have to rewrite a bunch of code otherwise
Cons: some things were not very intuitive, the chatting window is small and it's hard to copy text and read things in a slightly longer conversation - also wish it would just ingest the repo and figure out what to include in the context by itself, the method/section searching/selecting is tricky to use
All in all I'm very excited about what Glide can do and look forward to seeing its evolution
Thank you for building such amazing tool
If you're interested in staying involved I just created a discord and put a link in one of the comments.
some languages are still experimental - C, C++, Java, Go, Scala, HTML, CSS
there were also some blips on OpenAI availability, they say things are back to normal now though
Right now the parser intentionally tags modules as classes, we made some assumptions in the generic parser code for other languages that don't quite align with ruby's notion of modules, but that'll be adjusted in the near future.
Both items above coalesce into a desire to 'rewind' back to the top of the process and try to get the context understood at the beginning, and have that clearly reflected to the user so that as you move through to details it gets easier to handle the nitty gritty without as much editing.
We will look into why the Triage edits were not reflected in the plan. And I like your suggestions on being more specific on updates to the Plan.
I have a few suggestions in the meantime that could help to get you part of the way there as we work on these new additions:
- If you select a piece of the plan, you could ask that step to be regenerated based on new edits and then replace that piece either with the "Replace selection with summary" button or copying directly
- You can Ctrl-Z in the document editors to go back. It sounds like this might help a bit with rewinding as you describe
Let me know if that makes sense!
Specifically, in the "Repo to index" section, if you click the dropdown on the right side you should see the repo you have added.
Hopefully that helps but let us know if you're still having issues!
Glide is much more structured. I gave it a real test on one of my repos and it outlined something I would expect from a junior engineer who messed up a lot of little details- e.g. function names and parameters were slightly off. I think if there was a type evaluation step/it was typescript-specific it would match an engineer with 3-9mo of experience (great!). But it failed to match basic codebase conventions e.g. using require on a codebase that is all imports.
For the authors, I would recommend using GPT4-turbo and huge context windows, or have an option to upload an OpenAI key. GPT3.5 is just not worth it- and I do sort of suspect GPT4 isn't being used because of the speed of the output but I'm not sure.
[1] https://github.com/seveibar/dir-prompt
We'll have to figure out how to extract codebase conventions efficiently and inject them into the coding prompts. Using up all 128k tokens has given us worse results (common theme), but hopefully with newer 1m context window models this concern away
Sadly it errored with the classic NextJS:
Application error: a client-side exception has occurred (see the browser console for more information).
The error in the console was: 601-ce9691b65ce5066e.js:4 Error: An error occurred in the Server Components render. The specific message is omitted in production builds to avoid leaking sensitive details. A digest property is included on this error instance which may provide additional details about the nature of the error.
https://discord.gg/FxJmf2T6
"Safari cannot open the page because the address is invalid."
after accepting the invitation. It did work on Chrome though