I have been working on a new approach to using LLMs for software development.
Currently with most of the AI coding tools out there you are either using a fancy autocomplete mode in the IDE like copilot or some sort of conversational mode like chat GPT or copilot chat.
It seems to me that when changes are suggested in the chat mode it frequently erases code or otherwise mangles the existing source code in the file being edited. In the case of using chatGPT I find that you have to do a lot of manual copying and pasting of snippets between the chat UI and the IDE.
This workflow is not ideal.
I started looking at the problem and came to the conclusion that the right method to use excising LLMs in a coding workflow was to solve the merging problem. If the merging problem could be solved and snippets produced by the LLM could reliably be applied to an existing file you could prompt your way to building a complete application.
To solve the merging problem I decided to look at how code is actually structured. ASTs (abstract syntax trees) are able to parse a source code file and represent the actual structure of the code as nodes and branches in a tree structure. While the code is in the tree structure it can be manipulated in a robust fashion and then the whole code file can be regenerated from the tree structure.
By breaking the file down in to the tree structure I can surgically replace functions or methods that are provided in properly formatted snippets provided by LLMs. We parse both the original file and the new snippet and identify duplicate definitions. In the case of a function we simply replace the existing function with the new one from the snippet. In the case of a class we find the existing class and replace or add the methods provided in the snippet to the existing code.
Working with the code in this fashion guarantees that new code being inserted is syntactically correct and the existing structure of the code being edited is retained over multiple editing iterations.
The result of this experiment is this this tool.
It works only for JavaScript but the same methodology could be applied to other languages by implementing similar intelligent merging functionality and a bit of prompt engineering to tailor the approach to other languages.
https://github.com/mmiscool/aiCoder
Currently it supports using the following LLMs:
Local Ollama (no api key needed if running locally and you have a GPU)
OpenAI
Anthropic
google gemini
1 comment
[ 2.9 ms ] story [ 10.1 ms ] threadIt seems to me that when changes are suggested in the chat mode it frequently erases code or otherwise mangles the existing source code in the file being edited. In the case of using chatGPT I find that you have to do a lot of manual copying and pasting of snippets between the chat UI and the IDE.
This workflow is not ideal.
I started looking at the problem and came to the conclusion that the right method to use excising LLMs in a coding workflow was to solve the merging problem. If the merging problem could be solved and snippets produced by the LLM could reliably be applied to an existing file you could prompt your way to building a complete application.
To solve the merging problem I decided to look at how code is actually structured. ASTs (abstract syntax trees) are able to parse a source code file and represent the actual structure of the code as nodes and branches in a tree structure. While the code is in the tree structure it can be manipulated in a robust fashion and then the whole code file can be regenerated from the tree structure.
By breaking the file down in to the tree structure I can surgically replace functions or methods that are provided in properly formatted snippets provided by LLMs. We parse both the original file and the new snippet and identify duplicate definitions. In the case of a function we simply replace the existing function with the new one from the snippet. In the case of a class we find the existing class and replace or add the methods provided in the snippet to the existing code.
Working with the code in this fashion guarantees that new code being inserted is syntactically correct and the existing structure of the code being edited is retained over multiple editing iterations.
The result of this experiment is this this tool. It works only for JavaScript but the same methodology could be applied to other languages by implementing similar intelligent merging functionality and a bit of prompt engineering to tailor the approach to other languages. https://github.com/mmiscool/aiCoder
Currently it supports using the following LLMs: Local Ollama (no api key needed if running locally and you have a GPU) OpenAI Anthropic google gemini