Ask HN: How do you use AI for software engineering?
It feels like so much has changed in the last 18 months and I feel like I'm behind the curve.
I love software engineering and have been somewhat hesitant to use these new tools. But I am now committed to becoming an expert on using AI in my development workflow.
What are the tips you would give to somebody who is trying to figure this stuff out for the first time?
I have Copilot installed. Are there other tools that are making a big difference? Any particular workflows that make these tools most effective? Etc. I'm open to hearing everything!
Thank you all!
EDIT: So many amazing answers. One thing I'm curious about... if you can, I would love to see example transcripts from ChatGPT. Obviously many won't be able to be shared, but hopefully somebody has something that can!
52 comments
[ 3.3 ms ] story [ 88.0 ms ] threadWhen it comes to pure code, ChatGPT will deceive you. I’m at the point now where I’m very hesitant to copy and paste ChatGPT code. I’ll have to completely vet it first, or it’ll have to literally just be a line or two. And even then I’ll likely have to refactor it. ChatGPT is just wrong too often when it comes to the actual code either due to lack of context about the problem or just downright hallucinations. It’s even gotten some boilerplate wrong.
When it comes to Copilot, I honestly haven’t been able to quantify how beneficial it’s been for me. I’ve often had to delete or modify its code. I think it could partially be because I’m not as skilled with it. It could also be that I’ve just had to write a lot of JS lately, and there’s probably a ton of bad JS that it’s been trained on. I remember being more impressed with it when I write Rust. But I’m also new with Rust. I need to try it with something like C#, which has more difficultly than JS but I’m also proficient with it.
All in all, AI tools for me have been just that, tools I can use to help me get the job done.
This is a common phenomenon with humans, FWIW. The same sort of thing happens with traditional information sources. For example, when a media outlet reports on things we don't know much about personally, we believe them. Then they cover something where we have direct domain expertise and find all manner of misunderstandings and errors trivially, but instead of suspecting that they're likely just as wrong about a lot of other things as well, we assume it's a special case where they just got our special knowledge domain wrong.
In any case, to answer the OP... I use these new AI tools to generate content where the details don't matter and the cost of being wrong is near zero. Such as, graphics for slides, market/product/pitch blurb pablum, etc.
I use them to compensate for my limited artistic/graphic design skills and to overcome my propensity to tediously labor over copy despite that copy being basically throwaway.
Kind of in the same vein as the strategies around gamification, where some identified frailty of the human psyche is being leaned on for cash.
It's not quite a con because the technology is useful and certainly worth something, but it's not non-exploitive either.
The context sensitive stuff like highlighting a line of code and asking questions about it and the completions are okay too but the chat is the most useful part imo.
ChatGPT to find fixes for weird errors.
Copilot to autocomplete code.
I feel guilty for not using paid ChatGPT subscription more.
I have briefly tried it in my IDE, but so for I don’t like the experience of any of the plugins. They feel a bit awkward to me in their current state. But, I know other devs who love the integration directly in their IDE.
I also use copilot/chat in VSCode and have used GitHub copilot in the terminal in the past.
But really nothing above is as useful as ChatGPT.
Apparently Claude 3 Opus is it's first serious competitor, I haven't tried it.
Ignore negative feedback on ChatGPT from anybody who hasn't used the paid version, free stuff isn't the same.
“I’m going to give you a list of 50 variables and I want you to format them into functions like this:”
It’s faster and easier than writing a script to do it.
In some sense, it's a bit like pair programming... with a partner available 7/24 and responding immediately.
I also feel like this generation of senior/staff SWEs (the senior pre-AI engineer) is in a very unique position for the following reason: we know what the correct code should look like, because we've written it ourselves many times, and we know how to review and modify it quickly. I don't think new SWEs will have that same ability. At the same time, that ability may not be required in the future as AI gets better and better at following instructions correctly.
AI's future in coding is uncertain. I think there is a new unrealized engineering platform on the horizon that can only be realized through a very small and focused group of architects using AI (maybe even 1 person), and that this new platform, which is focused on complexity management and system evolution, will change how we "code" completely, at the same time resetting AI's copilot utility in this new world.
Also for refactoring/creating tests and writing/fixing docs
About the second point, yes. Sometimes the LLM will create "almost correct" code but you definitely need to understand it and know why it is wrong.
I'm currently dealing with immense legacy tooling (90s to 2010s) and codebases (C, Cpp, C#, Python, JS) in a central team that fell apart a year before I joined. They had 15+ people with diverse profiles who made snippets and undocumented solutions everywhere, and pretty much all quit at the same time due to working conditions at that time. Needless to say it's a daunting task to try and understand everything, even more so when you discover it all as it breaks down and prevents 500 employees from using a functionality you didn't even know existed.
While I can survive and navigate just fine the situation, it's been very helpful to use LLMs to run a parallel effort that helps me understand just why the hell somebody wrote a self-altering stored procedure in a 800GB MsSQL DB which fails due to poor design and high recursion. Sometimes it is just about fixing some syntax, but often enough it is about managing expectations and offering replacement solutions which create less technical debt - with minimal interruption to the entire company's workflow.
It isn't abnormal for my day to require reverse engineering and fixing a 500-line stored procedure, a Go CLI, multi-CI imbricated pipelines, IIS websites, cronjobs, and even c-shell scripts (!). That's when I'll occasionally throw a big pile into chatgpt as a bit of a glorified rubber ducky. Also tried local LLMs but my 3070ti can't sustain any worthwhile model.
Many of us here are in a very unique position of having seen the entire stack - from quantum tunneling all the way up to your CI badly parsing a groovy script or why an endlessly long JS program is stumbling over itself because its writer didn't factor in any estimate of runtime complexity. For me this is a beautiful and lucky thing to see through all those orders of magnitude with a certain equanimity, and I kinda wish I could find it in my own colleagues as much as I'm already seeing it in the illusion of competency created by LLMs.
Yeah, it'll be interesting to see how that changes things.
I think even pre-AI there's already a big difference in old-school programmers and the current generations of graduates.
People like myself who grew up programming in the 8-bit era are used to having to design things from scratch, out of necessity, whether that's custom algorithms or data structures, or something like a parser or editor, or whatever. It seems that current generations are more used to assembling things out of lego blocks - more system integration than what I would consider as programming. I guess this is fine when it works - when you can find the pieces you need, but if that's the only skill you have then there is nothing to fall back on if you need to create something complex truly from scratch.
On one hand, it makes me really think about my first job - when we automated a lot of things, but the people who did it knew how things really worked. After 2 years, all the newcomers only knew if the indicator is red or green without any understanding of what's really going in - it went downhill from here.
On the other hand, if I understand your last paragraph correctly, the real question is do we really need to know all those details ? Would it just be better to have it automated, and the AI agent just ask you questions relating to the desired outcome and outlines the limitations and impacts of your choices, as well as the options available.
I use the language models conversationally as a supplement to, and interface to documentation. Things like high level design questions. I also use it to parse through error logs that I don't understand. Sometimes it can point me in the right direction faster than doing web searches for random snippets in the log. My actual code generation is fairly minimal. At most I usually just ask it for an example to convey an idea in some language, and then I adapt the example to meet my needs.
2) Mechanical operations (e.g. translate this SQL query into a knex query)
3) Code review, verification of modern idiomatic approaches in languages I use infrequently
Design - ChatGPT
Major features - Aider
Fixups - GitHub Copilot
I'm thinking about forking Aider because I think it could easily handle the design portion if I had more control of how it applied diffs. Right now it decides when it applies diffs in chat and sometimes you want to go back and forth before it starts to wreck a file without enough requirements.
For example, making both redux and react-query drop in replacements using just rxjs.
I learned a great deal from these. I'm not going to use them but now I understand rxjs way more
The two areas where it's been particularly helpful:
- Tricky Postgres queries - it essentially explains the pretty dry docs to you like you're five
- Partially because I use nvim without intellisense (made the switch after 23 years of vim last year!) - occasional TypeScript type issues
However, not using ai today. Mainly because, I am running legacy ucode.