Ask HN: Am I hurting my programming career by not exploring AI coding tools?
I have 0 interest in using ChatGPT, Github's Copilot, or any other AI coding tool.
I have a hard enough time using and trusting snippet engines or doing due diligence in implementing a solution I find on stack overflow.
Roughly 30% of developers in the 2023 stack overflow survey responded the same way which is the minority and I would guess will shrink every year https://survey.stackoverflow.co/2023/#section-sentiment-and-usage-ai-tools-in-the-development-process
59 comments
[ 5.3 ms ] story [ 60.0 ms ] threadIt also entirely depends on what sort of career you want. There are a million flavors of software engineering careers. Some flavors will be affected much more than others.
Where I work, our manager announced their new AI policy: use it if you want, but the company won't be paying for it. 3/4ths of the engineers in my location have no plans to use it at all.
My answer is: "No". You are not hurting your career, but it is still something that could be fun to use.
Today, I don't think the tools are good enough to make a material difference. It may help a bad engineer tread water, but it won't take you from good to great. It may save you time writing basic boilerplate and individual functions, but I suspect 99% of engineers don't struggle with that. What's hard about our jobs is knowing how to orchestrate the whole thing and put structure around complexity. AI can't do that yet.
When I use it personally, it feels like a harder context switch trying to describe in english what I already know how to code. Then I still have to review the function to make sure it's accurate. It feels like a waste of time and an additional context switch.
Whenever the AI gets better, we'll have to use it to be productive I have no doubt. But the pool of engineers will change too - there will be a categories of engineers who can't debug the AI output and who still write crazy prompts.
Maybe I'm old, but I'll only be worried about AI when it can write and maintain a full app with no human intervention.
Agreed and very well put, but with a caveat:
It can also help when venturing outside one's domain.
I've tried using ChatGPT for my strengths and sometimes it helps for minutiae but for the most part it's faster for me just to write the code.
you just have to get used to the way it "thinks" and how it "understand" your request, to write better prompts, you can even manage to send it half incomplete sentences if you really know what matters to it if you want to save even more time
and of course if someone won't start using it regularly they'll never reach a point where it's faster to just ask it for a function or a script than to write it
I guess I am a little biased against the AI tools also as I've made a successful 34 year career in software development without using those tools, but I'm also aware that overnight, the world can change.
Copilot efficiency is directly linked to the readability of your code and the quality of your comments, so if you have a messy file, it can be better to ask ChatGPT in natural language, it's also better to use ChatGPT if important code related to what you're writing is spread across a lot of files because Copilot won't necessarily take everything into account.
On the other hand Copilot is better for one-liners, small functions, boilerplate, while ChatGPT can often do more complex stuff on the first try if your prompt is good enough and you don't need it to call something created after 2021, it can also sometimes be useful for debugging.
I'd say I autocomplete line or functions a few dozen times per hours with Copilot, and ask ChatGPT a question or two every hour.
There is also one thing to take into account, if you've been a professional for 34 years, it seems likely that you don't work with the latest popular language or framework. Models from OpenAI are order of magnitude worse at other less popular languages than Python or JS because they had less training data for less popular/older languages.
Words from a recruiter at a company I won't name for now:
"Unfortunately, at this time, we do not offer a take-home test option to our candidates. It is definitely something under discussion, and we will continue to evaluate this as we scale. The decision stems from a couple of our leaders who have had unfortunate experiences in the past with candidates who used outside resources to complete their tests, which has given them concern in allowing this as an option moving forward."
And then they added once I was rejected, presumably for continuing to try to push the take-home option and evaluate it on accommodations-for-disabilities grounds:
"I know we talked about adjusting the process with you for your preferences to do a take home test in lieu of live coding, or at least have you speak with a hiring manager before doing the live coding which is what we would be able to do if the team had interest in moving forward. However, the team did reach the conclusion that if doing the live coding wasn't something you were going to be interested in/had general trepidation around, would they really be getting a great read of your skillset if it's not something you're jazzed about?"
I hate how much employers seem to not want to evaluate candidates based on real conversations and instead rely on arbitrary assessments that don't map to the real-world day-to-day work.
Which is kind of the best we can hope for. Beyond that, people are people, and are going to keep on making weird decisions.
The problem is bias. Study after study has shown that those "casual chat" interviews are worse than useless at measuring anything at all.
Kahneman's book, Noise, has entire chapters on this problem. The only solution that empirically seems to work are a) interview panels and b) pre-defined standard rubrics with clear evaluation criteria.
Defining those rubrics is hard and the results aren't perfect. But when done well, you can get up to about a 70% correlation with on-the-job performance. Nobody is known to have achieved better.
I’m glad to have a vigorous discussion about code I wrote during my own time. Go ahead and create a standard rubric that covers the project itself and the follow-up discussion. This is what I’ve done when hiring. It’s great because it demonstrates the employer knows what they’re looking for from the role, and that the team has sufficient experience to conduct a conversation in the relevant domains.
When I hear there’s a minimal number of interviews, and the main one is an intense live leetcoding session, I tell them I have no interest but if anything changes on their end I’ll be glad to provide sample work and have a discussion about it. The problem is these live sessions are extremely draining to prep for, provide no gain for the candidate (unlike writing code that can be retained), and they reveal practically nothing about the company.
My advice, if the company is smaller, open up to asking about accommodation after gaining employment and showing that you're an asset.
Instead of painstakingly reviewing the output of AI, just write a bunch of tests. It's something you would do regardless.
Is it possible you are not breaking the problem the right way? I.e get better at understanding what it can do for you and conveying to it in a better way what you want.
To me it seems like a huge handicap not using these. At the same time if you use it too much your skills may degrade overtime (maybe the skills that degrade are not worth holding onto in this new world anyway... we'll see).
It reliably acts like a junior dev who can sort of complete the statement or add the next N branches after you add a few lines. But you have to be good at spotting bugs or inefficient implementations quickly and probe it till it corrects it.
ChatGPT saves time writing short functions maybe 10-20 lines. This is maybe 10 minutes a day.
ChatGPT helps start/debug things outside my domain where I would normally need to read some guides before even starting. This doesn't happen every week, but when it does, it's hours saved.
Copilot saves a bit of time with simple 1-3 line suggestions where you would already know how to write it, or can immediately verify correctness. This is probably 10 minutes per day saved as well.
So, on average, I get maybe 30 minutes or so per day saved with close to zero downside. It's not going to make or break a career but it's a good chunk of time.
Exactly. Copilot is really just a fancy autocomplete, and I have noticed I've started relying on it.
And I use ChatGPT for the same thing: debugging things I'm not super familiar with. Even if it can't offer me a complete solution (and it mostly can't), it is a really good pointer most of the time.
Usually it either confirms my initial assumptions or often enough shows a method I hadn't thought of.
Either way ultimately I still write the code.
I wouldn't worry about it. AI tools are trendy right now and are being hyped far beyond their actual usefulness. I've seen this before with other things. Eventually, things will calm down and AI tools will become stable and just another standard tool that will provide incremental improvements.
That will be the time to try them out and see if they are useful.
Typing code is not where the value of a software developer comes from: it's the thinking, the planning, the accumulated experience, and the communication with other people where the most value emerges. You will not hurt your programming career.
Way back when IDEs were the hot tool, some developers kept using the command line and vim. That choice did not hurt their careers: I've worked with such developers and they do just fine. IDEs make a lot of things easier, but they don't make the difference between success and failure.
You will need to be very good at programming to be competitive against people that use AI coding tools.
GPT4 is an enormous upgrade over standard ChatGPT (3.5 Turbo) for logic and code. I believe Code interpreter was created by taking the base model GPT4 (not the instruct model) and adding a "code layer".
I'm some % more productive because of Copilot and especially because of ChatGPT(with GPT-4). No idea what the number is but it feels significant since these tools often helps me get unstuck, generate new ideas, explain code, propose solutions, do some trivial tasks etc.
I think the people who aren't curious enough to experiment with it, or have some mental block against it because they feel threatened, or can't use the tools correctly are doing themselves a disservice in the long run.
I'm also noticing a significant difference in the performance of colleagues who use these tools a lot vs the ones who don't go anywhere near them.
This is interesting because in my workplace, I've noticed no difference between the two groups. The ones using these tools think they're more performant, but if they are, it's not to a significant enough degree as to be obvious to others.
My main use case is to ask it other ways to approach solving problems I already understand. Sometimes it reveals interesting ideas others have already had, which I like. The other day I was using promises in the JS world to create a sort of sequence of who-knows-when-it’ll-finish tasks, and GPT pointed out that what I actually wanted was an async generator. Kind of obvious but also, not at all — people rarely reach for generators in the JavaScript world. So that was cool.
Despite a fair amount of experimentation and building basic stuff with the API, this has been my most valuable use case yet which has real rewards. I made a tool which uncovers solutions to error stacks I pipe into it (in VS Code or in a standalone situation where it listens to your app’s errors), and it was interesting, kind of worked, but was mostly just a time sink. It could be fine tuned and I’m sure Co Pilot X will solve this elegantly, but there’s no real magic underneath this. You don’t need to know it. You’d figure it out fairly quickly if you needed to.
The secret sauce in AI isn’t something you learn easily, on the other hand. That’s a career in and of itself, and it’s not trivial to pursue. For the time being, I think it’s totally fine not to adopt these tools as a developer. Just keep an open mind to it, explore when you feel interested, and continue focusing on doing what humans do well but machines don’t.
These tools are clearly very powerful and likely will get more powerful. What they do would have been considered science fiction 3 years ago. Getting into the habit of asking your "assistant" to perform tasks seems like an investment which will give solid returns.
That being said, I don't think you are hurting yourself much today. But the trend is not in your favor.
PS: these tools are tremendously useful when creating documents. YMMV.
Just try it and see what the results are. You may be surprised how good they are.
And if you weren't planning on using SO or Google, then you don't need to think about it.
Your mileage may vary, but I've found it very helpful.
1. A better IntelliSense, and 2. Getting up to speed on things I have less experience with
Are you hurting yourself? Only you can answer that. I know I would be hurting myself if I didn't use all the tools that were useful.
Coding tools, like Copilot, are completely useless on the other hand. I think they would need to be integrated with the compiler and the IDEs indexes to be of real value. Perhaps you would also need another network architecture than transformers to really understand the tree shaped nature of code as well.
I found it quite a leap at first to trust ChatGPT to understand my questions and system but it really does give reasoned answers even if some of them are wrong. It’s worth knowing this and asking it questions about the code, in fact it’d be great to get definitions of functions used inline in the ChatGPT ui to be able to at least quickly check if the right parameters were being sent. Maybe this checking of lies is much harder for junior developers when they get the slightly broken output and struggle to refine it?