It is definitely increasing mine. It’s almost a mindreader.
Opus seems better at code though so I am mixing them ; copilot for continues and Opus for big/new slabs.
From the article: unpredictable is correct but with a retry or two that is usually fixed. Sometimes of course it just cannot do it.
It’s slow; that’s true but a matter of time; if you see things like Groq and the others that are very fast, you see the near future will be faster than you can read, so you can create multiple or even self-fixes by the time you are even noticed it did anything.
Claims about LLMs increasing programmer "productivity" should start with a measurable definition of productivity. I have 40+ years experience programming and I have yet to find a useful metric of "programmer productivity," so I am going to assume that all references to "productivity" come from subjective feelings and opinions -- like saying "I feel healthier" after a colon cleanse.
Junior programmers will likely find LLM coding tools magical because they don't have the skills and experience to outperform the "AI." I liken that to my parents thinking I have a special genius with computers when I get their wi-fi to work again or show them how to search for an email message -- let's call it a skills issue.
More senior programmers may find LLMs handy for filling in boilerplate or dropping in a well-known implementation of an algorithm or code snippet. Previously we would have to break stride and search StackOverflow and other places, or a book, or figure it out ourselves, so maybe we gain some time getting a chunk of code regurgitated for us, and lose some of that time carefully reviewing and testing what we got. A significant part of programming involves rote tasks, and in the most common domains (like web development) the code solutions tend to look the same. If GPT or Copilot can save me a few seconds of looking up how to reduce an array in Javascript, great, but I'm not sure I'll call that a big productivity boost because I don't measure my productivity in lines of code per minute.
To put it less charitably, the closer your skills approach 0x productivity (relative to the 1x - 10x programmers) the more an "AI assistant" will increase your output. But since we don't have a good way to measure productivity or compare it to past performance, much less to other programmers, we just talk about subjective experience made worse by Dunning-Kruger effects.
I unsubbed because it didn't seem $10 / month helpful to me, to be honest. I subbed for awhile and thought it was magical and a gamechanger and eventually became bored, and it's suggestions simply weren't useful enough. The magic of auto fill disappeared for me and the gimmick was over.
I had a ton of headaches from blindly accepting it's output when I'm in a rush and then later on realizing that it made my code dumber and reduced functionality. There were nuances as to why I was coding things the way that I was, why I used solution C, and not solution A, when A was the most obvious and rote, but broke features.
Copilot does kind of suck, it’s incredibly slow. I switched to Supermaven and it’s significantly faster at suggesting and the suggestions are at least twice as good, at least on my project.
I find your reasoning quite odd. It either must be 0 effect or actively harmful to not justify a $10/cost. How much do you value an hour of your time? Because even 1 percent productivity gain would be worth more than $10 a month for me.
They explained decently how it was harmful. Cleaning up after it regularly broke things, when given a chance.
To be blunt, it's perfectly reasonable. They didn't even get that proclaimed "one percent!"
How does one even meaningfully measure this? No matter.
In their case, instead of saving time or effort, it was adding. It was a removable cost.
If one really wants to dwell, there's game theory. Could it be made to be effective? Probably. Is it worth it? Don't know.
They told us 'no', we should listen for their situation.
At risk of splitting hairs, I need more than one percent. This isn't a vacuum, I'm not a spherical cow, etc.
Point being: wanting both my money and attention has a high bar of admission. Learning a thing has a cost, becoming dependent, and so on.
I'm well into rambling now, feel free to tune out. I'm not sacrificing autonomy for a pittance. I was fine before I knew about $OFFERING. Truth in advertising is idealistic, at best.
But saying it wasn't worth the $10 contradicts a bit with net negative harm. The way they worded it, made it sound like while there was harm, there was some sort of benefit, but it just wasn't worth specifically $10.
I guess I see it like how I see any autofill, like the one above my keyboard on my phone right now. When I first got a smartphone and saw it, I was thrilled and messed around with it. I would play with it and see how things would go if I just spammed it to see how sentences turn out. Eventually I just cut it out even if it might save me time. I don't care about it anymore, and life isn't just about reaching the end of a sentence as quickly as possible.
However in the case of AI, I am not sure it helps me code any faster. I have to do tons of code cleanup when it messes something up and I "forgot" the logic of how the application was working because I let AI take the reins.
I thought it was interesting that the post called out the somewhat unique project structure. My experience is that copilot is super helpful on new projects, less so on maintaining/fixing existing code.
17 comments
[ 2.1 ms ] story [ 47.9 ms ] threadOpus seems better at code though so I am mixing them ; copilot for continues and Opus for big/new slabs.
From the article: unpredictable is correct but with a retry or two that is usually fixed. Sometimes of course it just cannot do it.
It’s slow; that’s true but a matter of time; if you see things like Groq and the others that are very fast, you see the near future will be faster than you can read, so you can create multiple or even self-fixes by the time you are even noticed it did anything.
Junior programmers will likely find LLM coding tools magical because they don't have the skills and experience to outperform the "AI." I liken that to my parents thinking I have a special genius with computers when I get their wi-fi to work again or show them how to search for an email message -- let's call it a skills issue.
More senior programmers may find LLMs handy for filling in boilerplate or dropping in a well-known implementation of an algorithm or code snippet. Previously we would have to break stride and search StackOverflow and other places, or a book, or figure it out ourselves, so maybe we gain some time getting a chunk of code regurgitated for us, and lose some of that time carefully reviewing and testing what we got. A significant part of programming involves rote tasks, and in the most common domains (like web development) the code solutions tend to look the same. If GPT or Copilot can save me a few seconds of looking up how to reduce an array in Javascript, great, but I'm not sure I'll call that a big productivity boost because I don't measure my productivity in lines of code per minute.
To put it less charitably, the closer your skills approach 0x productivity (relative to the 1x - 10x programmers) the more an "AI assistant" will increase your output. But since we don't have a good way to measure productivity or compare it to past performance, much less to other programmers, we just talk about subjective experience made worse by Dunning-Kruger effects.
I had a ton of headaches from blindly accepting it's output when I'm in a rush and then later on realizing that it made my code dumber and reduced functionality. There were nuances as to why I was coding things the way that I was, why I used solution C, and not solution A, when A was the most obvious and rote, but broke features.
To be blunt, it's perfectly reasonable. They didn't even get that proclaimed "one percent!"
How does one even meaningfully measure this? No matter.
In their case, instead of saving time or effort, it was adding. It was a removable cost.
If one really wants to dwell, there's game theory. Could it be made to be effective? Probably. Is it worth it? Don't know.
They told us 'no', we should listen for their situation.
At risk of splitting hairs, I need more than one percent. This isn't a vacuum, I'm not a spherical cow, etc.
Point being: wanting both my money and attention has a high bar of admission. Learning a thing has a cost, becoming dependent, and so on.
I'm well into rambling now, feel free to tune out. I'm not sacrificing autonomy for a pittance. I was fine before I knew about $OFFERING. Truth in advertising is idealistic, at best.
However in the case of AI, I am not sure it helps me code any faster. I have to do tons of code cleanup when it messes something up and I "forgot" the logic of how the application was working because I let AI take the reins.
Not OP, but this is the vast majority of the improvement I've seen.