The amount of things I’m expected to be able to accomplish constantly increases. I get it, I’m becoming more senior, but at the same time, it’s kind of crazy that a single person can just be told to do the kind of things I’m told to do with the kind of deadlines I have. I can think of some takes I do by myself with code where a decade or two ago I’d be talking about hiring dozens of employees or contractors across the globe to physically enter a building and make a global business information system possible.
I think this is related to the productivity wage gap. I’m being paid lower than I was in 2018 adjusted for inflation and I’m far more capable, producing far more value for my employer. As my tools get more valuable my labor gets proportionally less valuable. (A certain historical figure predicted this end state of capitalism).
Here’s another take: I don’t give a fuck about the code quality at my company. I’m not paid extra to write good code. My bonus is based on basically nothing I can influence. If the whole company goes out of business on Monday I’ll enjoy my long weekend.
To build a house, it used to be necessary to hire experienced craftsmen. Now most of it is delivered pre-fabricated to the building site and laborers hired for the day nail it together.
Cars used to be hand built by machinists and metalworkers. Now laborers tighten a bolt as cars roll by on an assembly line. Or a robot does it.
Computer coding is going the same way. We lived and worked through the craftsman stage. Now we're becoming laborers.
This is why I am concerned that really good AI tools (be it LLMs, or not), are only going to make people dumber over time. It is true that those people that are highly motivated will be digitally enhanced by LLMs, but the hard reality is that this does not encompass most people. Most people will do the least amount of work possible to solve a task. Rather than taking the time to understand and critically reason about the output of an LLM, these people will instead take it and run, only asking questions later [if at all].
And as we zoom out and think more farther into the future, I see it getting much worse. If AI is really doing all the "hard stuff", then the general human incentive to learn and do them at all quickly treads to 0. This isn't going to be everyone, I think some people will absolutely become "10x developers" or the equivalent for other domains. But all this will result in is more inequity, in my naïve view. The universe has a fundamental property that things move from higher energy states into lower ones. From a human POV, I think you could apply a similar idea, if the need to be smart quickly recedes, then in general we may degrade unaugmented human collective intelligence over time.
I don't know, maybe we'll figure something out to make it much easier to learn and ingest new concepts, but it seems more and more to me that the high obstacles for human brains learning things (with poor memory) is too big a bottleneck to overcome any time soon.
Is that necessarily a bad thing? What if high quality isn't required to do the thing you wanted to accomplish?
Many other mass produced products we buy today are clearly lower quality than ones crafted by artisans 50 years ago, and yet they do what we need them to do at a fraction of the cost.
These cheaper products are also far more wasteful and taxing on natural resources. This is the “growth at any cost” mindset. It’s been great for us in the last 100 years, but I’m not convinced it’s actually sustainable in the long run.
I think that regardless of category, we should strive for making quality easier and cheaper to achieve and sustain. Unfortunately financial incentives favor velocity and volume at the cost of all else and so instead we’re increasingly pumping out vast amounts of garbage.
I think that's a sweeping generalization. For example, it's much better to have a bunch of food that's mostly garbage than to have a famine where all food is super high quality. There were points in history where this choice was made (obviously unconsciously because the choice is too obvious to even think about). Other examples abound.
Technology often makes things much cheaper while reducing quality. Sometimes that's bad, sometimes it's great.
There is a threshold of quality; once it goes below, it's a broken product. Personally, I'd rather have a high-quality product crafted by artisans than mass-produced ones, similar to preferring a dinner at a restaurant made by a chef over fast food. However, in engineering, every choice involves tradeoffs.
Internet made us dumber but also made a lot of things easier. Just like industrial machines made us weaker (or rather, not as rugged as we used to be) but also allowed us to get much more work done.
Not without a solid result establishing so. But good luck with this, you'd probably need to compare our world with internet and a similar enough world but without it, which does not exist.
In any case, internet can't be reduced to "tiktok et al"
TikTok may indeed be a contributing factor to the current collective brain rot, but I’ve learnt more so far from YouTube and access to random PDF documents on any technical subject I can imagine than I probably would have done in one lifetime without.
It was claimed about writing as well. Oral experience passing was considered superior. The Quran puts great emphasis on decoration of it so it can be transmitted and recited orally.
In a sense Socrates was right. It's always better to be able to recall something from your memory rather than having to look it up in a book. The classic example is the multiplication table. The problem is that humans have rather limited capacity for such a 'recallable on demand' memory.
The problem is that you have to invest actual effort to memorize things. Memorizing things you very rarely need is not worth it, even if you're amazing at memorizing things. It's impossible to memorize everything you would benefit from having access to in written form.
On the other hand just because something is written down somewhere is not at all equal to 'having access to it'. Once you forget it it might as well not exist at all. The information needs to be internalized to be any useful, and for that at least some of it needs to be kept in your memory
And that's why digital documents are better (for average use-cases) than paper. Another argument for using technology to make things easier at the cost of possibly losing skills you no longer urgently need (such as "maintaining a library of paper books").
I'm terrible at recalling specifics but I feel that my "superpower" is that I'm really good at remembering if say a solution to a certain problem exists, and just enough breadcrumbs to find that solution again, be it a few key words that I can search for or similar.
This coupled with an broad interest in just about anything means I can often help people far outside my specific field, usually rather swiftly as well.
In a conversation with Phaedrus, Socrates worries that writing could impair human memory, as people depend on written words instead of recalling information. [0]
It's true though, as human (+ tool) gets smarter, human (without tool) tends to get dumber in the domain the tool augments.
The question is will we one day have tools so powerful, that the human is vestigial, and tool (without human) is just as powerful and cheaper than tool (+ human)?
But there are still people playing chess professionally because playing well is not enough. (You have to entertain an audience, which is only possible by "just playing well" if you're human)
True in general, yes, but writing is an elegant solution where the longer something is, the more likely you are to write it down because you are less likely to be able to remember it. The shorter something is, the less likely you are to write it down because it’s a pain to get out your scratchpad (or iPhone) for ten words or less.
It’s also an early example of the medium being the message. Socrates and his interlocutors are sometimes a parable on the transition from oral to written culture.
I find that it is true that new mediums and new technologies for language have a numbing effect on certain senses and an amplification of others.
Writing is beneficial in one regard but does have an impact on memory. Epic poems of great length were memorized in their entirety, a skill that would be a lot easier to develop in a world without writing.
IQ is a very different thing from literacy (or generally any skills one learns). I think IQ is not relevant to this discussion.
It's basically impossible to improve an adult's g-factor, for example. For children, things like nutrition and hygiene (e.g. no parasites) play a big role. But the kinds of things a human learns or tools they use doesn't significantly affect G-factor.
The mistakes that will be made are obvious, and those who fear that are correct, yet they will also be overcome because the ones not learning will fail and the ones learning will succeed.
Whoever said it was right. One of the dumbest public figures of our time was elected president as a direct result of the reach and amplification provided by the internet.
Qanon was purely a creation of the internet. Now go take a look at how many people believe one, many or all of the various Qanon alternative facts.
Those are convincing arguments that internet has (major) downsides and bad consequences. The scale of surveillance it enables is another one.
They don't prove that it makes people dumber. You have to quantify and qualify people and to define "dumb".
Maybe people are not actually dumber because of the internet, but the internet is very good at spreading ideas, including incredibly dumb ones, especially (because of how human beings work) those likely to cause feeling of outrage.
Maybe people are not dumber, just too defenseless against the scale of bullshit they are faced with because of the internet. Maybe internet is an incredibly good tool, but strongly requires good learning / training of critical thinking and there's not enough of this yet.
People used to believe a lot of crazy conspiracies and superstitions throughout history. The reason why we're so appalled at Qanon is that "it should be so easy for them to correct their superstitions, given the tools we have".
It's hard to argue that nowadays people believe more crazy stuff than before the internet was invented. (It's very easy, of course, to claim this, as many like to do.)
Yes. It seems that the proportion of people for whom 50% of their beliefs constitute unfounded, unjustified nonsense is actually far lower than it was before the internet. One of the main things that has changed, though, is how quickly ideas (including bonkers ones) spread. So whereas we used to see a significantly new bonkers idea only every few centuries, we seemingly now see several a week.
This sounds pretty similar to all technology. I remember Slashdot threads about how IntelliSense was making software engineers dumber. I remember my teachers saying that allowing calculators on the SAT made kids dumber. I remember people saying that spell checkers would make people worse at writing. I remember people saying at the start of the pandemic that Zoom meetings would make everyone anti-social.
None of this really happened. I mostly use editor tools to automate away tedium that doesn't matter; I type "log.Inf<TAB>" and it adds "src/internal/log" as an import at the top of the file and types the o and open parenthesis for me. I have not forgotten how to do that myself, but it saves me a couple seconds. Calculators didn't really make people dumber, though I have to say that a lot of arithmetic I learned in elementary school did make me dumber ("touch math" was the killer for me; slows me down every time I do arithmetic in my head; I need some brainwashing program that deletes that from my brain). Spell check didn't make people worse at spelling; spelling things wrong still has a penalty (C-w to kill the last word and spell it correctly), so the incentive is to still to lurn how to spel wrds rite. Zoom meetings didn't ruin the corporate world; I personally found them very helpful for memorizing people's names with a high degree of certainty. You see it under their face for 40 minutes at a time, and you learn it fast. In real life, probably takes me a few weeks for people I only see once a week. So, honestly a benefit for me.
The current state of AI seems very similar to these technologies. I did a copilot free trial (and didn't renew it). With the free trial I think there were a couple things it was good at. One time I wanted a CPU profile for my app, so I just asked the AI to type it in. Open a file, check for an error, start profiling to the file, stop profiling, close the file. Would have taken me a minute or two to type in, but Copilot typed it in instantly. I also did something like "do the same as the function above, but for a YAML file instead of a JSON file". Again, super trivial to type in that code, it's really only one line, but the AI can handle that just fine in an instant. I don't really think it's more than slightly smarter IntelliSense, but without any access to the compiler toolchain, so it can sometimes just hallucinate stuff that doesn't exist.
I've found this to be kind of an interesting way to proofread documents and design APIs. Give ChatGPT a document you're working on, and then ask questions about it. If it gets the wrong answer, then your doc is no good! Similarly, ask it how to write some code using your new API (without showing it the API). Whatever it writes should be the surface area of your API. This avenue is pretty interesting to me, and it's not replacing humans, it's just a smarter rubber duckie.
Overall, I think we're in a little bit of a hype phase with AI. I look at it kind of like a dog that has read every book, if such a thing were possible. Pretty smart, but not quite human yet. As a result, it's not going to do well in the areas where people really want to apply it; customer service, insurance claims, loan underwriting, etc. But it is pretty good for asking questions like "does my document make sense" or "please find some boilerplate to cut-n-paste here, I am going to delete this before checking in anyway". Also not too bad at slightly modifying copyrighted images ;)
In Phaedrus, Plato has Socrates tell the story of a dialog between Egyptian gods as a caution against writing—that writing would cause people to lose their ability to remember. On the one hand, Thamus's warning was accurate—cultures that rely on writing generally do not have robust memorized oral traditions—but on the other hand, we only have this story today because Plato wrote it down, and he cannot have been ignorant of the irony.
Every tool has this trade-off, and the existence of skills that will be lost is not evidence that the tool will do more harm than good. I don't think anyone here would argue that Socrates was correct that writing would be the end of memory and wisdom.
> To [Thamus] came Theuth and showed his inventions ... when they came to letters, "This,* said Theuth, "will make the Egyptians wiser and give them better memories; it is a specific both for the memory and for the wit."
> Thamus replied: "O most ingenious Theuth, the parent or inventor of an art is not always the best judge of the utility or inutility of his own inventions to the users of them. And in this instance, you who are the father of letters, from a paternal love of your own children have been led to attribute to them a quality which they cannot have; for this discovery of yours will create forgetfulness in the learners' souls, because they will not use their memories; they will trust to the external written characters and not remember of themselves. The specific which you have discovered is an aid not to memory, but to reminiscence, and you give your disciples not truth, but only the semblance of truth; they will be hearers of many things and will have learned nothing; they will appear to be omniscient and will generally know nothing; they will be tiresome company, having the show of wisdom without the reality."
People have different workflows, but mine is frequently, skim the documentation, make a prototype, refine code a bit, add tests, move stuff around, break stuff, rework code, study documentation, refactor a bit more, and then at that point I have enough understanding of the problem to go in at yank out 80% of my code and do it right.
If Copilot gives me working code in the prototype stage, good enough that I can just move on to the next thing, my understanding is never going to be good enough that I can go in and structure everything correctly. It will effectively allow me to skip 90% of my workflow, but pay the price. That's not to say that Copilot can't be extremely helpful during the final steps of development.
If those findings are correct, I can't say that I'm surprised. Bad code is written by poor understanding and Copilot can't have any understanding beyond what you provide it. It may write better code than the average programmer, but the result is no better than the input given. People are extremely focused on "prompt engineering", so why act surprised when a poor "prompt" in VScode yields a poor result?
I'm not sure why you decided that "use copilot" also implies missing out most of your later steps. Who decides to skip all those steps? Presumably you?
My experience is that Copilot is great at getting me started. Sometimes the code is good, sometimes it's mediocre or completely broken.
But it's invaluable at getting me thinking. I wasted a lot more time before I started using it. That might just be my weird brain wiring...
(Edited to sound less narky. I shouldn't post from a mobile device)
I recently tried Copilot out of curiosity and this is my experience too: It helps me getting started, which for me is 99% of the challenge. I know how to solve problems, even complex ones, but for some reason getting started is just so extremely hard, sometimes.
Copilot lets me get started, even if it's wrong sometimes. There have been times where I have been surprised by how it took something I wrote for a server, and presented the correct client-side implementation.
I've used it a few times to describe a problem and let it handle the solution. It's not very good, but I wonder if one should place more blame on PEBCAK and put more time into problem-description. I gave it a few more paragraphs to describe the problem, and eventually I could take it from there. It was still wrong, but enough to get me started. Immensely helpful that way.
Another aspect that I'm wondering about is if it will be able to do more with better documented code. Anyone have experience with that? I've started to write more doxygen comments, and hoping to see if there's a slow shift to more accurate predictions.
This is like writing in general. It's easy to edit crappy text you've written into something better.
It's completely impossible to do it to a text you didn't write at all.
LLM models are pretty good in doing the crappy first version. It might use abandoned packages or old APIs but the skeleton is there. It's not that hard to add some meat on the bones when the structure exists.
Recently I had to parse a pretty crappy XML format (planned by committee) with Go. I just fed the XML to GPT4 and asked it to parse specific values from it. It got like 95% there. I just had to do a few fixes and polish it a bit. Saved me a lot of headache and poking around in documentation.
I’ve circumvented all of this “getting started” trouble with the pomodoro method. It’s simple and I don’t have to real with maybe broken code and it works for everything in my life. Worth a try.
I decided to use ChatGPT to build a clone of Yourls using Django/Python. I gave it specific instructions to not only allow for a custom shortened URL but to track the traffic. It didn’t properly contemplate how to do that in the logic or data model. I had to feed it specific instructions afterwards to get it fixed.
AI tools are akin to having a junior developer working for you. Except they are much much faster.
If you don’t know what you’re doing they just accelerate the pace that you make mistakes.
> AI tools are akin to having a junior developer working for you. Except they are much much faster.
Honestly, this is brilliant. The other day I had to add table name prefixes to a SELECT statement column aliases, since such a feature just doesn't exist for some reason, a bit like:
-- fails because of duplicate column names (e.g. when creating a view)
SELECT
*
FROM table_a
JOIN table_b ON ...
JOIN table_c ON ...
...
-- this would solve my issue, if WITH_PREFIX did exist (or anything like it)
SELECT
table_a.* WITH_PREFIX 'table_a',
table_b.* WITH_PREFIX 'table_b',
table_c.* WITH_PREFIX 'table_c'
FROM table_a
JOIN table_b ON ...
JOIN table_c ON ...
...
So I just gave ChatGPT the schema definitions/query and it wrote out the long list of like 40 columns to be selected for me, like:
SELECT
table_a.id AS 'table_a_id',
table_a.email AS 'table_a_email',
...
table_b.id AS 'table_b_id',
table_b.start_date AS 'table_b_start_date',
...
and so on. I haven't found another good way to automate things like that across different RDBMSes (different queries for system tables that have schema information) and while it's possible with regex or a bit of other types of text manipulation, just describing the problem and getting the output I needed was delightfully simple.
Aside from that, I just use the LLMs as autocomplete, which also encourages me to have good function naming, since often enough that's sufficient information for the LLM to get started with giving me a reasonable starting point. In particular, when it comes to APIs or languages I haven't used a lot, but the problems that I face have been solved by others thousands of times before. I don't even have to use StackOverflow much anymore.
That's why I bought Copilot (though JS/HTML autocomplete in JetBrains IDEs is visually buggy for some reason) and use ChatGPT quite a lot.
LLMs are definitely one of my favorite things, after IntelliSense (and other decent autocomplete), codegen (creating OpenAPI specs from your controllers, or bootstrapping your EF/JPA code from a live dev database schema), as well as model driven development (generating your DB schema migrations/tables from an ER model) and containers (easily packaged, self-contained environments/apps) and smart IDEs (JetBrains ones).
> it wrote out the long list of like 40 columns to be selected for me
It seems like the process of reviewing its generated code to make sure all 40 columns are there and then either re-doing this or manually going through that list whenever the schema changes would take longer than just writing the script? And now you're asking your code reviewers to the same both boring-and-slow manual check on the commit rather than just reviewing the three lines of the script?
I'm a junior, and I have Codeium installed in VSCode. I've found it very distracting most of the times, I don't really understand why so many people uses this kind of assistants.
I find stuff like Phind useful, in the sense that sometimes something happens that I don't understand, and 60% of the times Phind actually helps me to understand the problem. Like finding trivial bugs that I didn't spot because I'm tired, dumb, etc.
On the other hand, with Codeium, I guess it may be useful when you're just churning boilerplate code for some framework, but in my little expericence (writing scrapers and stupid data pipelines & vanilla JS + HTML/CSS) cycling through suggestions is very irritating, specially because many times it doesn't work. Most of the times for stupid reasons, like lacking an argument or something like that, but then it's time you have to spend debugging it.
Another problem I have is that I find there's a common style of JS which consist in daisy-chaining a myriad of methods and anonymous functions, and I really struggle with this. I like to break stuff into lines, name my functions and variables, etc. And so many times code suggestions follow this style. I guess it's what they've been trained on.
Codeium is supposed to learn from this, and sometimes it does, to be fair.
But what I worry the most is that, If I'm a junior and I let this assistants do the code for me ¿How the hell I'm supposed to learn? Because giving Phind context + questions helps me learn or gives me direction to go on find it by myself in the internet, but if the only thing I do is press tab, I don't know how the hell I'm supposed to learn.
I found a couple days ago that many people (including devs) are not using LLMs to get better but it's just a substitute of their effort. Isn't people afraid of this? Not because companies are going to replace you, but it's also a self-reflection issue.
Coding is not the passion of my life, addmitedly, but I like it. I like it because it helps me to make stuff happen and to handle complexity. If you can't understand what's happening you won't be able to make stuff happen and much less to spot when is complexity going to eat you.
I think probably the best use of AI, so far, was when I went into a controller and told it to generate an openAPI spec ... and it got it nearly right. I only had to modify some of the models to reflect reality.
BUT (and this is key), I've hand-written so many API specs in my career that 1) I was able to spot the issues immediately, and 2) I could correct them without any further assistance (refining my prompt would have taken longer than simply fixing the models by hand).
For stuff where you know the domain quite well, it's amazing to watch something get done in 30s that you know would have taken you the entire morning. I get what you're saying though, I wouldn't consider asking the AI to do something I don't know how to do, though I do have many conversations with the AI about what I'm working on. Various things about trade-offs, potential security issues, etc. It's like having a junior engineer who has a PHD in how my language works. It doesn't understand much, but what it does understand, it appears to understand it deeply.
> I wouldn't consider asking the AI to do something I don't know how to do
My experience has been the opposite so far. I benefit much more from such tools when I can easily check if something works correctly and would have to learn/look up a lot of easy and elementary stuff to do it from scratch.
For example, adding to some existing code in a language I don't know and don't have time or need to learn (I guess not many people are often in that situation). I get a lot of hints for what methods and libraries are available, I don't have to know the language syntax, for easy few-line snippets (that do standard things and which I can test separately) the first solution usually just works. This is deliberately passing on an opportunity for deeper and faster learning, which is a bad idea in general, but sometimes the speed trade-off is worth it.
On the other hand, for problems where I know how to solve them, getting some model to generate the solution I want (or at least one I'm happy with) tends to be more work than just doing it myself.
I probably could improve a lot in how I use the available tools. Haven't had that much opportunity yet to play with them...
> Coding is not the passion of my life, addmitedly, but I like it.
It may not be the passion of your life but I haven't seen anybody articulate better (in recent memory) what they want to get out of coding and how they evaluate their tools. Keep at it, don't change and you'll go places, you are definitely on the right path.
The tool and design of the tool matters a lot. I've used Codeium in VSC and GH Copilot in Intellij, and the experience (and quality) of the GH + Intellij paring is much better than Codeium + VSC.
My biggest use for AI assistants has been speeding up test writing and any "this but slightly different" repetitive changes to a code base (which admittedly is also a lot of test writing). At least in intellij + GH, things like, a new parameter that now needs to be accounted for across multiple methods and files is usually a matter of "enter + tab" after I've manually typed out the first two or three variants of what I'm trying to do. Context gives it the rest.
In VSC with Codeium, the AI doesn't seem quite as up to snuff, and the plugin is written in such a way that its suggestions and the keys for accepting them seem to get in the way a lot. It's still helpful for repetitive stuff, but less so for providing a way of accomplishing a given goal.
> Another problem I have is that I find there's a common style of JS which consist in daisy-chaining a myriad of methods and anonymous functions, and I really struggle with this. I like to break stuff into lines, name my functions and variables, etc.
I think your whole comment is excellent but I just wanted to tell you, you're on the right track here. Certain developers, and in particular JS developers, love to chain things together for no benefit other than keeping it on one line. Which is no benefit at all. Keep doing what you're doing and don't let this moronic idiom infect your mind.
Sometimes making something a one-liner is itself a benefit for readability. Especially if you’re used to reading it. But admittedly it’s very easy (and can be tempting) to take it too far…
I know what you mean, but in this situation it shouldn't be a major problem. These would be variables scoped locally and one would hope that this scope would not be more than about a page of code, and hopefully much less. Also - one would hope that local variables are not being re-used!
This is just another coding style. After 1-2 weeks you get used to whatever you're reading. Try it and you'll see.
It's the high-level code that can become an issue (structuring the state of your program, using dependency injection incorrectly, having a convoluted monad transformer stack, putting very specifically typed effects in your Reader etc.). If you make mistakes there, you will struggle to read, write and reuse code, and even then, not all is lost. If there's bad structure you can most often transform it into a good one. When there's no structure, that's a problem.
Seeing .map.filter becomes a quick pattern match. You know what's happening there. It does not matter if it's a named variable or just part of a long
a.map
.filter
.reduce
.map
chain.
I agree, if your goal is to hire a lot of people, then you might want a style that does not strain the pattern matching abilities too much. We can compare which style is the best for that.
Nothing stops you from extracting a sequence from a long chain into a function to reuse it elsewhere.
pipe(
object,
map,
filter,
...
)
Many languages today allow declaring functions inside functions. I'd argue that in that case it's better you declare functions as close as possible to the place where you'll call them, which can be inside another function.
While I can't speak to Codeium, you might want to try Copilot in a more mature codebase that reflects your style of composition.
The amazing part for me with the tech is when it matches my style and preferences - naming things the way I want them, correctly using the method I just wrote in place of repeating itself, etc.
I haven't used it much in blank or small projects, but I'd imagine I'd find it much less ideal if it wasn't so strongly biased towards how I already write code given the surrounding context on which it draws.
Maybe it's worth reevaluating our definition of quality?
In a world where AI can read our codebase, ingest a prompt, and quickly output "correct" if not clean and concise code, and then be able to iterate on code with more prompt, do we need all the same patterns we used to adopt when humans were painstakingly writing every line of code?
This reminds of of the CISC to RISC migration - now that computers are in the loop writing the tedious parts, we don't need to burden our codebase with patterns meant to relieve humans from the tedium.
I find myself, for instance, writing more long form, boring configuration files that once upon a time I would have built some abstraction reduce the boilerplate and verbosity. But now that co-pilot can just auto-complete the next section for me, why bother?
> In a world where AI can read our codebase, ingest a prompt, and quickly output "correct" if not clean and concise code, and then be able to iterate on code with more prompt, do we need all the same patterns we used to adopt when humans were painstakingly writing every line of code?
That world does not exist, so currently this line of thinking is academic.
Perhaps it will exist in the future, but it's far from a certainty if that will come to pass, and unclear on what kind of time-frame. Personally I'm quite skeptical any of us will see it within our lifetimes.
Patterns aren't intended to "relieve humans from the tedium," they're to make the code more intelligible. Code generators create notoriously difficult to understand code. Apparently the current crop of LLMs are no better in that regard.
> find myself, for instance, writing more long form, boring configuration files that once upon a time I would have built some abstraction reduce the boilerplate and verbosity. But now that co-pilot can just auto-complete the next section for me, why bother?
The real issue is not writing the code, but debugging it.
The patterns exist to make the code readable, debuggable, and thus maintainable.
"Don't repeat yourself" is not to save you typing, it's to be able to fix a bug once, rather than hunting through all the code for similar instances and fixing each one, introducing new errors.
If you hope that with enough prompts AI will find and fix its own bugs, I think your level of optimism is enviable :-)
The methodology seems to be: compare commit activity from 2023 to prior years, without any idea of how many involve Copilot. Then interpret those changes with assumptions. That seems a bit shakey.
Also: "The projections for 2024 utilize OpenAI's gpt-4-1106-preview Assistant to run a
quadratic regression on existing data." ...am I to understand they asked gpt to do a regression on the data (4 numbers) rather than running a simple regression tool (sklearn, r, even excel can do this)? Even if done correctly, it is not very compelling when based off of 4 data points and accounting for my first concern.
check out the paper, not just the summary. They explain their methodology. The output has four data points because it’s a summary. The input is … more data than that.
More data, but OP is right on the weaknesses of the study—the author posted here [0] and acknowledged that they can't actually say anything about causality, just that 2023 looked different than 2020.
You can even see a truncated RankWarning in the output in the appendix. Low rank indeed! Numpy is crying out in agony and the authors didn’t notice
They’re using GPT….to write code that does “quadratic regression” on two whole data points…which just extends the slope of each [2022, 2023] line one year further!
I’m sympathetic to the study results since I have seen similar things anecdotally but I agree their data is not really warranting the conclusions they reach. For all we know it could because of the covid hiring spree and subsequent layoffs.
Not even that, the prompt used is "Looking only at the years 2022 and 2023, what would a quadratic regression predict for 2024" as mentioned in the appendix.
So quadratic regression makes it sound all fancy, but with two data points, it's literally just "extend the line straight". So the 2024 prediction is essentially meaningless.
Feels like taking the average of all text on the internet, does result in barely OK output. Very few people are truly “average” in the numerical sense of the word, so you’re going to see better results from most devs compared to the coefficient soup that tries to find the commonalities between the best and worst devs.
Truth be told though, speaking as someone that still does not use LLM tools at work… “just OK” is totally viable for a lot of things. Prototypes yes, products expected to be around a while, maybe not.
I recently had this discussion at work. There's a difference between writing software that will be seen again in a few months and writing software that probably will never be touched again (or at least touching it again will be pretty risky). Identifying which one you're working on is super important.
When you're writing software that is touched fairly often, the "tribal knowledge" of how it works will likely live-on. You can be a little bit clever at times, you don't need to comment as heavily, and your variable names can be a bit wonky.
When you're writing software that is hardly ever touched ... everything needs to be crystal clear. Write lots of comments, explaining the "why" things are the way they are. You want to avoid cleverness except in the name of performance (but then comment how you expect it to work so that when someone comes along to change it, they understand wtf you are doing and the constraints you used to define the code). It's a totally different ballgame.
AI doesn't get this distinction, hell, most programmers don't either.
You're absolutely right. Why are they maligning contractors? Never mind the fact that the comment was insulting - it's not even factual. I'm hired as a contractor to fix broken projects.
Couldn't agree more. Knocking on contractors seems random, out of place, factually inaccurate, and it doesn't really add anything to the substance of this article, so also completely unnecessary to sustain the actual point the article is trying to make. It's just bad writing.
Original research author here. It's exciting to find so many thinking about long-term code quality! The 2023 increase in churned & duplicated (aka copy/pasted) code, alongside the reduction in moved code, was certainly beyond what we expected to find.
We hope it leads dev teams, and AI Assistant builders, to adopt measurement & incentives that promote reused code over newly added code. Especially for those poor teams whose managers think LoC should be a component of performance evaluations (around 1 in 3, according to GH research), the current generation of code assistants make it dangerously easy to hit tab, commit, and seed future tech debt. As Adam Tornhill eloquently put it on Twitter, "the main challenge with AI assisted programming is that it becomes so easy to generate a lot of code that shouldn't have been written in the first place."
That said, our research significance is currently limited in that it does not directly measure what code was AI-authored -- it only charts the correlation between code quality over the last 4 years and the proliferation of AI Assistants. We hope GitHub (or other AI Assistant companies) will consider partnering with us on follow-up research to directly measure code quality differences in code that is "completely AI suggested," "AI suggested with human change," and "written from scratch." We would also like the next iteration of our research to directly measure how bug frequency is changing with AI usage. If anyone has other ideas for what they'd like to see measured, we welcome suggestions! We endeavor to publish a new research paper every ~2 months.
> That said, our research significance is currently limited in that it does not directly measure what code was AI-authored -- it only charts the correlation between code quality over the last 4 years and the proliferation of AI Assistants
So, would a more accurate title for this be "New research shows code quality has declined over the last four years"? Did you do anything to control for other possible explanations, like the changing tech economy?
> We hope it leads dev teams, and AI Assistant builders, to adopt measurement & incentives that promote reused code over newly added code.
imo, this is just replacing one silly measure with another. Code reuse can be powerful within a code base but I've witnessed it cause chaos when it spans code bases. That's to say, it can be both useful and inappropriate/chaotic and the result largely depends on judgement.
I'd rather us start grading developers based on the outcomes of software. For instance, their organizational impact compared to their resource footprint or errors generated by a service that are not derivative of a dependent service/infra. A programmer is responsible for much more than just they code they right; the modern programmer is a purposefully bastardized amalgamation of:
- Quality Engineer / Tester
- Technical Product Manager
- Project Manager
- Programmer
- Performance Engineer
- Infrastructure Engineer
Edit: Not to say anything of your research; I'm glad there are people who care so deeply about code quality. I just think we should be thinking about how to grade a bit differently.
> this is just replacing one silly measure with another
> Not to say anything of your research
The second statement isn't true just because you want it to be true. The first statement renders it untrue.
> I'd rather us start grading developers based on the outcomes of software. For instance, ... errors generated by a service
yeah you should click through and read the whitepaper and not just the summary. The authors talk about similar ideas. For example, from the paper:
> The more Churn becomes commonplace, the greater the risk of mistakes being deployed to production. If the current pattern continues into 2024, more than 7% of all code changes will be reverted within two weeks, double the rate of 2021. Based on this data, we expect to see an increase in Google DORA's "Change Failure Rate" when the “2024 State of Devops” report is released later in the year, contingent on that research using data from AI-assisted developers in 2023.
The authors are describing one measurable signal while openly expressing interest in the topics you're mentioning. The thing is: what's in this paper is a leading indicator, while what you're talking about is a lagging indicator. There's not really a clear hypothesis as to why, for example, increased code churn would reduce the number of production incidents, the mean time to resolution of dealing with incidents, etc.
Thankfully in life no third person gets to dictate what I mean by my own words. There's plenty of good research that comes from studying the silly things, science is filled with things that even an average person would say "duh" or "don't do that". That doesn't make them meaningless. If you disagree that's cool, but I still mean what I said exactly how I said it.
> yeah you should click through and read the whitepaper and not just the summary. The authors talk about similar ideas
I read the article that was linked, which is generally what's expected of me on HN.
> The authors are describing one measurable signal...
I'm aware of the research around this topic, it's something I like reading about and I've read a lot of takes both academic and colloquial. That may be why I put that idea into words.
Maybe, just maybe, in our future interactions you can avoid being so unnecessarily hostile?
That paper benchmarked the performance of the most popular LLMs on refactoring tasks on real-world code. The study found that the AI only delivered functionally correct refactorings in 37% of the cases.
AI-assisted coding is genuinely useful, but we (of course) need to keep skilled humans in the loop and set realistic expectations beyond any marketing hype.
I'm not surprised. AI tools can be great at providing a quick, working example in simple scenarios, but it's just that: a quick(often dirty) working example. But I've seen people taking it as-is and putting it into a production codebase. Sure, a function that iterates over an array, and checks if some item exists - fine. In those cases it will do the job simply because you(it) can't get it wrong. However I had this experience where senior developers were fully invested into using it. And because managers see code just erupting like a volcano, they embrace it and in fact rise their expectation when it comes to delivering a feature: "you could do it in a week before, you should be able to do it in 2 hours now, right?". And on more than one occasion this has backfired massively in everyone's face. The last time me and another dev spent two straight days rewriting and debugging everything cause there was an international exhibition that was about to start and the company was at the front of the line and everyone else simply pushed a ton of code that was 75% AI-generated, completely ignoring the edge cases, which were more than people anticipated.
But probably the most off-putting thing I've experienced is an OKR session with 50 people in it, where a lead dev publicly opened chatgpt, prompted "how do we reduce the number of critical bugs by 30% in the next quarter", chatgpt came up with a generic response, everyone said "perfect", copy-paste that into Jira and call it a day. And I'm just sitting there and wondering if there was something rotten in my breakfast and I'm hallucinating. Unfortunately my breakfast was fine and that really happened. The few times I've tried using those, they were only helpful with dumb, repetitive and mundane tasks. Anything else, you have to rinse and repeat until you get a working solution. And when you read it(for those of us that do), you realize you might have been better off recruiting a freelancer from year one in University to spend a day mashing it up and likely coming up with something better.
But I bet much of those year ones would be doing this exact thing day in and day out until they come up with a solution: Occasionally I will grab my laptop and go work at a cafe on my personal projects for a change and I can't tell you how many times I've seen people just copy pasting stuff from chatgpt and pasting it back into their IDE/editor and calling it a day - students and clearly people who are doing this for a living. Not to mention copilot, that's the de-facto standard at this point.
In fact I had this conversation last year with a guy(developer) who is 20-something years older than me(so mid 50-s): most of the LLM's are trained on stuff that is in the documentation, examples, reddit and stackoverflow. It's only a question of time until the content found in those exact locations where the training data is pulled from will become more and more AI-generated, models will be re-trained on those and eventually shit hitting the fan. I don't think we are too far off from this event.
What you said gave me an idea of how to really make AI useful at work! :-)
Every quarter HR requires that we write our "perspective" for the next quarter; what we are going to do and how we are going to improve ourselves. It's purely bureaucratic exercise, has no meaning and no impact on anything anybody does, but on an off-chance somebody reads it I cannot just fill it with nonsense. Writing something resembling sensible, in a stilted language required, always makes me struggle much more than writing code or something with meaning.
Strange that I haven't though about using an LLM or this before; seems like a perfect job for it.
Yeah, admittedly I've used chatgpt for such things. That and resignation letters. Funny enough one of the team leads sent me a message over chat when I was leaving my old jobs, saying that he'd like to stay in touch with me cause we have similar interests and he really values me as a developer. He also used chatgpt for that message. How do I know? His English was insanely limited and couldn't have come up with such sophisticated words. Also grammar was 10/10. I laughed it off and moved on though.
But yeah, I abhor bureaucracy and trivial bs like the one you mentioned so I'm more than happy to outsource this issue to someone/something else.
Sometimes less dry code can actually be easier to read and understand at the point of usage than dry code that has been more highly abstracted and requires grokking a sort of quasi DSL that defines the abstraction. Assuming that AI contributions will only increase, if a codebase were almost completely written by AI perhaps the benefits of DRY would diminish vs on the spot readability by humans only trying to understand the code and not personally add to it
Well as with anything, DSLs are subject to the rules of good design. A really well-designed DSL (such as SQL) takes on a life of its own and becomes incredibly ubiquitous and useful to know in its own right. Many other DSLs are totally unknown, not worth learning, and serve as barriers to code understanding.
I don’t know of too many people who would advocate replacing SQL with hand-written C manipulating B-trees and hash tables. Similarly, it’s pretty rare that you want to hand-roll a parser in C over using a parser generator DSL or even something like regex.
Some people are thinking hard about a complex, nuanced topic of which we have very little past experience to draw on. I'm glad the conclusion is so self-evident to you. I must be a little slow.
ML/LLMs are nuanced and complex topics as they are the inner workings of automation. People using ML/LLM to get around having to write code already understood well enough isn’t a complex nuanced topic because it is the outer workings of automation and has been studied in other fields quite extensively. No one should be surprised at the trend of lazier development from wide adoption of automation tools.
When I look into the future, and I know that I really can't, one thing I really believe in is that there will be a shift in how quality will be perceived.
With all things around me there is a sense that technology is to be a saviour for many very important things - ev's, medicine, it, finance etc.
At the same time it is more and more clear to me that technology is used primarily to grow a market, government, country etc. But it does that by layering on top of already leaking abstractions. It's like solving a problem by only trying to solvent be its symptoms.
Quality has a sense of slowness to it which I believe will be a necessary feat, both due to the fact that curing symptoms will fall short and because I believe that the human species simply cannot cope with the challenges by constantly applying more abstractions.
The notion about going faster is wrong to me, mostly because I as a human being do not believe that quality is done by not understanding the fundamentals of a challenge, and by trying to solve it for superficial gains is simply unintelligent.
LLMs is a disaster to our field because it caters to the average human fallacy of wanting to reach a goal but without putting in the real work to do so.
The real work is of course to understand what it is that you are really trying to solve with applying assumptions about correctness.
Luckily not all of us is trying to move faster but instead we are sharpening our minds and tools while we keep re-learing the fundamentals and applying thoughtful decisions in hope to make quality that will stand the test of time.
Presumably, this was supposed to show how parent's arguments can equally well argue for the opposite or for something nonsensical. However, it seems incoherent to me (in a way that parent does not).
I don't even know what it's arguing for... I guess something literally about sewing?
It’s a style-transfer of OPs post onto Luddites. This same tired argument in 3 parts gets made every time a potentially job-destroying technology gets introduced or discussed.
Since nobody has made the concluding argument yet, I’ll supply it; the Luddites were correct that it (mechanization) would destroy their lucrative profession, they missed the degree to which it would enable many other lucrative professions, and they were right that many Luddites would not make the transition.
Society was net better off but there were definitely losers in the transition.
I think he’s making a point about mechanization inevitably causing both accelerated progress and also, in some ways, reduced quality. There’s a challenge to grapple with there for traditional craftsmanship.
> The real work is of course to understand what it is that you are really trying to solve with applying assumptions about correctness.
In how far do you think LLMs stand in the way of that?
My experience has been very much the opposite: Instead of holding the hard part of the process up by digging through messy apis or libraries, LLMs (at least, in their current form but I suspect that this will theoretically simply remain true) make it painfully obvious when my thinking about a task of any significance is not sound.
To get anywhere with a LLM, you need to write. To write, you have to think.
Very often I find the most beneficial part of the LLM-coding-process is a blended chat backlog that I can refer back to, consisting of me carefully phrasing what it is that I want to do, being poked by a LLM, and me through this process finding gaps and clarifying my thoughts at the same time.
I find this tremendously useful, specially when shaping the app early, to keep track of what I thought needed to be done and then later being able to reconsider if that is actually still the case.
I don't get so much from going over previous conversations, but needing to articulate a problem well enough to ask chatgpt a question is extremely useful. Far moreso than coming up with search phrases, I find.
This is how I’ve been most successful so far at using LLMs. They help me poke as you say at the problem until a satisfying solution appears or until I have enough info to know what to look for.
But that is a terrible way of dealing with hard problems.
I recommend Rich Hickey's "Hammock Driven Development" talk. You don't solve hard problems by poking at it repeatedly until something works, that is the recipe for terrible code and abstractions. You instead take a step back from your computer and digest it until you come with a well-understood solution.
This approach is what separates the experienced engineer from the junior. Code is the least of your problems.
I don't know what experience you draw from, and what is lost in assumption here (on both sides, "hard problems" is open to a lot of interpretation), but the entire premise behind agile is, that thinking a solution up is not how software development works. You chunk things up, you iterate, you stay flexible, precisely because sitting down and thinking hard is not realistically working for a sizeable, messy, real world business solution.
I draw from my 17 years of experience in the field, but I am well aware that people work differently.
There are the John Carmack types, whose output depends on how much time on they spend at keyboard, and the Rich Hickey types, whose output depend on how much time they spend on a hammock with their eyes closed (or under the shower in my case). I am afraid I am of the latter type. My best solutions are found away from the keyboard, as I have learned to simply depend on my subconscious to process and digest them while I'm doing other things.
Check out that talk still, it has deep insight on how the human brain operates.
Poke around as in, at least in my case, discussing and throwing ideas back and forth. this is not incompatible with hammock-driven development (big fan here). It's however a good way of "discussing" ideas and solutions, and has been working well for me as part of an overall strategy to solve my problems. (30 years of experience in the industry if we're counting)
> LLMs is a disaster to our field because it caters to the average human fallacy of wanting to reach a goal but without putting in the real work to do so.
It's a tool. It doesn't make sense to blame the tool. Is it the screwdriver's fault it gets used as a hammer? Or a murder weapon?
Used intelligently Copilot & Co can help. It can handle the boilerplate, the mundane and free up the human element to focus on the heavy lifting.
All that aside, it's early days. It's too early to pass judgement. And it seems unlikely it's going to go away.
There is an interview with the great jazz pianist Bill Evans (conducted by his brother) in which he muses that most amateur musicians make the mistake of overplaying. They go out to the club and hear a professional and they come home and try to approximate what the professional does. What they end up with is a confused mess with no foundation. He insists that you have to learn to be satisfied with doing the simple things and gradually building up a stronger foundation.
I think his insight applies nearly as well to using code generated by an ai.
When I look into the future, and I know that I really can't, one thing I really believe in is that there will be a shift in how quality will be perceived.
IKEA furniture is a great example of this. I build my own furniture and being around it is a much much nicer thing than some piece of cardboard from IKEA.but it seems like cost, soeed an convenience are the most important thing in peoples minds.
But the tradeoff of cost and convenience vs quality is everywhere in life. Most people (including me) do not have the time, money, nor (in my case most importantly) workspace to build their own furniture. IKEA and other budget furnishing companies are a perfect solution for people in this situation and I can buy a handmade piece of furniture if I ever feel that something is not up to quality.
Very well put, like what's the point of doing work of the art if the art don't accompany artist story of struggle, mental experience and creative expression while reaching to end form of his art. What ai model does is rob us all inate experience and give us only cream of end result, it's is like watching porn instead of forming real relationship with person to win the sex.
Quality is best thought of as a process, and that process got pushed out of the SDLC by Agile process and its metric of velocity. The use of LLM-generated code to further increase velocity in the absence of quality process is an obvious result.
I am worried that AI assisted code will be a competitive advantage so that the downsides will not be addressed until there are serious failures in critical code. Boeing but for power plants, for example.
There was already a backlash against DRY code occurring before "AI" assistants hit the market, sadly. It was a growing movement when I was using Twitter in 2019-2022.
Some younger developers have a very different attitude to code than what I was brought up with. They have immense disdain for the Gang of Four and their design patterns (probably without realising that their favourite frameworks are packed to the gills with these very patterns). They speak snidely about principles like DRY and especially SOLID. And on places like Twitter the more snide and contrarian you can be, the more engagement you'll get. Very disturbing stuff.
DRY is mostly bad. You couple a lot of things together by making them all share the same code and then a small change to that code breaks various unrelated components.
For sure, there are obvious egregious examples of repeating one's self. There are also many more examples of developers programming by unexamined catch phrases.
God I would love that. Are you kidding? I've always hoped I'd get to inherit one. A codebase where each component "vendors" all its dependencies so you can fearlessly make changes and not affect anything else.
You're describing heaven for a maintenance programmer -- I'm only looking at the codebase because there's a bug in some component, I likely even have a stack trace. If can just read what that bit of code does top to bottom, fix the error in just that component, write a test and ship I'll send the original author chocolates.
Sure, that would fix the bug immediately in front of you, but what if the same bug exists in several of the other copies of the dependency? Are you going to be able to track all of the occurrences down? I think that's the big tradeoff.
I'll take that every time, to zeroth order if the bug occurs other places it's free to err other places and I'll deal with it. But if it doesn't then does it matter? If we're experiencing a bug in a specific code-path is the right move to make a change that fixes it in one codepath but potentially changes the currently working n other codepaths that depend on it? If it weren't DRY'd would you bother changing the others? Probably not.
To first order :vimgrep is totally mechanical and brain-off repeat fix is super easy.
This is the kind of thinking that leads to unmaintainable AbstractFactoriesFactory classes.
Sometimes allowing repeat code is good because two functions might drift away from common functionality in the future which would require a major rework of whatever abstraction you put over them to get them under the same roof.
Even in simpler cases it can be problematic. For example, refactoring a tiny bit of repetitive code into a separate function might seem like a good idea. But, over time, things diverge and now additional parameters and branches have to be introduced to support. A small amount of repeated code is optimal.
Maybe because SOLID is overrated / overhyped marketing term which somehow made it to academy despite being far from actual computer science / software engineering fundamentals?
We just cant stand acting as if that random list of principles created by Java's OOP mind was some source of truth for software modeling.
We're just tired of seeing bilionth discussion about how to understand SOLID
You probably don't see people arguing against CAP theorem because it is not some arbitrary collection of ideas (not even fully authored by SOLID author) which composes fancy mnemonic
>There was already a backlash against DRY code occurring before "AI" assistants hit the market, sadly.
As everything else - DRY can be abused too and people backlash against acting as those things were flawless when they arent.
I’ve noticed similar trends. After a while, I started to realise that a lot of the critics don’t really understand the principle they’re criticising.
For example, take DRY. The important principle was never really about repeating code. It was about repeating ideas. For any given concept in your system, ideally there should be a single source of truth, and therefore a single place you need to understand or change if you’re working with that concept. It’s true that this means copying and pasting non-trivial amounts of code instead of creating a meaningful abstraction is often a bad idea. But it is also a warning that any time you do repeat an idea, you now have an ongoing liability because you need to keep those different representations in sync. That could refer to database migrations that define your schema and separate ORM class definitions, or an API you define in your back-end code and a client for that API you define in your front-end code, or a retained mode UI where you have a current value in some form field that corresponds to a specific value in your internal application state, or some invariant in your data model that can be represented in both types and unit tests.
People who object to combining duplicate or near-duplicate code that represents different ideas but happens to have a similar implementation at the time, on the basis that it’s a maintenance hazard for later on, aren’t wrong. They’re just objecting to a straw man that was never really the point of DRY in the first place, but has been treated as if it were due to some kind of cargo cult/gaslighting effect.
The question I have now is where and when in our industry do we expect new developers to learn these principles so they do understand them properly? Some people have a formal background in CS or the like, but not everyone does, and in any case it’s not necessarily the role of an academic CS course to teach a lot of practical software development skills. I had a discussion the other day about how when I was starting out, the senior developers would give real, substantial training to the juniors to help us learn and understand these principles, but with the job-hopping culture today and the resulting general aversion to hiring juniors as a long-term investment, that just doesn’t seem to happen much any more. There are formal courses that cost a lot by personal standards but almost nothing by business standards, but there must be a tiny proportion of new developers who actually get sent on them by their employers. There are a few books worth reading, but what 20-something in 2024 wants to deal with presentation as antiquated as ink on sliced bits of tree? I suspect a lot of what today’s up and coming developers learn about these ideas comes from sources like blogs and YouTube videos, where again there is some great material out there, but as ever the problem is finding it among all the poorly understood and dubiously presented dross.
And then we wonder why tools come along that seem like magic, producing a dozen lines of code in a heartbeat that seem to mostly work, and young developers think they’re great even while having little idea of all the deeper things that may be wrong with that code. It’s not really surprising, and I’m not sure it’s really anyone’s fault, but it’s definitely a problem and I wish I knew what we should do about it.
The backlash isn't against proper DRY (concerned with single source of truth) but fake DRY (concerned with syntactically-similiar code).
Immense disdain does accurately describe how I feel towards whatever it is happens in corporate codebases. No, creating layers upon layers of indirection via classes is not ok, no matter what your SOLID guru tells you. Best practices, DRY, and SOLID are just excuses.
> We further find that the percentage of 'added code' and 'copy/pasted code' is increasing in proportion to 'updated,' 'deleted,' and 'moved 'code. In this regard, AI-generated code resembles an itinerant contributor, prone to violate the DRY-ness [don't repeat yourself] of the repos visited.
Couldn’t this equally be explained by the cost of refactoring becoming incredibly cheap? If most of your code is generated, and you don’t have to make the investment of hand crafting everything, aren’t you naturally going to be regularly replacing vast tracts? Obviously the trend may have implications, but in large part aren’t we just seeing the impact of code cheapening? Serious question.
Those who can't code better than an LLM will welcome it, those who can will abhor it. Unfortunately it seems there are far more of the former than the latter, and they aren't going to get better either.
Progress comes from human intellect, mediocrity comes from regurgitation.
338 comments
[ 2.2 ms ] story [ 309 ms ] threadI think this is related to the productivity wage gap. I’m being paid lower than I was in 2018 adjusted for inflation and I’m far more capable, producing far more value for my employer. As my tools get more valuable my labor gets proportionally less valuable. (A certain historical figure predicted this end state of capitalism).
Here’s another take: I don’t give a fuck about the code quality at my company. I’m not paid extra to write good code. My bonus is based on basically nothing I can influence. If the whole company goes out of business on Monday I’ll enjoy my long weekend.
To build a house, it used to be necessary to hire experienced craftsmen. Now most of it is delivered pre-fabricated to the building site and laborers hired for the day nail it together.
Cars used to be hand built by machinists and metalworkers. Now laborers tighten a bolt as cars roll by on an assembly line. Or a robot does it.
Computer coding is going the same way. We lived and worked through the craftsman stage. Now we're becoming laborers.
And as we zoom out and think more farther into the future, I see it getting much worse. If AI is really doing all the "hard stuff", then the general human incentive to learn and do them at all quickly treads to 0. This isn't going to be everyone, I think some people will absolutely become "10x developers" or the equivalent for other domains. But all this will result in is more inequity, in my naïve view. The universe has a fundamental property that things move from higher energy states into lower ones. From a human POV, I think you could apply a similar idea, if the need to be smart quickly recedes, then in general we may degrade unaugmented human collective intelligence over time.
I don't know, maybe we'll figure something out to make it much easier to learn and ingest new concepts, but it seems more and more to me that the high obstacles for human brains learning things (with poor memory) is too big a bottleneck to overcome any time soon.
Many other mass produced products we buy today are clearly lower quality than ones crafted by artisans 50 years ago, and yet they do what we need them to do at a fraction of the cost.
I think that's a sweeping generalization. For example, it's much better to have a bunch of food that's mostly garbage than to have a famine where all food is super high quality. There were points in history where this choice was made (obviously unconsciously because the choice is too obvious to even think about). Other examples abound.
Technology often makes things much cheaper while reducing quality. Sometimes that's bad, sometimes it's great.
Like any idea about squishy human brains and its products it remains to be seen and can’t be as easily proven as for example research in Physics.
I would say current research has a probability of 50% of being correct at best.
Not without a solid result establishing so. But good luck with this, you'd probably need to compare our world with internet and a similar enough world but without it, which does not exist.
In any case, internet can't be reduced to "tiktok et al"
https://blogs.ubc.ca/etec540sept13/2013/09/29/socrates-writi...
This coupled with an broad interest in just about anything means I can often help people far outside my specific field, usually rather swiftly as well.
In a conversation with Phaedrus, Socrates worries that writing could impair human memory, as people depend on written words instead of recalling information. [0]
[0]: https://en.wikipedia.org/wiki/Phaedrus_(dialogue)#Discussion...
The question is will we one day have tools so powerful, that the human is vestigial, and tool (without human) is just as powerful and cheaper than tool (+ human)?
AI might replace it.
Writing "replaced" some forms of intellectual effort.
And it's yet to be seen how AI will play out.
I find that it is true that new mediums and new technologies for language have a numbing effect on certain senses and an amplification of others.
Writing is beneficial in one regard but does have an impact on memory. Epic poems of great length were memorized in their entirety, a skill that would be a lot easier to develop in a world without writing.
https://ourworldindata.org/literacy
In already developed countries the Flynn effect seems to be reversing, ie IQ is leveling off or even dropping.
It's basically impossible to improve an adult's g-factor, for example. For children, things like nutrition and hygiene (e.g. no parasites) play a big role. But the kinds of things a human learns or tools they use doesn't significantly affect G-factor.
Qanon was purely a creation of the internet. Now go take a look at how many people believe one, many or all of the various Qanon alternative facts.
They don't prove that it makes people dumber. You have to quantify and qualify people and to define "dumb".
Maybe people are not actually dumber because of the internet, but the internet is very good at spreading ideas, including incredibly dumb ones, especially (because of how human beings work) those likely to cause feeling of outrage.
Maybe people are not dumber, just too defenseless against the scale of bullshit they are faced with because of the internet. Maybe internet is an incredibly good tool, but strongly requires good learning / training of critical thinking and there's not enough of this yet.
It's hard to argue that nowadays people believe more crazy stuff than before the internet was invented. (It's very easy, of course, to claim this, as many like to do.)
Taking into account that such blunt statements always hide a lot of nuances, this seems to capture the reality.
None of this really happened. I mostly use editor tools to automate away tedium that doesn't matter; I type "log.Inf<TAB>" and it adds "src/internal/log" as an import at the top of the file and types the o and open parenthesis for me. I have not forgotten how to do that myself, but it saves me a couple seconds. Calculators didn't really make people dumber, though I have to say that a lot of arithmetic I learned in elementary school did make me dumber ("touch math" was the killer for me; slows me down every time I do arithmetic in my head; I need some brainwashing program that deletes that from my brain). Spell check didn't make people worse at spelling; spelling things wrong still has a penalty (C-w to kill the last word and spell it correctly), so the incentive is to still to lurn how to spel wrds rite. Zoom meetings didn't ruin the corporate world; I personally found them very helpful for memorizing people's names with a high degree of certainty. You see it under their face for 40 minutes at a time, and you learn it fast. In real life, probably takes me a few weeks for people I only see once a week. So, honestly a benefit for me.
The current state of AI seems very similar to these technologies. I did a copilot free trial (and didn't renew it). With the free trial I think there were a couple things it was good at. One time I wanted a CPU profile for my app, so I just asked the AI to type it in. Open a file, check for an error, start profiling to the file, stop profiling, close the file. Would have taken me a minute or two to type in, but Copilot typed it in instantly. I also did something like "do the same as the function above, but for a YAML file instead of a JSON file". Again, super trivial to type in that code, it's really only one line, but the AI can handle that just fine in an instant. I don't really think it's more than slightly smarter IntelliSense, but without any access to the compiler toolchain, so it can sometimes just hallucinate stuff that doesn't exist.
I've found this to be kind of an interesting way to proofread documents and design APIs. Give ChatGPT a document you're working on, and then ask questions about it. If it gets the wrong answer, then your doc is no good! Similarly, ask it how to write some code using your new API (without showing it the API). Whatever it writes should be the surface area of your API. This avenue is pretty interesting to me, and it's not replacing humans, it's just a smarter rubber duckie.
Overall, I think we're in a little bit of a hype phase with AI. I look at it kind of like a dog that has read every book, if such a thing were possible. Pretty smart, but not quite human yet. As a result, it's not going to do well in the areas where people really want to apply it; customer service, insurance claims, loan underwriting, etc. But it is pretty good for asking questions like "does my document make sense" or "please find some boilerplate to cut-n-paste here, I am going to delete this before checking in anyway". Also not too bad at slightly modifying copyrighted images ;)
Every tool has this trade-off, and the existence of skills that will be lost is not evidence that the tool will do more harm than good. I don't think anyone here would argue that Socrates was correct that writing would be the end of memory and wisdom.
> To [Thamus] came Theuth and showed his inventions ... when they came to letters, "This,* said Theuth, "will make the Egyptians wiser and give them better memories; it is a specific both for the memory and for the wit."
> Thamus replied: "O most ingenious Theuth, the parent or inventor of an art is not always the best judge of the utility or inutility of his own inventions to the users of them. And in this instance, you who are the father of letters, from a paternal love of your own children have been led to attribute to them a quality which they cannot have; for this discovery of yours will create forgetfulness in the learners' souls, because they will not use their memories; they will trust to the external written characters and not remember of themselves. The specific which you have discovered is an aid not to memory, but to reminiscence, and you give your disciples not truth, but only the semblance of truth; they will be hearers of many things and will have learned nothing; they will appear to be omniscient and will generally know nothing; they will be tiresome company, having the show of wisdom without the reality."
https://www.gutenberg.org/files/1636/1636-h/1636-h.htm
If Copilot gives me working code in the prototype stage, good enough that I can just move on to the next thing, my understanding is never going to be good enough that I can go in and structure everything correctly. It will effectively allow me to skip 90% of my workflow, but pay the price. That's not to say that Copilot can't be extremely helpful during the final steps of development.
If those findings are correct, I can't say that I'm surprised. Bad code is written by poor understanding and Copilot can't have any understanding beyond what you provide it. It may write better code than the average programmer, but the result is no better than the input given. People are extremely focused on "prompt engineering", so why act surprised when a poor "prompt" in VScode yields a poor result?
My experience is that Copilot is great at getting me started. Sometimes the code is good, sometimes it's mediocre or completely broken.
But it's invaluable at getting me thinking. I wasted a lot more time before I started using it. That might just be my weird brain wiring...
(Edited to sound less narky. I shouldn't post from a mobile device)
Copilot lets me get started, even if it's wrong sometimes. There have been times where I have been surprised by how it took something I wrote for a server, and presented the correct client-side implementation.
I've used it a few times to describe a problem and let it handle the solution. It's not very good, but I wonder if one should place more blame on PEBCAK and put more time into problem-description. I gave it a few more paragraphs to describe the problem, and eventually I could take it from there. It was still wrong, but enough to get me started. Immensely helpful that way.
Another aspect that I'm wondering about is if it will be able to do more with better documented code. Anyone have experience with that? I've started to write more doxygen comments, and hoping to see if there's a slow shift to more accurate predictions.
This is like writing in general. It's easy to edit crappy text you've written into something better.
It's completely impossible to do it to a text you didn't write at all.
LLM models are pretty good in doing the crappy first version. It might use abandoned packages or old APIs but the skeleton is there. It's not that hard to add some meat on the bones when the structure exists.
Recently I had to parse a pretty crappy XML format (planned by committee) with Go. I just fed the XML to GPT4 and asked it to parse specific values from it. It got like 95% there. I just had to do a few fixes and polish it a bit. Saved me a lot of headache and poking around in documentation.
And the thing about AI is that their negative impact is clearly visible above the noise.
AI tools are akin to having a junior developer working for you. Except they are much much faster.
If you don’t know what you’re doing they just accelerate the pace that you make mistakes.
Honestly, this is brilliant. The other day I had to add table name prefixes to a SELECT statement column aliases, since such a feature just doesn't exist for some reason, a bit like:
So I just gave ChatGPT the schema definitions/query and it wrote out the long list of like 40 columns to be selected for me, like: and so on. I haven't found another good way to automate things like that across different RDBMSes (different queries for system tables that have schema information) and while it's possible with regex or a bit of other types of text manipulation, just describing the problem and getting the output I needed was delightfully simple.Aside from that, I just use the LLMs as autocomplete, which also encourages me to have good function naming, since often enough that's sufficient information for the LLM to get started with giving me a reasonable starting point. In particular, when it comes to APIs or languages I haven't used a lot, but the problems that I face have been solved by others thousands of times before. I don't even have to use StackOverflow much anymore.
That's why I bought Copilot (though JS/HTML autocomplete in JetBrains IDEs is visually buggy for some reason) and use ChatGPT quite a lot.
LLMs are definitely one of my favorite things, after IntelliSense (and other decent autocomplete), codegen (creating OpenAPI specs from your controllers, or bootstrapping your EF/JPA code from a live dev database schema), as well as model driven development (generating your DB schema migrations/tables from an ER model) and containers (easily packaged, self-contained environments/apps) and smart IDEs (JetBrains ones).
It seems like the process of reviewing its generated code to make sure all 40 columns are there and then either re-doing this or manually going through that list whenever the schema changes would take longer than just writing the script? And now you're asking your code reviewers to the same both boring-and-slow manual check on the commit rather than just reviewing the three lines of the script?
100%
And if you know what you are doing, they will accelerate the way you're building stuff.
I think companies will want more code faster to the extent that fewer people will emerge from the churn really knowing what they are doing.
I find stuff like Phind useful, in the sense that sometimes something happens that I don't understand, and 60% of the times Phind actually helps me to understand the problem. Like finding trivial bugs that I didn't spot because I'm tired, dumb, etc.
On the other hand, with Codeium, I guess it may be useful when you're just churning boilerplate code for some framework, but in my little expericence (writing scrapers and stupid data pipelines & vanilla JS + HTML/CSS) cycling through suggestions is very irritating, specially because many times it doesn't work. Most of the times for stupid reasons, like lacking an argument or something like that, but then it's time you have to spend debugging it.
Another problem I have is that I find there's a common style of JS which consist in daisy-chaining a myriad of methods and anonymous functions, and I really struggle with this. I like to break stuff into lines, name my functions and variables, etc. And so many times code suggestions follow this style. I guess it's what they've been trained on.
Codeium is supposed to learn from this, and sometimes it does, to be fair.
But what I worry the most is that, If I'm a junior and I let this assistants do the code for me ¿How the hell I'm supposed to learn? Because giving Phind context + questions helps me learn or gives me direction to go on find it by myself in the internet, but if the only thing I do is press tab, I don't know how the hell I'm supposed to learn.
I found a couple days ago that many people (including devs) are not using LLMs to get better but it's just a substitute of their effort. Isn't people afraid of this? Not because companies are going to replace you, but it's also a self-reflection issue.
Coding is not the passion of my life, addmitedly, but I like it. I like it because it helps me to make stuff happen and to handle complexity. If you can't understand what's happening you won't be able to make stuff happen and much less to spot when is complexity going to eat you.
BUT (and this is key), I've hand-written so many API specs in my career that 1) I was able to spot the issues immediately, and 2) I could correct them without any further assistance (refining my prompt would have taken longer than simply fixing the models by hand).
For stuff where you know the domain quite well, it's amazing to watch something get done in 30s that you know would have taken you the entire morning. I get what you're saying though, I wouldn't consider asking the AI to do something I don't know how to do, though I do have many conversations with the AI about what I'm working on. Various things about trade-offs, potential security issues, etc. It's like having a junior engineer who has a PHD in how my language works. It doesn't understand much, but what it does understand, it appears to understand it deeply.
My experience has been the opposite so far. I benefit much more from such tools when I can easily check if something works correctly and would have to learn/look up a lot of easy and elementary stuff to do it from scratch.
For example, adding to some existing code in a language I don't know and don't have time or need to learn (I guess not many people are often in that situation). I get a lot of hints for what methods and libraries are available, I don't have to know the language syntax, for easy few-line snippets (that do standard things and which I can test separately) the first solution usually just works. This is deliberately passing on an opportunity for deeper and faster learning, which is a bad idea in general, but sometimes the speed trade-off is worth it.
On the other hand, for problems where I know how to solve them, getting some model to generate the solution I want (or at least one I'm happy with) tends to be more work than just doing it myself.
I probably could improve a lot in how I use the available tools. Haven't had that much opportunity yet to play with them...
It may not be the passion of your life but I haven't seen anybody articulate better (in recent memory) what they want to get out of coding and how they evaluate their tools. Keep at it, don't change and you'll go places, you are definitely on the right path.
My biggest use for AI assistants has been speeding up test writing and any "this but slightly different" repetitive changes to a code base (which admittedly is also a lot of test writing). At least in intellij + GH, things like, a new parameter that now needs to be accounted for across multiple methods and files is usually a matter of "enter + tab" after I've manually typed out the first two or three variants of what I'm trying to do. Context gives it the rest.
In VSC with Codeium, the AI doesn't seem quite as up to snuff, and the plugin is written in such a way that its suggestions and the keys for accepting them seem to get in the way a lot. It's still helpful for repetitive stuff, but less so for providing a way of accomplishing a given goal.
I think your whole comment is excellent but I just wanted to tell you, you're on the right track here. Certain developers, and in particular JS developers, love to chain things together for no benefit other than keeping it on one line. Which is no benefit at all. Keep doing what you're doing and don't let this moronic idiom infect your mind.
It's the high-level code that can become an issue (structuring the state of your program, using dependency injection incorrectly, having a convoluted monad transformer stack, putting very specifically typed effects in your Reader etc.). If you make mistakes there, you will struggle to read, write and reuse code, and even then, not all is lost. If there's bad structure you can most often transform it into a good one. When there's no structure, that's a problem.
Seeing .map.filter becomes a quick pattern match. You know what's happening there. It does not matter if it's a named variable or just part of a long
chain.I agree, if your goal is to hire a lot of people, then you might want a style that does not strain the pattern matching abilities too much. We can compare which style is the best for that.
Nothing stops you from extracting a sequence from a long chain into a function to reuse it elsewhere.
Many languages today allow declaring functions inside functions. I'd argue that in that case it's better you declare functions as close as possible to the place where you'll call them, which can be inside another function.The amazing part for me with the tech is when it matches my style and preferences - naming things the way I want them, correctly using the method I just wrote in place of repeating itself, etc.
I haven't used it much in blank or small projects, but I'd imagine I'd find it much less ideal if it wasn't so strongly biased towards how I already write code given the surrounding context on which it draws.
In a world where AI can read our codebase, ingest a prompt, and quickly output "correct" if not clean and concise code, and then be able to iterate on code with more prompt, do we need all the same patterns we used to adopt when humans were painstakingly writing every line of code?
This reminds of of the CISC to RISC migration - now that computers are in the loop writing the tedious parts, we don't need to burden our codebase with patterns meant to relieve humans from the tedium.
I find myself, for instance, writing more long form, boring configuration files that once upon a time I would have built some abstraction reduce the boilerplate and verbosity. But now that co-pilot can just auto-complete the next section for me, why bother?
That world does not exist, so currently this line of thinking is academic.
Perhaps it will exist in the future, but it's far from a certainty if that will come to pass, and unclear on what kind of time-frame. Personally I'm quite skeptical any of us will see it within our lifetimes.
> find myself, for instance, writing more long form, boring configuration files that once upon a time I would have built some abstraction reduce the boilerplate and verbosity. But now that co-pilot can just auto-complete the next section for me, why bother?
Again, so humans can understand it more easily.
Also: "The projections for 2024 utilize OpenAI's gpt-4-1106-preview Assistant to run a quadratic regression on existing data." ...am I to understand they asked gpt to do a regression on the data (4 numbers) rather than running a simple regression tool (sklearn, r, even excel can do this)? Even if done correctly, it is not very compelling when based off of 4 data points and accounting for my first concern.
[0] https://news.ycombinator.com/item?id=39168841
They’re using GPT….to write code that does “quadratic regression” on two whole data points…which just extends the slope of each [2022, 2023] line one year further!
So quadratic regression makes it sound all fancy, but with two data points, it's literally just "extend the line straight". So the 2024 prediction is essentially meaningless.
Truth be told though, speaking as someone that still does not use LLM tools at work… “just OK” is totally viable for a lot of things. Prototypes yes, products expected to be around a while, maybe not.
When you're writing software that is touched fairly often, the "tribal knowledge" of how it works will likely live-on. You can be a little bit clever at times, you don't need to comment as heavily, and your variable names can be a bit wonky.
When you're writing software that is hardly ever touched ... everything needs to be crystal clear. Write lots of comments, explaining the "why" things are the way they are. You want to avoid cleverness except in the name of performance (but then comment how you expect it to work so that when someone comes along to change it, they understand wtf you are doing and the constraints you used to define the code). It's a totally different ballgame.
AI doesn't get this distinction, hell, most programmers don't either.
We hope it leads dev teams, and AI Assistant builders, to adopt measurement & incentives that promote reused code over newly added code. Especially for those poor teams whose managers think LoC should be a component of performance evaluations (around 1 in 3, according to GH research), the current generation of code assistants make it dangerously easy to hit tab, commit, and seed future tech debt. As Adam Tornhill eloquently put it on Twitter, "the main challenge with AI assisted programming is that it becomes so easy to generate a lot of code that shouldn't have been written in the first place."
That said, our research significance is currently limited in that it does not directly measure what code was AI-authored -- it only charts the correlation between code quality over the last 4 years and the proliferation of AI Assistants. We hope GitHub (or other AI Assistant companies) will consider partnering with us on follow-up research to directly measure code quality differences in code that is "completely AI suggested," "AI suggested with human change," and "written from scratch." We would also like the next iteration of our research to directly measure how bug frequency is changing with AI usage. If anyone has other ideas for what they'd like to see measured, we welcome suggestions! We endeavor to publish a new research paper every ~2 months.
So, would a more accurate title for this be "New research shows code quality has declined over the last four years"? Did you do anything to control for other possible explanations, like the changing tech economy?
imo, this is just replacing one silly measure with another. Code reuse can be powerful within a code base but I've witnessed it cause chaos when it spans code bases. That's to say, it can be both useful and inappropriate/chaotic and the result largely depends on judgement.
I'd rather us start grading developers based on the outcomes of software. For instance, their organizational impact compared to their resource footprint or errors generated by a service that are not derivative of a dependent service/infra. A programmer is responsible for much more than just they code they right; the modern programmer is a purposefully bastardized amalgamation of:
- Quality Engineer / Tester
- Technical Product Manager
- Project Manager
- Programmer
- Performance Engineer
- Infrastructure Engineer
Edit: Not to say anything of your research; I'm glad there are people who care so deeply about code quality. I just think we should be thinking about how to grade a bit differently.
> Not to say anything of your research
The second statement isn't true just because you want it to be true. The first statement renders it untrue.
> I'd rather us start grading developers based on the outcomes of software. For instance, ... errors generated by a service
yeah you should click through and read the whitepaper and not just the summary. The authors talk about similar ideas. For example, from the paper:
> The more Churn becomes commonplace, the greater the risk of mistakes being deployed to production. If the current pattern continues into 2024, more than 7% of all code changes will be reverted within two weeks, double the rate of 2021. Based on this data, we expect to see an increase in Google DORA's "Change Failure Rate" when the “2024 State of Devops” report is released later in the year, contingent on that research using data from AI-assisted developers in 2023.
The authors are describing one measurable signal while openly expressing interest in the topics you're mentioning. The thing is: what's in this paper is a leading indicator, while what you're talking about is a lagging indicator. There's not really a clear hypothesis as to why, for example, increased code churn would reduce the number of production incidents, the mean time to resolution of dealing with incidents, etc.
> yeah you should click through and read the whitepaper and not just the summary. The authors talk about similar ideas
Ah, this whitepaper that's gated behind supplying my business email address?: https://www.gitclear.com/coding_on_copilot_data_shows_ais_do...
I read the article that was linked, which is generally what's expected of me on HN.
> The authors are describing one measurable signal...
I'm aware of the research around this topic, it's something I like reading about and I've read a lot of takes both academic and colloquial. That may be why I put that idea into words.
Maybe, just maybe, in our future interactions you can avoid being so unnecessarily hostile?
There is actual AI benchmarking data in the Refactoring vs Refuctoring paper: https://codescene.com/hubfs/whitepapers/Refactoring-vs-Refuc...
That paper benchmarked the performance of the most popular LLMs on refactoring tasks on real-world code. The study found that the AI only delivered functionally correct refactorings in 37% of the cases.
AI-assisted coding is genuinely useful, but we (of course) need to keep skilled humans in the loop and set realistic expectations beyond any marketing hype.
But probably the most off-putting thing I've experienced is an OKR session with 50 people in it, where a lead dev publicly opened chatgpt, prompted "how do we reduce the number of critical bugs by 30% in the next quarter", chatgpt came up with a generic response, everyone said "perfect", copy-paste that into Jira and call it a day. And I'm just sitting there and wondering if there was something rotten in my breakfast and I'm hallucinating. Unfortunately my breakfast was fine and that really happened. The few times I've tried using those, they were only helpful with dumb, repetitive and mundane tasks. Anything else, you have to rinse and repeat until you get a working solution. And when you read it(for those of us that do), you realize you might have been better off recruiting a freelancer from year one in University to spend a day mashing it up and likely coming up with something better.
But I bet much of those year ones would be doing this exact thing day in and day out until they come up with a solution: Occasionally I will grab my laptop and go work at a cafe on my personal projects for a change and I can't tell you how many times I've seen people just copy pasting stuff from chatgpt and pasting it back into their IDE/editor and calling it a day - students and clearly people who are doing this for a living. Not to mention copilot, that's the de-facto standard at this point.
In fact I had this conversation last year with a guy(developer) who is 20-something years older than me(so mid 50-s): most of the LLM's are trained on stuff that is in the documentation, examples, reddit and stackoverflow. It's only a question of time until the content found in those exact locations where the training data is pulled from will become more and more AI-generated, models will be re-trained on those and eventually shit hitting the fan. I don't think we are too far off from this event.
I don't think I can believe this story as told...
Every quarter HR requires that we write our "perspective" for the next quarter; what we are going to do and how we are going to improve ourselves. It's purely bureaucratic exercise, has no meaning and no impact on anything anybody does, but on an off-chance somebody reads it I cannot just fill it with nonsense. Writing something resembling sensible, in a stilted language required, always makes me struggle much more than writing code or something with meaning.
Strange that I haven't though about using an LLM or this before; seems like a perfect job for it.
But yeah, I abhor bureaucracy and trivial bs like the one you mentioned so I'm more than happy to outsource this issue to someone/something else.
"Poor code quality due to AI assistants GitHub Copilot and ChatGPT" [1](21 points, 2 days ago, 10 comments)
[0]: https://news.ycombinator.com/item?id=39142285 [1]: https://news.ycombinator.com/item?id=39144366 [2]: https://news.ycombinator.com/item?id=39156643 [3]: https://news.ycombinator.com/item?id=39164079
I don’t know of too many people who would advocate replacing SQL with hand-written C manipulating B-trees and hash tables. Similarly, it’s pretty rare that you want to hand-roll a parser in C over using a parser generator DSL or even something like regex.
Cleanly enough that there's no room for debate, doubt or discussion?
(Edit. Not the same person. I keep making this mistake in discussion threads)
Previously I wrote:
> It's your tone of "this is obvious, people! Why are you still wasting time thinking about it?" that I'm taking exception to.
I would say instead of reacting to the rhetorical remarks, bring up the actual interesting discussion around it in your response.
With all things around me there is a sense that technology is to be a saviour for many very important things - ev's, medicine, it, finance etc.
At the same time it is more and more clear to me that technology is used primarily to grow a market, government, country etc. But it does that by layering on top of already leaking abstractions. It's like solving a problem by only trying to solvent be its symptoms.
Quality has a sense of slowness to it which I believe will be a necessary feat, both due to the fact that curing symptoms will fall short and because I believe that the human species simply cannot cope with the challenges by constantly applying more abstractions.
The notion about going faster is wrong to me, mostly because I as a human being do not believe that quality is done by not understanding the fundamentals of a challenge, and by trying to solve it for superficial gains is simply unintelligent.
LLMs is a disaster to our field because it caters to the average human fallacy of wanting to reach a goal but without putting in the real work to do so.
The real work is of course to understand what it is that you are really trying to solve with applying assumptions about correctness.
Luckily not all of us is trying to move faster but instead we are sharpening our minds and tools while we keep re-learing the fundamentals and applying thoughtful decisions in hope to make quality that will stand the test of time.
I don't even know what it's arguing for... I guess something literally about sewing?
Since nobody has made the concluding argument yet, I’ll supply it; the Luddites were correct that it (mechanization) would destroy their lucrative profession, they missed the degree to which it would enable many other lucrative professions, and they were right that many Luddites would not make the transition.
Society was net better off but there were definitely losers in the transition.
In how far do you think LLMs stand in the way of that?
My experience has been very much the opposite: Instead of holding the hard part of the process up by digging through messy apis or libraries, LLMs (at least, in their current form but I suspect that this will theoretically simply remain true) make it painfully obvious when my thinking about a task of any significance is not sound.
To get anywhere with a LLM, you need to write. To write, you have to think.
Very often I find the most beneficial part of the LLM-coding-process is a blended chat backlog that I can refer back to, consisting of me carefully phrasing what it is that I want to do, being poked by a LLM, and me through this process finding gaps and clarifying my thoughts at the same time.
I find this tremendously useful, specially when shaping the app early, to keep track of what I thought needed to be done and then later being able to reconsider if that is actually still the case.
I recommend Rich Hickey's "Hammock Driven Development" talk. You don't solve hard problems by poking at it repeatedly until something works, that is the recipe for terrible code and abstractions. You instead take a step back from your computer and digest it until you come with a well-understood solution.
This approach is what separates the experienced engineer from the junior. Code is the least of your problems.
There are the John Carmack types, whose output depends on how much time on they spend at keyboard, and the Rich Hickey types, whose output depend on how much time they spend on a hammock with their eyes closed (or under the shower in my case). I am afraid I am of the latter type. My best solutions are found away from the keyboard, as I have learned to simply depend on my subconscious to process and digest them while I'm doing other things.
Check out that talk still, it has deep insight on how the human brain operates.
It's a tool. It doesn't make sense to blame the tool. Is it the screwdriver's fault it gets used as a hammer? Or a murder weapon?
Used intelligently Copilot & Co can help. It can handle the boilerplate, the mundane and free up the human element to focus on the heavy lifting.
All that aside, it's early days. It's too early to pass judgement. And it seems unlikely it's going to go away.
I think his insight applies nearly as well to using code generated by an ai.
IKEA furniture is a great example of this. I build my own furniture and being around it is a much much nicer thing than some piece of cardboard from IKEA.but it seems like cost, soeed an convenience are the most important thing in peoples minds.
Some younger developers have a very different attitude to code than what I was brought up with. They have immense disdain for the Gang of Four and their design patterns (probably without realising that their favourite frameworks are packed to the gills with these very patterns). They speak snidely about principles like DRY and especially SOLID. And on places like Twitter the more snide and contrarian you can be, the more engagement you'll get. Very disturbing stuff.
DRYing code repeated for the same reason is mostly good.
DRYing code coincidentally repeated for different reasons will sow code churn or inadvertent behavior shifts.
You're describing heaven for a maintenance programmer -- I'm only looking at the codebase because there's a bug in some component, I likely even have a stack trace. If can just read what that bit of code does top to bottom, fix the error in just that component, write a test and ship I'll send the original author chocolates.
Also, not all maintenance involves correcting inborn defects.
To first order :vimgrep is totally mechanical and brain-off repeat fix is super easy.
We just cant stand acting as if that random list of principles created by Java's OOP mind was some source of truth for software modeling.
We're just tired of seeing bilionth discussion about how to understand SOLID
You probably don't see people arguing against CAP theorem because it is not some arbitrary collection of ideas (not even fully authored by SOLID author) which composes fancy mnemonic
>There was already a backlash against DRY code occurring before "AI" assistants hit the market, sadly.
As everything else - DRY can be abused too and people backlash against acting as those things were flawless when they arent.
For example, take DRY. The important principle was never really about repeating code. It was about repeating ideas. For any given concept in your system, ideally there should be a single source of truth, and therefore a single place you need to understand or change if you’re working with that concept. It’s true that this means copying and pasting non-trivial amounts of code instead of creating a meaningful abstraction is often a bad idea. But it is also a warning that any time you do repeat an idea, you now have an ongoing liability because you need to keep those different representations in sync. That could refer to database migrations that define your schema and separate ORM class definitions, or an API you define in your back-end code and a client for that API you define in your front-end code, or a retained mode UI where you have a current value in some form field that corresponds to a specific value in your internal application state, or some invariant in your data model that can be represented in both types and unit tests.
People who object to combining duplicate or near-duplicate code that represents different ideas but happens to have a similar implementation at the time, on the basis that it’s a maintenance hazard for later on, aren’t wrong. They’re just objecting to a straw man that was never really the point of DRY in the first place, but has been treated as if it were due to some kind of cargo cult/gaslighting effect.
The question I have now is where and when in our industry do we expect new developers to learn these principles so they do understand them properly? Some people have a formal background in CS or the like, but not everyone does, and in any case it’s not necessarily the role of an academic CS course to teach a lot of practical software development skills. I had a discussion the other day about how when I was starting out, the senior developers would give real, substantial training to the juniors to help us learn and understand these principles, but with the job-hopping culture today and the resulting general aversion to hiring juniors as a long-term investment, that just doesn’t seem to happen much any more. There are formal courses that cost a lot by personal standards but almost nothing by business standards, but there must be a tiny proportion of new developers who actually get sent on them by their employers. There are a few books worth reading, but what 20-something in 2024 wants to deal with presentation as antiquated as ink on sliced bits of tree? I suspect a lot of what today’s up and coming developers learn about these ideas comes from sources like blogs and YouTube videos, where again there is some great material out there, but as ever the problem is finding it among all the poorly understood and dubiously presented dross.
And then we wonder why tools come along that seem like magic, producing a dozen lines of code in a heartbeat that seem to mostly work, and young developers think they’re great even while having little idea of all the deeper things that may be wrong with that code. It’s not really surprising, and I’m not sure it’s really anyone’s fault, but it’s definitely a problem and I wish I knew what we should do about it.
Immense disdain does accurately describe how I feel towards whatever it is happens in corporate codebases. No, creating layers upon layers of indirection via classes is not ok, no matter what your SOLID guru tells you. Best practices, DRY, and SOLID are just excuses.
Couldn’t this equally be explained by the cost of refactoring becoming incredibly cheap? If most of your code is generated, and you don’t have to make the investment of hand crafting everything, aren’t you naturally going to be regularly replacing vast tracts? Obviously the trend may have implications, but in large part aren’t we just seeing the impact of code cheapening? Serious question.
Progress comes from human intellect, mediocrity comes from regurgitation.