48 comments

[ 0.28 ms ] story [ 34.8 ms ] thread
I really don't know what to feel about AI generated code. It's clear that it improves velocity dramatically in code output. It has some value to it. But what is the cost?

While I code without the help of AI, sometimes I feel like I am being left behind.

Ohh you wrote only 400 lines today? I shipped a whole app while being in the gym... hits hard.

You are left behind. That's like saying: "I am using punch cards for programming and I feel left behind as others are using real-time IDEs" You don't need to vibe code (and even this is only a matter of time as llms become better and better), but not using AI at all will definitely leave you behind.
As an AI user you're not that different from the punch cards programmer.

You're more like a cobbler who adopted some new electrical tools for their craft, and feel smug against backwards cobblers using manual tools, while in the meantime the newer versions of those tools in the form of factory automation move in to make the whole craft obsolete.

Please don’t do this if your livelihood depends on it. You’re just going to make your life more painful.

As a professional programmer, you’re selling a service. The people buying your service have fully bought into AIs and accepted their shortcomings. You can’t fight this.

You can still scratch your itch with hobby apps, or contributing to human-first projects like Zig.

> But what is the cost?

The cost is similar to doing any other task in a rushed, careless manner, with little regard for the consequences.

> sometimes I feel like I am being left behind.

And why does that bother you? “Forward” does not equal “good” or “desirable”. If a van is driving towards a cliff at high speed, would you not rather be left behind? “Being left behind” is a talking point to cause you FOMO, don’t fall for it.

All you need to us actually look around, has software quality dramatically increased over the last year? Have useful features been coming out at an unprecedented rate?

Because after all this talk about being left behind and I see nothing to show for it.

Within my own company, yes. We’ve shipped a lot more features this year with the same number of people. They’re also better features. We get from 0 to 90% in an hour and then still put a lot of effort into that last 10%. The projects have better test coverage, docs, logging, etc. then they did 2 years ago. Stretch goals are reached much more frequently. What used to be 3 week projects are closer to 3 days. Our commit history is cleaner, easier to understand, has has better commit messages because we have Claude rewrite the history before opening the PRs. Our product is faster, more stable, and has nicer UX.

But this is anecdotal. It’s a team of senior devs who already knew the codebase well. We have a ton of detailed guidelines that make sure it’s writing code we deem acceptable. We review each others code, and get LLMs to review, a LOT.

So I'm guessing your user satisfaction has gone up a bunch and revenue now reflects your increased productivity?
In my case very much yes to both questions.
Hey we should see this increase of so called productivity in the global GDP growth numbers. If AI really is this productive the slope of GDP growth should be steeper the last 2~ 3 years. If there is no change in the GDP growth slope, there's no general productivity improvement.
People rarely see when something works well. They only see when it breaks. For example, security has been dramatically improved with the help of AI finding vulnerabilities, only being limited by staff actually making fixes (https://medium.com/@the_atomic_architect/ai-zero-day-vulnera...). Life is full of "silent heroes" doing things without notice. Rarely do we see giant leaps overnight. It is a slow, silent advancement invisible to most. We are also still in the "growing pains" of finding the best use cases for AI.
The code itself indeed has become cheap but the question still is out whether it solves the right problem. I've shifted my work significantly toward designing more high-level portions where multiple things would interact in somewhat unpredictable manner (read as failure modes) that's where I found the models being quite poor, they do suggest many things to solve these or that problems but in a unity these suggestions miss second order effects. I was quite surprised that models can't hold all of the connected pieces simultaneously to reason about. But once I have a complete design after a week of deliberations and various attempts, the models can produce lots of correct code to cover that. And then another pass starts because they still deviate in implementation of various pieces and once you start questioning those the models start self-correcting. Judging the output is still way faster than writing the code yourself and indeed models still show some cases that slipped once mind. It's a definitive increase in terms of quality and value of the end code no doubt. Whether it can one-shot a complex problem, not really.
Code is merely the means to an end. It does not matter who writes it as long as it does what you want. Correctly.

Stands to reason though that someone who knows programming AND has domain knowledge can get LLMs to produce much better output compared to someone who does not.

I never used to have time to make all the stuff I needed or was interested in. With LLMs, I can.

This "shipped a whole app while being in the gym" does not work for me though. It takes me a couple of days to a week to produce solid, functional software (~10KLOC). Simple tools (3-400LOC)? Yeah, those you can produce in 30-60 minutes.

As is with any engineering… a key skill is knowing which tool to use when and for what.

There will always be people who push the extreme limits of either way. “I will not use AI for anything!” Vs “I will vibe code this while I’m at the gym”.

The winner is the one that knows when and how to use AI as a _tool_ and focus on the better outcome.

Building a business on vibe coded software feels like building your house on sand.

Sooner or later you'll end up troubleshooting something you just don't understand.

And the tricky part about AI is that it only has to make one bad decision confidently, to then base all future decisions on that one bad one.

AI-assisted coding helps me a lot to churn through boilerplate, straightforward implementations, test writing, exploration of unfamiliar codebases, and other ancillary tasks in coding. It does not replace tacit knowledge/domain knowledge required to design a good solution, no matter how much of the architecture is described as documentation it cannot predict the future to have judgement about a proposed design.

In general AI is an assistance, it doesn't replace the intellectual work that is needed for a good implementation.

In my opinion, approach it like you'd approach any other tool: there are good use-cases for it while being definitely not the right tool for other cases.

I think if you review the code and keep a good mental model of the codebase, the costs are pretty minimal but the value is huge. I'm not going back...
You’re missing out on people getting better at working with ai to improve better long term goals, like testing suites and guardrails, velocity is inconsistent in my experience.

And vibe coding apps in the gym, is like playing a game by typing a cheat code, honestly it’s not satisfying

Many have come to the conclusion it is a tool. If you are doing proper due diligence, the AI writes while you are at the gym, but you spend a bit of time when you get back to make sure it does what it is supposed to. It is often still leagues more efficient this way, even accounting for errors. We all know coworkers that produce just as many if not more errors we have to fix anyway.
The pressure to use LLMs is high in the companies that sell shovels in this gold rush. It would look bad if they did not use them. Linux developers work at those companies.
in your version of the conspiracy theory, the developers are pressured to use AI for their patches in Linux from the company. not that they want to use it because they want to? Like everyone else?
I just wanted to bring up that it's AI companies (which all computer hardware companies are now) that develop the kernel. In that light 17.25% is actually a tiny share of commits to be written by the Machine God.
Its not just that, you are implying that they are pressured to use AI and not using it out of their own will
The company is doing very well now and they know that it is all thanks to the AI data center build out. Their manager and leadership want them to use it. If the bubble pops, their stock based compensation becomes worthless. Their colleagues are in the same situation and some are tokenmaxxing.

These things are what the pressure consists of. Some probably do feel pressured against their will. For some the encouragement changes their will. Some want to use it regardless of anything.

But whether you call it pressure, encouragement, or incentives, its there and it does affect how people think and act.

Or maybe.. people like the tech and use them? Instead of all these theories?
I work for a company that contributes significantly to the Linux kernel, along with many other ventures. I can tell you for a fact that we are _heavily_ pressured to use AI tools for everything we do during the day (including writing documents for management review).

Anecdotal, but the only people I've met who are truly excited about AI tooling were the lowest contributers prior to AI tooling becoming prevalent. The senior devs I interact with on a regular basis are largely negative on the trend, both for code quality issues and the burden it puts on them as reviewers. I can personally say that I am entirely burned out from reviewing the crappiest code I've ever seen, all day, every day, with no end in sight. When it takes less time to generate a revision than it does to review it, this is inevitable. Most people I work with have simply lowered their standards for code quality to meet management expectations for code velocity.

Do you or your team contribute to the Kernel or you just happen to work in a company that contributes?
Alright, I'll bite this one time, but based on your other responses I'm getting the feeling that you're not interested in points of view that differ from your own.

I am not a kernel developer. That being said, I've been here for a bit and have worked across many different teams. Management communication and direction is not a secret here. It is well known what the expectations are for devs. I was directly told as much during a recent internal interview with a post-silicon team.

If my lack of direct kernel experience somehow negates the messaging and mandates from management that I see every day, then I guess there's not much more I can contribute to this discussion.

I think using AI for kernel dev vs AI for CRUD dev is an important distinction. In my company of mostly CRUD development, we are not pressured to use AI but we all use it willingly. There are a few who complain but they are a loud minority - almost everyone likes this tech.

The reason I asked you is because I already have anecdotes and real life data for CRUD devs.

I just can't fathom why people would be pressured to use AI. My hunch is that some people kind of begrudgingly admit that it actually makes people faster and they use it. These kind of people would rather like doing things a bit slowly.

> I think using AI for kernel dev vs AI for CRUD dev is an important distinction.

I completely agree!

> I just can't fathom why people would be pressured to use AI.

I can only speculate, and I obviously don't have any insight into why the mandates and usage targets exist, so I won't comment on that. I absolutely see value in using AI tooling as an additional validation step, but the generative side is what gives me pause. Maybe I've just seen too many cases of irresponsible use, but I'm not fully convinced that using the tools for _writing_ code produces better results.

You were right to callout the distinction, and my apologies if I responded too bluntly. One of your original replies referred to company mandates as "conspiracy theories", which is where my hesitation to engage came from.

Anyway, I think we're largely in agreement regarding your closing paragraph.

No issues, glad that we agree. I personally do use it for generative and I fully see the value for my CRUD engineer type of use. I think kernel development requires more scrutiny and we are not there yet where we can blindly use LLMs for Linux.

But my main point is that it is a bit strange to see these patches that are self disclosed as AI use and then come to conclusion that they were pressured to use it rather then use it because it is useful. I have never come across this kind of pressure, which is why I call it a conspiracy theory.

Seems low. 80% people are manually typing in things? In my circle it's close to 0% code written manually.

Do linux devs live in a completely different microcosm?

The Linux kernel is essentially the nightmare scenario for a LLM.

No tests, not much documentation to work with, a very high acceptability bar, multiple ways to do the smallest change and every single word on your commit messages is very important.

You can start with a LLM but there going to be some manual modifications you will have to make, even with the best model available

Large legacy projects are where LLMs are the worst, historically. And I suspect the hard part of Linux isn't typing things, and that the rigorous spec required to get an LLM to generate the correct code is basically the code itself.

I'm reminded of some of the coding stories at NASA where there'd be a ton of discussion for just a few lines of code. And the space shuttle code was apparently some of the most defect-free coffee ever written.

> In September, AI generated code has made up 17.25% of all Linux Kernel patches

I'm curious ... how would they even know?

Just thinking loudly here, I guess Co-Author in commits is a hint. PRs where LLMs have been involved is probably another measurement. Github Copoilot has its own account too.
They require disclosure of what tool was used if submitting AI-helped code.

(Which probably means this is slightly underreported from people not disclosing.)

Yeah, I wonder which part of this figure is people doing "let me ask the LLM to do X... aw shucks, that's not how you do X you dumb clanker, fine, I'll do it myself" and not deleting the metadata left by the LLM in the initial draft (a very common scenario in my personal experience). Or people using LLMs for "dumb" large scale changes instead of conjuring an ad-hoc grep+sed script.
Huh.

Wait until it blows.

And I say this as someone who uses AI (for fun) and to check things I may havbe overlooked.

There's a place and time for everything. Using AI where you shouldn't is gonna backfire, big time.

Also, just please show me ONE single app/product that has been generated 100% by AI--that works and competes with human-written ones?

I'll be waiting, right here.

Midwits can only downvote when they can't engage in a convo/debate.
So the Linux kernel has now just become by definition "slop"? I mean, there is a definition here which says it is now "tainted". [0]

Does that mean we should stop using it?

[0] https://codeberg.org/ethical-foss/open-slopware

Yes. Codeberg running on top of linux and git just shows that they are huge hypocrites.
I agree.
Codeberg existed before Linux became AI sloppified.

It's hard to turn a large ship around.

I still don't get how are they dealing with copyright here. Looks like the ostrich algorithm (hide your head in the sand and pretend nothing happens).

Those patches are not copyrightable unless human made. There's a reason why Oracle doesn't accept AI contributions to Java.

Did Bryan Lunduke measure incoming patches or actual merged patches? Are the AI patches bigger on average? Who or what says the patch was made with AI in the fitst place?
This is awful. Linus Torvalds really surrendered the kernel to AI slop. He was warned, and he made nasty comments to those who called him out, but he owns this mess now.
Wouldn't we have to know if the patches were good or not before concluding that it was a mess?