i would be may more impressed if LLMs could do code compression. more code == more things that can break, and when llms can generate boatloads of it with a click you can imagine what might happen
This actually sparked an idea for me. Could code complexity be measured as cumulative entropy as measured by running LLM token predictions on a codebase? Notably, verbose boilerplate would be pretty low entropy, and straightforward code should be decently low as well.
Not quite, I think. Some kinds of redundancy are good, and some are bad. Good redundancy tends to reduce mistakes rather than introduce them. E.g. there's lots of redundancy in natural languages, and it helps resolve ambiguity and fill in blanks or corruption if you didn't hear something properly. Similarly, a lot of "entropy" in code could be reduced by shortening names, deleting types, etc., but all those things were helping to clarify intent to other humans, thereby reducing mistakes. But some is copy+paste of rules that should be enforce in one place. Teaching a computer to understand the difference is... hard.
Although, if we were to ignore all this for a second, you could also make similar estimates with, e.g., gzip: the higher the compression ratio attained, the more "verbose"/"fluffy" the code is.
Fun tangent: there are a lot of researchers who believe that compression and intelligence are equivalent or at least very tightly linked.
Interpreting this comment, it would predict low complexity for code copied unnecessarily.
I'm not sure though. If it's copied a bunch of times, and it actually doesn't matter because each usecase of the copying is linearly independent, does it matter that it was copied?
Over time, you'd still see copies being changed by themselves show up as increased entropy
Code complexity can already be measured deterministically with cyclomatic complexity. No need to use an AI fuzzy logic at this. Especially when they're bad at math.
> Could code complexity be measured as cumulative entropy as measured by running LLM token predictions on a codebase? Notably, verbose boilerplate would be pretty low entropy, and straightforward code should be decently low as well.
I agree. It seems like counting lines of generated code is like counting bytes/instructions of compiled code - who cares? If “code” becomes prompts, then AI should lead to much smaller code than before.
I’m aware that the difference is that AI-generated code can be read and modified by humans. But that quantity is bad because humans have to understand it to read or modify it.
Not if it’s already stable and has been running for years. Legacy doesn’t necessarily mean “need replacement because of technical debt”. I’ve seen lots of people want to replace code that has been running basically bug free for years because “there are better coding styles and practices now”
More code could just be useless code that no longer serves any purpose but still looks reasonable to the naked eye. An LLM can certainly figure out and suggest maybe some conditional is impossible given the rest of the code.
I can also suggest alternatives, like using existing library functions for things that might have been coded manually.
When I was there, way more than 25% of the code was copying one proto into another proto, or so people complained. What sort of memes are people making now that this task has been automated?
Memegen is there but unrecognizable now. A dedicated moderator team deletes memes, locks comments, bans people for mentioning "killing a process" (threatening language!) and contacts their managers.
I am very interested in how this 25% number is calculated, and if it's a lot of boilerplate that in the past would have been just been big copy-paste jobs like a lot of protobuffers work. Would be curious if any Googlers could comment.
Not that I'm really discounting the value of AI here. For example, I've found a ton of value and saved time getting AI to write CDKTF (basically, Terraform in Typescript) config scripts for me. I don't write Terraform that often, there are a ton of options I always forget, etc. So asking ChatGPT to write a Terraform config for, say, a new scheduled task for example saves me from a lot of manual lookup.
But at the same time, the AI isn't really writing the complicated logic pieces for me. I think that comes down to the fact that when I do need to write complicated logic, I'm a decent enough programmer that it's probably faster for me to write it out in a high-level programming language than write it in English first.
Well, the article has a paywall so it might go into this.
I'm not sure this stat is as important as people point it out to be. If I start of `for` and the AI auto-completes `for(int i=0; i<args.length; i++) {` then a lot more than 25% of the code is AI written but it's also not significant. I could've figured out how to write the for-loop and its also not a meaningful amount of time saved because most of the time is figuring out and testing which the AI doesn't do.
We should watch for dev layoffs as a sign/signal of the impact of generated code. I remember reading about an anime shop that fired 80% of its illustrators due to ai-images.
By some intuitive measures, it’s surprising they have very many still writing their code. Google’s product quality isn’t what it once was. There is no amount of AI accelerators and energy they can burn through to fix that without humans.
The code would be getting written anyways, its an invariant. The difference is less time wasted typing keys (albeit small amount of time) and more importantly (in my experience) it helps A LOT for discoverability.
With g3's immense amount of context, LLMs can vastly help you discover how other people are using existing libraries.
my experience dabbling with the ai and code is that it is terrible at coming up with new stuff unless it already exists
in regards to how others are using libraries, that’s where the technology will excel— re-writing code. once it has a stable AST to work with, the mathematical equation it is solving is a refactor.
until it has that AST that solves the business need, the game is just prompt spaghetti until it hits altitude to be able to refactor.
Nothing at all. The headline talks about the proportion of code written by AI. Contrary to what a lot of comments here are assuming, it does not say that the volume of code written has increased.
Google could be writing the same amount of code with fewer developers (they have had multiple layoffs lately), or their developers could be focusing more of their time and attention on the code they do write.
I would hope a CEO, especially a technical one, would have enough sense to couple that statement to some useful business metric, because in isolation it might be announcement of public humiliation.
The elitism of programmers who think the boilerplate code they write for 25% of the job, that's already been written before by 1000 other people before, is in fact a valuable use of company time to write by hand again.
IMO it's only really an issue if a competent human wasn't involved in the process, basically a person who could have written it if needed, then they do the work connecting it to the useful stuff, and have appropriate QA/testing in place...the latter often taking far more effort than the actual writing-the-code time itself, even when a human does it.
You're probably thinking of just raw codebases, your company source code repo. Programmers do far, far more boilerplate stuff than raw code they commit with git. Debugging, data processing, system scripts, writing SQL queries, etc.
Combine that with generic functions, framework boilerplate, OS/browser stuff, or explicit x-y-z code then your 'boilerplate' (ie repetitive, easily reproducible) easily gets to 25% of code you're programmers write every month. If your job is >75% pure human cognition problem solving you're probably in a higher tier of jobs than the vast majority of programmers on the planet.
Depends on how you define "boilerplate". E.g. Terraform configs count for a significant number of the total lines in one of my repos. It's not really "boilerplate" in that it's not the exact same everywhere, but it is boilerplate in the since that setting up, say, a pretty standard Cloud SQL instance can take many, many lines of code just because there are so many config options.
It's only boilerplate if you write it again to set almost the same thing again. What, granted, if you are writing bare terraform config, it's probably both.
But on either case, if your terraform config is repetitive and a large part of the code on an entire thing (not a repo, repos are arbitraty divisions, maybe "product", but it's also a bad name). Than that thing is certainly close to useless.
Is it though? It seems to me like a team ownership boundary question rather than an architecture question.
Architecturally, it sounds like different architecture components map somewhere close to 1:1 to teams, rather than teams hacking components to be closer coupled to each other because they have the same ownership.
I'd see too much boilerplate as being a organization/management org issue rather than a code architecture issue
To add: it’s been my experience that it’s the company that thinks the boilerplate code is some special, secret, proprietary thing that no other business could possibly have produced.
Not the developer who has written the same effective stanza 10 times before.
25% of new code might be boilerplate. All my apps in my organization start out roughly the same way with all the same stuff. You could argue on day one that 100% of the code is boilerplate and by the end of the project it is only a small percentage.
That is a fantastic way to put it. I’d argue that you’ve described a bubble, which fits perfectly with the topic and where _most_ of it will eventually end up.
I read these threads and the usual 'I have to fix the AI code for longer than it would have taken to write it from scratch' and can't help but feel folks are truly trying to downplay what is going to eat the software industry alive.
I’m not convinced it’s there yet. I think it’s actively eating part of the software industry, but I wonder where that’ll stop—at least for some time—and a new shape of the industry is settled upon.
There are still things I do in my IDE that I can’t seem to get AI to do. It’s not really close yet. I don’t doubt it could get there eventually, but I suppose I don’t believe it’s about to eat those parts of the industry.
I do anticipate a massive issue from lower skill software jobs vanishing. I don’t know what entry into the industry will look like. There will be a strange gap that’s filled by AI and some people who use it to do basic things but have no idea how it does it. They will be somewhat like data entry workers, knowing how to use a spreadsheet or word processor but having no idea how the program actually works let alone the underlying operating system. I fully expect that to happen, and I can’t properly imagine what the implications will be.
If you manage to convince software engineers that you are doing them a favour by employing them then they will approach any workplace negotiations with a specific mindset which will make them grab the first number it gets thrown to them.
I think it just shows how much noise there is in coding. Code gets reviewed anyways (although review quality was going down rapidly the more PMs where added to the team)
Most of the code must be what could be snippets (opening files and handling errors with absl::, and moving data from proto to proto).
One thing that doesn't help here, is that when writing for many engineers on different teams to read, spelling out simple code instead of depending on too many abstractions seems to be preferred by most teams.
I guess that LLMs do provide smarter snippets that I don't need to fill out in detail, and when it understands types and whether things compile it gets quite good and "smart" when it comes to write down boilerplate.
Not so sure nowadays. Given how often big tech lays off employees and the abundance of recently laid off tech talent, trying to start your own company sounds a lot more appealing than ever.
I consider myself risk-averse and even I am contemplating starting a small business in the event I get laid off.
Interesting. I think the same thing but I wonder if the market is not ready for products created by the big guys, what can I offer? Have you thought in that line?
Well the whole point is that you have some edge that the "big guys" cannot compete with or you have discovered an opportunity they have not (making you ripe for acquisition).
New successful businesses are being created all the time. We just focus on the ones that have already been successful for a long time.
You’re thinking about it wrong. Most large companies won’t put development time into an idea that would only make them $1-5M per year. On the other hand, $1-5M per year is great money to an individual. So there’s a lot of untouched markets that can make you rich but just don’t interest the big guys.
All of that said, there are a lot of products that are produced by large companies and are just bad. Don’t be afraid to go after a Goliath if you see an opportunity.
> trying to start your own company sounds a lot more appealing than ever.
It really isn't. Even if you get laid off from a large tech company, you probably didn't have to pay a cent to get the job there in the first place, and you started drawing a paycheck right away (after the initial delay due to the pay cycle). If you only work there for 6 months, you can save a really good amount of money if you have frugal habits.
Starting a company isn't nearly as easy, usually requires up-front investment, and there can be a long time before you generate any profit. Either you need some business idea that's going to generate profit (or at least enough revenue to give the founder(s) a paycheck), or a business loan or other funding, which means convincing someone to invest in your company somehow.
Starting your own company only sounds appealing if you ignore reality, or have the privilege of having plenty of cash saved up for such a venture.
If I was working a typical corp job, I would "quiet quit" and start using my excess savings to run small experiments. Maybe run like 3 in parallel a month and let them cook.
Example - start a niche blog based on my hobby, hire 3 writers and pay them up to $3k a month to write content for the blog. Let it cook for months. If it gets traffic, monetize with ads. If very profitable, quit job.
Starting an internet based business if fairly cheap nowadays, especially with cursor and ai.
Writing more code means more needs to be maintained and they are cleverly hiding that fact. Software is a lot more like complex plumbing than people want to admit:
More lines == more shit to maintain. Complex lines == the shit is unmanageable.
But wall street investors love simplistic narratives such as More X == More revenue. So here we are. Pretty clever marketing imo.
Google is getting enshittified. It's already visible in many small ways. I was just using Google maps and in the route they called X (bus) Interchange as X International. I can only assume this happened because they are using AI to summarise routes now. Why in the world are they doing that? They have exact location names available.
I'm not sure I get your point. Google created Gemini and whatever internal LLM their employees are using for code generation. Who are they paying,
and for what seats? Not Microsoft or OpenAI or Anthropic...
I don't write code as I'm a sysadmin. Mostly just scripts. But is this like saying intellisense writes 25% of my code? Because I use autocomplete to shortcut stuff or to create a for loop to fill with things I want to do.
You just made it less attractive to the target corps who are to buy this product from Google. Saying, intellisense means corps already have license of various of these and some are even mostly free. Saying AI generate our 25% code sounds more attractive to corps, because it feels like something new and novel and you can imagine laying off 25% of the personnel and justify buying this product from Google.
When someone who uses a product says it, there is a 50% chance of it being true, but when someone far away from the user says it, it is 100% promotion of product and setup for trust building for a future sale.
Don't disagree. But I think it's pretty accepted that sysadminy scripts and full blown applications are different. Just don't want to give the wrong impression that I know what I'm talking about I guess.
1,049 comments
[ 4.6 ms ] story [ 398 ms ] threadAlthough, if we were to ignore all this for a second, you could also make similar estimates with, e.g., gzip: the higher the compression ratio attained, the more "verbose"/"fluffy" the code is.
Fun tangent: there are a lot of researchers who believe that compression and intelligence are equivalent or at least very tightly linked.
I'm not sure though. If it's copied a bunch of times, and it actually doesn't matter because each usecase of the copying is linearly independent, does it matter that it was copied?
Over time, you'd still see copies being changed by themselves show up as increased entropy
In fact it's the one thing they are explicitly designed to do, the rest is more or less a side-effect.
WinRAR can do that for you quite effectively.
I’m aware that the difference is that AI-generated code can be read and modified by humans. But that quantity is bad because humans have to understand it to read or modify it.
What’s the point of shorter code if you can’t trust it to do what it’s supposed to?
I’ll take 20 lines of code that do what they should consistently over 1 line that may or may not do the task depending on the direction of the wind.
fun story: today I had an LLM write me a non-trivial perl one-liner. It tried to be verbose but I insisted and it gave me one tight line.
I understand more code as being more edge cases
I can also suggest alternatives, like using existing library functions for things that might have been coded manually.
Not that I'm really discounting the value of AI here. For example, I've found a ton of value and saved time getting AI to write CDKTF (basically, Terraform in Typescript) config scripts for me. I don't write Terraform that often, there are a ton of options I always forget, etc. So asking ChatGPT to write a Terraform config for, say, a new scheduled task for example saves me from a lot of manual lookup.
But at the same time, the AI isn't really writing the complicated logic pieces for me. I think that comes down to the fact that when I do need to write complicated logic, I'm a decent enough programmer that it's probably faster for me to write it out in a high-level programming language than write it in English first.
I'm not sure this stat is as important as people point it out to be. If I start of `for` and the AI auto-completes `for(int i=0; i<args.length; i++) {` then a lot more than 25% of the code is AI written but it's also not significant. I could've figured out how to write the for-loop and its also not a meaningful amount of time saved because most of the time is figuring out and testing which the AI doesn't do.
With g3's immense amount of context, LLMs can vastly help you discover how other people are using existing libraries.
in regards to how others are using libraries, that’s where the technology will excel— re-writing code. once it has a stable AST to work with, the mathematical equation it is solving is a refactor.
until it has that AST that solves the business need, the game is just prompt spaghetti until it hits altitude to be able to refactor.
Google could be writing the same amount of code with fewer developers (they have had multiple layoffs lately), or their developers could be focusing more of their time and attention on the code they do write.
The truth is likely somewhere in between.
IMO it's only really an issue if a competent human wasn't involved in the process, basically a person who could have written it if needed, then they do the work connecting it to the useful stuff, and have appropriate QA/testing in place...the latter often taking far more effort than the actual writing-the-code time itself, even when a human does it.
That said, I've seen even higher ratios. But never in any place that survived for long.
Everything is getting forced into a scalable, general purpose way, that most apps have to add a ridiculous amount of boilerplate.
Combine that with generic functions, framework boilerplate, OS/browser stuff, or explicit x-y-z code then your 'boilerplate' (ie repetitive, easily reproducible) easily gets to 25% of code you're programmers write every month. If your job is >75% pure human cognition problem solving you're probably in a higher tier of jobs than the vast majority of programmers on the planet.
It's only boilerplate if you write it again to set almost the same thing again. What, granted, if you are writing bare terraform config, it's probably both.
But on either case, if your terraform config is repetitive and a large part of the code on an entire thing (not a repo, repos are arbitraty divisions, maybe "product", but it's also a bad name). Than that thing is certainly close to useless.
Architecturally, it sounds like different architecture components map somewhere close to 1:1 to teams, rather than teams hacking components to be closer coupled to each other because they have the same ownership.
I'd see too much boilerplate as being a organization/management org issue rather than a code architecture issue
Not the developer who has written the same effective stanza 10 times before.
That may explain why google search has, in the past couple of months, become so unusable for me that I switched (happily) to kagi.
> New tool bypasses Google Chrome’s new cookie encryption system
https://news.ycombinator.com/item?id=41988648
- You know which 20K lines need changing - You have perfect QA - Nothing ever goes wrong in deployment.
I think there's a tendency in our industry to only take the hypotenuse of curves at the steepest point
There are still things I do in my IDE that I can’t seem to get AI to do. It’s not really close yet. I don’t doubt it could get there eventually, but I suppose I don’t believe it’s about to eat those parts of the industry.
I do anticipate a massive issue from lower skill software jobs vanishing. I don’t know what entry into the industry will look like. There will be a strange gap that’s filled by AI and some people who use it to do basic things but have no idea how it does it. They will be somewhat like data entry workers, knowing how to use a spreadsheet or word processor but having no idea how the program actually works let alone the underlying operating system. I fully expect that to happen, and I can’t properly imagine what the implications will be.
These statements are brilliant.
Most of the code must be what could be snippets (opening files and handling errors with absl::, and moving data from proto to proto). One thing that doesn't help here, is that when writing for many engineers on different teams to read, spelling out simple code instead of depending on too many abstractions seems to be preferred by most teams.
I guess that LLMs do provide smarter snippets that I don't need to fill out in detail, and when it understands types and whether things compile it gets quite good and "smart" when it comes to write down boilerplate.
I consider myself risk-averse and even I am contemplating starting a small business in the event I get laid off.
New successful businesses are being created all the time. We just focus on the ones that have already been successful for a long time.
All of that said, there are a lot of products that are produced by large companies and are just bad. Don’t be afraid to go after a Goliath if you see an opportunity.
It really isn't. Even if you get laid off from a large tech company, you probably didn't have to pay a cent to get the job there in the first place, and you started drawing a paycheck right away (after the initial delay due to the pay cycle). If you only work there for 6 months, you can save a really good amount of money if you have frugal habits.
Starting a company isn't nearly as easy, usually requires up-front investment, and there can be a long time before you generate any profit. Either you need some business idea that's going to generate profit (or at least enough revenue to give the founder(s) a paycheck), or a business loan or other funding, which means convincing someone to invest in your company somehow.
Starting your own company only sounds appealing if you ignore reality, or have the privilege of having plenty of cash saved up for such a venture.
If I was working a typical corp job, I would "quiet quit" and start using my excess savings to run small experiments. Maybe run like 3 in parallel a month and let them cook.
Example - start a niche blog based on my hobby, hire 3 writers and pay them up to $3k a month to write content for the blog. Let it cook for months. If it gets traffic, monetize with ads. If very profitable, quit job.
Starting an internet based business if fairly cheap nowadays, especially with cursor and ai.
More lines == more shit to maintain. Complex lines == the shit is unmanageable.
But wall street investors love simplistic narratives such as More X == More revenue. So here we are. Pretty clever marketing imo.
it's like companies paying all those todolist and tutorial apps left running on aws ec2 instances in 2007ish.
I'd be worried if i were a google investor. lol.
When someone who uses a product says it, there is a 50% chance of it being true, but when someone far away from the user says it, it is 100% promotion of product and setup for trust building for a future sale.
Not what I thought when I heard "AI coding", but seems pretty neat.
.... so what do you put in your scripts if not code?
Alphabet ($GOOG) 2024 Q3 earnings release
https://news.ycombinator.com/item?id=41988811