441 comments

[ 2.8 ms ] story [ 298 ms ] thread
Seems plausible, especially in combination with the AI-coma that occurs when you tab-complete your way through problems at full speed.
know what this will be about without reading

Python 3.12-style type annnotations are a good example imo, no one uses the type statement because dataset inertia

(comment deleted)
Usually, i remember that type annotations exist when I'm debugging after things aren't working. If you look at the python code that I've written, type annotations are a sure sign that "there was a problem here". it's like a scar on repaired code
Type annotations by themselves, are little more than a comment
They're more than that as they allow for easy consistency checking of types across calls. This makes all the difference.
Is this such a bad result? Do we need office CRUD apps to use bleeding edge technologies?
It's also a problem when adopting new functionality in existing frameworks (eg. upgrading an app to new Android release), dropping the use of deprecated functionality, taking advantage of more readable syntax in programming languages, etc.
> taking advantage of more readable syntax in programming languages

"AI" is the programming language here. The readability of any lower level language(s) that may exist as a compiler target of the "AI" is about as important as how readable assembly language is to someone writing software in Rust.

If they don’t, then will the bleeding edge technologies ever be used by more serious apps? Will it even be ready if we don’t test it with easier use-cases?
(comment deleted)
Probably not, but I guess the worry would be, if nobody adopts and uses them, bleeding edge technologies don't become the new normal. Unless you think we've reached perfection, it's almost guaranteed that future developers will look at React/Python/etc as we look at developing in Assembly or COBOL.
Along similar lines, I found Google auto complete to constrict my search space. I would only search the terms that auto complete.
If AI stifles the relentless churn in frontend frameworks then perhaps it's a good thing.
It isn't only frontend frameworks.

I currently AI-coma / tab-complete C++17 with decent results for stuff ridiculously far away from frontend, but I do wonder who is providing the training data for C++23 and onwards as there isn't wide adaptation yet.

I can chime in with a similar anecdote. I use co-pilot extensively in "fancy tab completion" mode. I don't use the conversational features - just the auto-complete to help my coding along.

I specifically found it very useful when dealing with a bunch of boilerplate C++ shim code that used some advanced template magic to declare families of signatures for typed thunks that wrapped and augmented some underlying function call with record/replay log.

It was arcane, bespoke stuff. Highly unlikely to be imitated in training data. The underlying code was written by a colleague (the CTO) but it was hard to grok just because of all the noise the template magic added to the syntax, and carefully keeping track of what was getting substituted where.

The changes I was making were mostly to add new shims to this collection of shims, and co-pilot happily used the pattern of shims in previous lines of code to extrapolate what my new shims should look like and offer me tab-completions that were sensible.

That included some bad guesses (like inferred names for other templated forms that referred to different argument types), but overall it was structurally close enough to use as a reference point and fix up as needed.

It was really great. Took what would have been about day's worth of work carefully figuring out from first principles how the template system was structured.. and made it into about half an hour of "get it to generate the next shim, verify that the shim does the right thing".

> It was really great. Took what would have been about day's worth of work carefully figuring out from first principles how the template system was structured.. and made it into about half an hour of "get it to generate the next shim, verify that the shim does the right thing".

That also seems to highlight the disadvantage too - if you'd taken a day, you would have come away with a deeper understanding of the template system.

Fair point. In that particular circumstance I had no desire to learn the details of the system - the need of the day was to get in there, get the shims in, and get out to the other code that mattered.

If I hadn't worked for more than a decade with C++ and already been reasonably fluent in template semantics, there's a good chance I might have introduced bugs as well.

I think the issue is that these feel like incredibly safe tools because all they do is generate text, but at the same time it can lead to bad hygiene.

I personally never use AI to write prose (I'll keep my voice and my quirks thanks). And for code, I utilize the system specifically as a fancy pattern extension system. Even in well-abstracted code, about 90% of the structure of it can be inferred from the other parts. I let the AI complete that for me with tab-completions and verify that each line is what I would have otherwise typed before moving on.

I'm not comfortable engaging at a deeper level with these tools yet.

I agree. Most frontend rewrites are totally unnecessary, caused by resume-driven development, desire for the latest fancy thing and most JS frameworks having a short lifespan. If LLM's reduce that behavior by steering devs towards most popular solutons, then it's only a good thing.
EXACTLY.

I suspect there has been a decade long Three-card monte on the front end in that change is good because change keeps front end salary up.

Personally, the sooner LLMs make all front end developers unemployed the better.

As the saying goes:

    while (React.isPopular) {
      React.isPopular = true
    }
It's actually quite sad because there are objectively better models both for performance and memory including Preact, Svelte, Vue, and of course vanilla.
Does that really matter to most companies/developers? I’d much rather have a good enough solution with a large ecosystem built around it. It also takes a lot of investment for companies to change their tech stack

    > Does that really matter to most companies/developers?
If you're asking about performance and memory, then yes, it does.

This is especially true in e-commerce where many studies have shown that overall page performance has a correlation to conversion. Add to that the fact that a lot of e-commerce has moved to mobile web, there's a strong case for picking the best performing technologies versus developer preference -- especially if AI is generating it.

But even outside of e-comm, consider government websites where you may have low income users that are using cheaper, older, lower capability devices to access information and services using lower speed networks to boot.

I do my day-to-day work on an M3 Max with 64GB RAM and fiber to the home; it's easy for developers to forget that many times, their end users can be on older devices, on low performing networks, and other factors that affect performance and usability of web applications.

    > ...with a large ecosystem built around it
When you can generate whatever you want nearly instantly, what does "ecosystem" mean? Your possibilities are endless and infinite. Your mindset is in a world where it's necessary to depend on the work of others because it's too expensive for you to write your own component library. Yes, that was true 1 year ago; why would you waste time and energy to create your own calendar component? But if an LLM can generate any bespoke component that you need in < 3 seconds, do you still need a third party component library?

In fact, you may be better off for not creating the added external dependency.

> I do my day-to-day work on an M3 Max with 64GB RAM and fiber to the home

And you still pay a few cents extra in power used because of all those inefficient and memory hungry "applications". You just don't notice.

So, if you're a large e-commerce company that's trying to juice out last percentage points of conversion, and are researching alternate javascript libraries, is it plausable that the only research a development team would be to ask ChatGPT?
Unfortunately not the case because of GPT's bias towards React (the point of the article).
> If you're asking about performance and memory, then yes, it does.

Most places just don't care. I've worked 15 years as a contractor and only in once place have the business cared about optimisation. As long as it wasn't unbearable than it was "good enough".

> This is especially true in e-commerce where many studies have shown that overall page performance has a correlation to conversion. Add to that the fact that a lot of e-commerce has moved to mobile web, there's a strong case for picking the best performing technologies versus developer preference -- especially if AI is generating it.

This may have been true back in 2014. 5G networks are pretty fast and the the mobile web is pretty bloated. Performance is way down the list of concerns typically even by places that should care. I can write blazingly fast custom JS frameworks, the number of times anyone cares is exactly one time.

> I do my day-to-day work on an M3 Max with 64GB RAM and fiber to the home; it's easy for developers to forget that many times, their end users can be on older devices, on low performing networks, and other factors that affect performance and usability of web applications.

I have a 2010 Dell E6410 with 8GB of ram and an i7 640M (Dual Core, 4 thread). Almost every modern phone is faster now.

I am not arguing we should make things bloated. I am just saying there isn't an incentive to optimise for low end devices because low end is better than a reasonably power Business Laptop of 10-15 years ago.

> why would you waste time and energy to create your own calendar component? But if an LLM can generate any bespoke component that you need in < 3 seconds, do you still need a third party component library?

The code from the LLM probably hasn't been battle tested. The open source react component library with 1000s of stars on github definitely has been. If you run into a problem with the LLM code you are probably going to be by yourself fixing it. I will take the component library over the LLM code everyday of the week.

Have you ever worked for a place that cared about meeting CWV? Poor JS performance will definitely hurt rankings.
Yes. It was one contract only and I was the lead frontend dev.

I wrote a very lightweight JS framework (it was just a few classes really) so we could have plugins. A plugin implemented was just an object that implemented an interface, I also wrote a poor man's React for two or three pages that needed to build a lot of DOM Dynamically.

At launch the site was getting basically 100 on the lighthouse tests with the gzipped CSS and JS coming it at ~80KB. Of course that lasted for a week because people will put up a huge image that hasn't been optimised for the web.

The site was fast because I wrote it like a website from mid-2000s. Everything was basic OOP and almost all the code was procedural.

If you use next.js with react server components you can get enough performance out of react for e commerce.

Also the react compiler is improving client side performance as well by automatically memo-izing everything to reduce rerenders

The auto-memoization is:

1) Trading memory pressure for performance

2) An admission of a broken model because it's taken them 2+ (almost 3?) years to build as a recognition that developers can't get it right

The reason other frameworks don't need this is because they use signals connected to callbacks instead of the reactive callback pointing to the component (as React does). Thus Preact, Solid, Svelte, Vue, etc. do not have this issue and you rarely (if ever?) have to manually memoize.

The React team foot-gunned themselves and every React dev out there.

I have some examples that walk through this concept using JSFiddle so you can understand exactly why this design choice from React created this problem in the first place: https://chrlschn.dev/blog/2025/01/the-inverted-reactivity-mo...

Fwiw, I work at a saas company and we do have some performance issues. It's about 50/50 split between not using react optimally and slow backend.

If we were using svelte we would still have performance issues, but they would probably be centered more on "is the data in a sane shape such that this page can query it?"

React has become the Java of late 90’s to mid 2000’s.

Loads of libraries, documentation, and developers which creates a flywheel that will grow those aspects over the next X years.

Until something comes up that is magnitude better in performance/maintainability, and even then it’ll take years to dethrone the status quo.

Good questions in these comments essentially asking, does the level of training data on these models now contribute to the inertia we see from libraries, documentation, developer support?

I believe so, but then again I think we’ll soon have more niche models for specific areas of development (like openart has with a variety of image gen models)

Most of the alternatives on the GP's comment have more than one order of magnitude better performance.

Maintainability doesn't even enter the question. React's way is to rewrite. All of the alternatives on the GP's comment are possible to maintain.

> React has become the Java of late 90’s to mid 2000’s.

Not comparable.

Java may not have had big releases during this time but there were patches and support. Java 8 had numerous versions (>400). You can get paid support from Sun/Oracle. In terms of frameworks Spring was constantly upgraded and supported.

React has none of these. Older React versions rarely get upgraded and just look at the amount of minor/patch releases React gets these days. It's almost as though Meta no longer cares. Earlier (<16) React was constantly updated. Nowadays it's just to peddle Vercel.

At least Java is good. React is absolutely terrible.
But equally impressive that the Javascript community has actually managed to continue to use a single framework for more than five minutes without jumping to the next.
had to have hooks though. And vanilla-OO had to go in favor of trapping state in closures, which is cooler because it has functions and not methods.
You can still use vanilla-OO React.

Do you complain when other frameworks add new features without breaking backwards compatibility?

I certainly did complain! And I'm sure I'll do it again if the new features aimed to supplant the old ones are worse.

If you search anything about React now, 90% of the docs are hook-based. Beginners of React in 2025 will be guided to use a default pattern which has worse runtime footprint and adds a whole suite of new tool-specific coding guidelines ("rules of hook"). After years with it, I struggle to see what it has added in terms of building front-end SPas, yet the pattern is now the default for all using React.

    > You can still use vanilla-OO React
What we want is signals-based React because it would singularly fix the issue that the compiler is solving for and remove the need to even have `useMemo` and `useCallback`, improve performance, and remove a huge footgun.

Because it has such a huge marketshare, a fix like this would have tremendous positive effects in web dev. We are all "forced" to use React so it would do all of us a great service if the team would just backstep and say "Oops, we made a design mistake here!". Instead, they spent almost 3 years building the compiler to sprinkle in memoization primitives because enough devs cannot get it right.

React is almost 12 years old and has dominated frontend development for almost a decade. I'd bet most JS backend projects still use Express as the webserver, and it's even older than React.

Can we please retire this meme? It's stale and adds nothing to the conversation.

Yes thanks to React we can finally retire this meme. It was very true before react became dominant though.
(comment deleted)
and of course vanilla

That depends on who is writing it and what the app is. Most frontend code is written by people who don't have as much time to focus on performance and optimization as core framework developers, so their once their apps reach a critical mass of 'actually big enough to benefit from a framework' the app is worse than it would have been if it was written with a framework in the first place.

The problem for all of us, and where frameworks often make the web suck, is that very few apps are actually that big. Frontend developers love to put React in a page that has one form input a button, which is dumb.

Ostensibly, an unbiased, well-trained model would solve this because it would/could write fast, performant vanilla. As a product owner, you probably care more about the performance of the product over the specific technical implementation details.

I take that to be the point of the article: the bias towards React and of course the training data being stale means that generated code will always have a bit of staleness and as we provide less context for the AI (e.g. StackOverflow), the bias towards staleness will amplify given the large body of stale information it has ingested and amalgamated.

Doesn't Preact use the same model but prioritise bundle size over performance?
It's not the same model.

Many, many (if not most) devs probably do not realize that React has an "inverted" model of reactivity and is in fact the root cause of it's performance woes.

To the extent that the React team spent 2+ (almost 3?) years working on a compiler to address the issue by adding in the correct memoization primitives in a compile phase (trading increased memory consumption for more runtime performance...).

I wrote about it here with code examples that work in JSFiddle: https://chrlschn.dev/blog/2025/01/the-inverted-reactivity-mo...

The short of it is that by pointing the reactive callback to the component function, it means that state within the component function has to be managed carefully. This doesn't happen in Vanilla, Preact, Solid, Svelte, and Vue because they point the reactive callback (via "signals") to a handler function that captures the component state in a closure. This is also why they are all faster and consume less memory than React.

Because React points the reactive callback to the component function, it effectively starts from a clean slate each re-render so the purpose of React Hooks is to move state out and inject them back (thus they are called "hooks") when it re-renders. In Preact, this is not the case since it uses signals: https://preactjs.com/guide/v10/signals/

A short video of the same examples if you prefer: https://youtu.be/7OhyP8H7KW0

The blogpost in the first link doesn't mention Preact at all.

Preact is mostly API-compatible with React, and it having a different underlying model is an extraordinary claim that requires extraordinary evidence.

I've read the docs on Preact signals, and they look like React refs, but put outside of components.

edit: the last paragraph about refs

    > ...a different underlying model is an extraordinary claim that requires extraordinary evidence
You are looking at the syntax and not the reactivity model (how it determines what functions to call when a change happens).

The post doesn't need to mention Preact because every other framework is signals-based except for React. Vue is simply the stand-in for "signals-based reactivity". Vue has different syntax from Preact (though it, too, can also use JSX), but it has the same reactivity principle.

https://preactjs.com/guide/v10/signals/

    > What makes Signals unique is that state changes automatically update components and UI in the most efficient way possible. Automatic state binding and dependency tracking allows Signals to provide excellent ergonomics and productivity while eliminating the most common state management footguns.
It uses the same underlying reactivity model as Vue, Svelte, Solid, and Qwik.

Vue docs: https://vuejs.org/guide/extras/reactivity-in-depth.html#conn...

Solid docs: https://www.solidjs.com/docs/latest/api#createsignal

Svelte: https://svelte.dev/blog/runes#Signal-boost

Qwik docs: https://qwik.dev/docs/components/state/#usesignal

In the blog post, Vue is the stand-in for signals-based reactivity. All signals-based reactivity models work the same way at a high level (their difference being primarily in their low-level DOM diff and update).

My prediction is that even React will eventually end up signals based because of TC39 Signals: https://github.com/tc39/proposal-signals

Preact is not signals-based, it uses the React model with state and props as the basic model, but also provides Signals as a parallel model.
I guess that's like arguing a Prius Prime isn't electric because even though it has a battery and can drive all electric, it also has a gas engine. :shrug:

But very clearly, Preact has the option of the exact same reactivity primitive and design as Vue, Solid, Qwik, and Svelte

https://preactjs.com/guide/v10/signals/

    > In Preact, when a signal is passed down through a tree as props or context, we're only passing around references to the signal. The signal can be updated without re-rendering any components, since components see the signal and not its value. This lets us skip all of the expensive rendering work and jump immediately to any components in the tree that actually access the signal's .value property.
I'll accept an analogy with a hybrid car. In this analogy, Preact would be a gas car with an additional electric engine bolted on as an afterthought.
But you know, the Prius Prime is a PHEV as in "plug in hybrid electric vehicle"
Oh, I see what you mean now! This sounds like a very good analogy.
I think react hits a really good sweet spot in the trade off space. Sure it’s not the best thing that can exist, but it really does solve a lot of problems in a way that isn’t overly restrictive.

My personal opinion is that a lot of the hate directed at react is due to experiences with code bases that aren’t good react code.

React is better than all of those because of the existence of React Native, React Three Fiber, Remotion, etc…

It has the best ecosystem of libraries and it’s not even close.

If you write your web app in Vue and decide you want mobile apps later you won’t be able to share much code there.

it's my goto along with NodeJS on the backend and Electron as a wrapper or React Native

NGINX for my server though recently I ran into an out of connections problem that was new on an Azure VM

Herein lies the key for IP protection. Never use cloud hosted coding tools as the world will soon be able to copy your homework at zero cost.
I for one love it that we can copy each other's homework. The open source mindset is what made me fall in love with this industry, and I love the fact that sharing code got easier. If you really want to continue reinventing wheels, go ahead.
Yes, open is overall a great thing. But what if I want my work not to be open source, and the LLMs make it so without my consent. As we've seen, this and related matters of fair use, are working their way through the courts.
Perhaps reasoning will help?
Yes! Reasoning is the answer! It will solve all of our problems! General AI is just around the corner!!!!!!!!!!!!!!!!!!!
Guess I figured out my niche as a SWE: have a later knowledge cutoff date than LLMs
>With Claude 3.5 Sonnet, which is generally my AI offering of choice given its superior coding ability, my “What personal preferences should Claude consider in responses?” profile setting includes the line “When writing code, use vanilla HTML/CSS/JS unless otherwise noted by me”. Despite this, Claude will frequently opt to generate new code with React, and in some occurrences even rewrite my existing code into React against my intent and without my consultation.

I noticed this too. Anyone found out how to make Claude work better?

Since the system prompt tied to the artifacts feature seems to be the reason for it having a preference for react, seems like the solution would be to use the API instead. Plenty of front ends available nowadays that let you use your own API key. I've been using typingmind since I paid for it over a year ago but I'd be interested to know if some good open source alternatives have popped up more recently.
The main blocker to using the API with an alternative frontend is the cost.

Daily API usage can easily go above the $20/month subscription cost since output tokens are expensive and each new message reuses the whole message chain as input tokens. Especially true if you often upload images or documents.

Third party providers work through the APIs and thus do not have these issues. I use Kagi but there are many options out there for a price similar to the normal subscription, and plus you get a wider choice of models.
(comment deleted)
Claude is particularly bad about this, almost makes it unusable for my frontend use cases. I specify the exact tech stack in my prompt and it responds with a solution using whichever packages are available in its environment (Tailwind, shadcn/ui, etc.).

My request to model providers: the strength of your offering is its generality. Please let it be general-purpose and resist adding features (alignment, system prompts, custom stuff like Claude artifacts) that limit this.

So... it slows down adoption by providing easier alternatives for beginners? I guess you could look at it that way too.

Eventually it will go either of the two ways, though:

- models will have enough generalization ability to be trained on new stuff that has passed the basic usefulness test in the hands of enthusiasts and shows promise

- models will become smart enough to be useful even for obscure things

I think if you specify a technology in your prompt, any LLM should use that technology in its response. If you don't specify a technology, and that is an important consideration in the answer, it should clarify and ask about technology choices, and if you don't know, it can make a recommendation.

LLMs should not have hard-wired preferences through providers' prompt structure.

And while LLMs are stochastic parrots, and are likely to infer React if a lot of the training corpus mentions React, work should be done to actively prevent biases like this. If we can't get this right with JS frameworks, how are we going to solve it for more nuanced structural biases around ethnicity, gender, religion or political perspective?

What I'm most concerned about here is that Anthropic is taking investment from tech firms who vendor dev tooling - it would not take much for them to "prefer" one of those proprietary toolchains. We might not have much of a problem with React today, but what if your choice of LLM started to determine if you could or couldn't get recommendations on AWS vs Azure vs GCP vs bare metal/roll your own? Or if it suggested only commercial tools instead of F/LOSS?

And to take that to its logical conclusion, if that's happening, how do I know that the history assignment a kid is asking for help with isn't sneaking in an extreme viewpoint - and I don't care if it's extreme left or right, just warped by a political philosophy to be disconnected from truth - that the kid just accepts as truth?

This is actually a very interesting insight, not only do you have to worry about sponsored results but people could game the system by spamming their library/language in a places which will be included in the training set of models. This will also present a significant challenge for security, because I can have a malicious library/package spam it in paths that will be picked up in the training set and have that package be referenced by the LLM.
(comment deleted)
The more likely and far more mundane outcome isn’t that LLM providers actively tip the scales, but rather that they just entrench existing winners.

As others have pointed out, it's a flywheel: Popular library gains traction → LLMs are trained to produce the “most likely response”, which naturally aligns with what’s already popular → people stop seeking alternative solutions and instead double down on the existing mainstream choices. (Hypothetically) It's not that OpenAI decides to push AWS, it's just that at the time it was trained AWS was the only real option so it just regurgitates a common view from a point in time.

To extend your analogy, the more realistic scenario isn’t that kids slip in extreme view points and take them as ground truth in their history assignments, it’s that they don’t take a stance on anything at all: Their essays become like CSPAN transcripts, perfectly regurgitating what happened without taking any position or applying any critical thinking one way or the other.

Imagine kids writing about civil rights, but all their reference material was stuck in time at 1953: That's what's more likely to happen.

> The more likely and far more mundane outcome isn’t that LLM providers actively tip the scales, but rather that they just entrench existing winners

100%. This isn't that different from the previous status quo (googling how to build a web app will give me guides from digital ocean, vercel, etc about deploying the currently popular technology on their platforms)

As in all things, though, the new technology reinforces this feedback loop faster.

Fwiw, I haven't had any trouble using Claude in cursor to write svelte5 code - there are tools (cursorrules, the svelte 1-pager documentation for LLMs) that you can use to make sure it uses the tech you want. It just requires intention from the prompter and good documentation from the tooling

> I think if you specify a technology in your prompt, any LLM should use that technology in its response. If you don't specify a technology, and that is an important consideration in the answer, it should clarify and ask about technology choices, and if you don't know, it can make a recommendation.

I'm sure we'd all love that but this pipe dream is simply incompatible with the way LLMs work.

orchestration/deployment/agent networks may be able to do that, but that's basically impossible for the LLM itself.

This will be solved eventually on the AI model side. It isn't some law of nature that it takes a million tokens for an AI to learn something; just the fact that we can prompt these models should convince you of that.
Maybe, but why would they bother? If 80% of the demand is met by generating really good Python and generating really good X is a lot more work but only 2% of the demand it’s likely there isn’t going to be a reason to solve that problem well.
That's assuming it's a novel problem to deal with, see e.g. C++, JavaScript or every standard in networking ever. The barrier between better tech and worse tech that accidentally made it into production and became legacy cruft has always been high, without any AI.
LLM based AI tools are the new No/Low Code.
It is, but it's an order of magnitude better than the last set of no/low code tools for anyone who has the basics of programming already.
Yeah maybe. But I think the thing I like is that is takes me a much shorter amount of time to create solutions for my users and myself. Then I can worry about “tech adoption” once I’ve achieved a relevant solution to my users.

If performance is an issue then sure let’s look at options. But I don’t think it’s appropriate to expect that sort of level of insight into an optimised solution from llms - but maybe that’s just because I’ve used them a lot.

They’re just a function of their training data at the end of the day. If you want to use new technology you might have to generate your own training data as it were.

Larger context windows are helping solve this, though.

I use ALpineJS which is not as well known as React etc, but I just added a bunch of examples and instructions to the new cursor project rules, and it's now close to perfect.

Gemini models have up to 2M context windows, meaning you can probably fit your whole codebase and a ton of examples in a single request.

Furthermore, the agenetic way Cursor is now behaving, automatically building up context before taking action, seems to be another way around this problem

I also suspect reasoning models will start contributing genuine innovations to public repositories in the near future.
I was thinking the other day how coding assistants would hinder new languages adoption.
Pathologists as a specialty has been grousing about this for several years, at least since 2021 when the College of American Pathologists established the AI Committee. As a trivial example: any trained model deployed will necessarily be behind any new classification of tumors. This makes it harder to push the science and clinical diagnosis of cancer forward.

The entire music community has been complaining about how old music gets more recommendations on streaming platforms, necessarily making it harder for new music to break out.

It's absolutely fascinating watching software developers come to grips with what they have wrought.

The healthcare diagnosis one may be wrong. For existing known diagnoses, (or at least the sliver of diagnoses in this one study), AI can beat doctors - and doctors don't like listening when it challenges them, so it will disrupt them badly as people learn they can provide data from tests directly to AI agents. Sure, this doesn't replace new diagnoses, but the vaaaast majority of failures to diagnose are for existing well classified diagnoses.

https://www.advisory.com/daily-briefing/2024/12/03/ai-diagno...

Edit: yeah, people don't like this.

I'm familiar with the linked study, which presents legitimately challenging analytic problems. There's a difference between challenging analytic problems and new analytic problems.

A new platform poses new analytic problems. A new edition of the WHO's classification of skin tumors (1), for example, presents new analytic problems.

(1) https://tumourclassification.iarc.who.int/chapters/64

Right, but the vast majority of patient issues today are missing existing diagnoses, not new ones.
I think OP was referring to the case where new illnesses that are not part of the training set are never going to be diagnosed by AI.
It's only a problem if hospitals replace doctors with AI. If they employ AI as well then outcomes will improve. Using AI to find the ones AI can identify means doctors have more time to focus on the ones that AI can't find.

Of course, that's not what's going to happen. :/

> Using AI to find the ones AI can identify means doctors have more time to focus on the ones that AI can't find.

That's not how that would work in the real world. In a lot of places a doctor has to put their signature or stamp on a medical document, making them liable for what is on that paper. Just because the AI can do it, that doesn't mean the doctor won't have to double check it, which negates the time saved.

I would wager AI-assisted would be more helpful to reduce things doctors might miss instead of partially or completely replacing them.

Interesting. Do you see any versions of the future where use of AI could actually make the physician take more time?
Let's assume you program it so that if it believes with 95% certainty that a patient has a certain condition it will present it to the doctor. While the doctor doesn't agree with it, the whole process between doctor-patient-hospital-insurer might be automated to the point where it's simpler to put the patient through the motions of getting additional checks than the doctor fighting the wrong diagnosis, thus the doctor will have to spend more time to follow up on confirming that this condition is not really present.

I don't have a crystal ball, so this is a made-up scenario.

Never is a long time.

Sure, LLMs might not do this anytime soon, but once models understand enough biology, they're going to identify patterns we don't and propose new diagnoses. There's no reason why they wouldn't.

Unfortunately, that's not how LLMs work.
It has been interesting to see the excuses from doctors, why we need error prone humans instead of higher quality robots.

>Empathy (lol... from doctors?)

>New undetectable cases (lol... AI doesnt have to wait 1 year for an optional continuing education class. I had doctors a few years ago recommending a dangerous expensive surgery over a safer cheaper laser procedure)

>corruptible (lmaooo)

We humans are empathetic to the thought our 'friendly' doctor might be unemployed. However, we shouldn't let that cause negative health outcomes because we were being 'nice'.

So… we put all of our trust (wait, at that point it might be called faith) into this machine…

If it ever turns on us, begins to malfunction in unforeseen ways, or goes away completely—then what?

Shortsighted, all of it.

I doubt it is easier to retrain a large, dispersed group of humans on a new classification of tumors than it is to retrain a model on the same.
I think it depends on what you mean by “easier”. Dispersing knowledge through people is more intuitive, and tends to happen organically.
Not if they're trained to work through the problem each time they encounter it and stay up with their clinical training. The day the new classification drops many have already heard about it.

You also assume that all the models in use will in fact be retrained.

Generally, this position flies in the face of lived experience. AI is in fact stifling adoption of new things across many industries.

My position is informed by my own experience; I am not a physician, but have worked closely with a large number of them in a healthcare-oriented career. I've repeatedly noted long-term resistance of many physicians to updating their priors based on robust new evidence.

There are definitely many physicians who do take in the latest developments judiciously. But I find the long tail of default resistance to be very, very long.

I was just explaining to a UK colleague about how the American health care system makes getting treatment (and getting it paid for) into a DIY project. And so as a medical shopper, if I’m getting a very standard established treatment I might go for the older experienced doctor, but if it’s a new thing I’d opt for someone more recently graduated.

I’m sure the same thing applies worldwide.

It's not bad actually, considering how many times the new shiny thing has turned out to be quite dangerous a few years later. In a field as high stake as healthcare you want the whole spectrum, from early adopters to die hard skeptics. Especially since we know the reproducibility problem of research, the influence of big pharma and big insurance on healthcare, etc etc
Where I live, specialists can't even speak English, so I doubt very much that they are up to date on anything. And I live in a first-world country.
On a first glance this may look like bad comment but there is sound reasoning, I see it with friends who are top notch doctors and surgeons in french part of Switzerland.

Many articles and conferences are in english, often dont get translated (well), and one friend who is urology surgeon specifically mentioned this as an issue in his (former) department in Switzerland's biggest hospital (HUG in Geneva). They simply lag behind bleeding edge a bit.

Can't comment on other languages/cultures, but french speaking folks and english often dont pair well together.

I'm also in a country where English isn't the first language and for the doctors that do wish to stay up to date on what's going on, there are ways for them to do it, and translation technology is pretty top-notch already.

Aside from time constraints and perhaps no incentive to stay up to date, we do have to remember that some of these new discoveries always take time to find their way into becoming SOTA treatments everywhere, due to costs, regulations needing updates, special training or equipment being required, as well as sometimes only being marginally better than existing treatment options.

> You also assume that all the models in use will in fact be retrained.

And that deploying the retrained models won’t require a costly and time consuming recertification process. This is medicine after all.

There's no way that doctors in the US continue their training.

I suffer from a chronic illness. I saw 5 different specialists over the past 2 years, and each one gave me different treatment. A couple even relied on information from the 70s and 80s. One even put me in the ER because he changed my treatment after I explicitly told him not to.

Another example: Back in my 20s, I injured my back. I took my MRI results to six different doctors - and I'm serious here, every one gave me a different diagnosis. In the end, I fixed it myself by doing my own research and treatment (2 years of physical therapy). One doctor said I had pain because spine fragments were lodged in my spinal cord (not true). Two of the doctors were even pushing me into invasive surgeries, and I'm so glad I told them no.

I don't understand the praise for doctors. If I had to generalize, I'd guess the majority give up learning after achieving wealth and status. seems like an art and not a science. I will emphatically welcome AI treatment.

If you don't respect the field in general, why do you think an AI amalgamation of that field to be better?
I respect the research and researchers - but medical researchers are far removed from medical practitioners.
Continuous improvement is the norm in cancer treatment, not in other areas where diseases are ill-defined and have a huge psychogenic component: back pain is the index case for that, so is TMJ dysfunction. In either case you might go from disabled to 'I have a bad day once a month' with a 20% change in habits and 80% change in attitude.

My dad, who worked in construction, got disabling back pain just in time for the Volcker-Freedman recession [1] His doc wanted to give him surgery which had the risk of being even more disabling, he said he'd go to a chiropractor, his doc said "if you don't do that then don't see me". I remember him taking me along for his chiropractor visits and getting a waterbed (bad idea.) He was on workman's comp at the time but got better around the time the economy got better and work was available again. Not to say he was consciously malingering, but work-associated pain has a lot to do with how you feel about your work.

[1] https://en.wikipedia.org/wiki/Early_1980s_recession

It's not, in my experience doctors are woefully behind the curve on the cutting edge, and even a bit hostile towards it.
Well, the difference is that people eventually die or retired so they are constantly being replaced
I think a one year gap in adoption of new tech is not that bad. Isn't it better to always go for the mature tech first ? The real change will come from the fact that because of AI, compute will be so cheap in the coming years: https://medium.com/thoughts-on-machine-learning/a-future-of-...
(comment deleted)
This is assuming that new technology will grow the same as in pre-LLM times, and merely be picked up a year late. But use of LLMs is likely to cause new developments to grow and spread slower, because of the reduced visibility. It may take much longer for a new development to gain currency to the extent that it becomes sufficiently visible in the training data. This also slows competition between evolving technologies.

In addition, as the article describes, the LLM services have biases built in to them even among existing technologies. It amplifies existing preferences, leading to less diversity and competition between technologies. Tech leads will have to weigh between the qualities of a technology on its own merits against how well it is supported by an LLM.

I know nothing about pathology, but in terms of software, I think slower adoption to new tech is what we need, especially when the "new tech" is just a 5% faster javascript framework.

By the way, for content creation, the only platfrom that really favors new creators is TikTok. Whether it leads to higher content quality is left for one's judgement.

That's not wrong. There is a lot of hype-driven development in the programming world. People are always jumping on the latest web frameworks and such. A little bit more stability is not a bad thing.

That being said, I think that people underestimate how fast LLM technology can evolve. At the moment, lots of training data is needed for LLMs to learn something. This may not always be the case. In 2 to 5 years, it may be possible to train an LLM to be helpful with a new programming language with much less data than is needed today. No reason to assume that the current situation is what things will be like forever. It's not like this technology isn't evolving incredibly fast.

After watching the entire world’s reaction to AI, at this point my conclusion is that hype driven development is human nature, and we just need to come to terms with that (but you will have to drag me kicking and screaming).
Maybe if you think artificially inflating the hype through massive ad campaigns, marketing campaigns, and shoe-horning AI into every product then yeah the world has a reaction to AI. It's mostly been meh, things like Apple Intelligence and Office Copilot have largely fallen flat.

If the hype was real none of these AI initiatives would be struggling to make money, but they are.

I don't really see it different than the artificial web3 hype, the only difference being that LLMs are use for extreme happy path scenarios.

The problem is that Apple intelligence is currently kinda useless. They rushed it into production in a misguided effort to "stay relevant". It may take a few years but we should eventually get useful personal assistant type AIs.

I would say LLMs are very useful for specific scenarios. They're also getting better. Just takes time to iron out the kinks.

> I don't really see it different than the artificial web3 hype

It's also little different than the .com bubble...

I think this teaches us that a thing can be hyped into the stratosphere, suck very much, crash and burn, and then go on to eat the world...

> I know nothing about pathology, but in terms of software, I think slower adoption to new tech is what we need, especially when the "new tech" is just a 5% faster javascript framework.

I hope that's not the definition people are using when discussing adoption of "new tech".

When it comes to the topic of AI and "new tech adoption", I think about something like the Rust programming language.

I apologize if it chafes the people reading this comment that I'm something of a Rust evangelist and I'm working from a point of view that Rust's very existence is a (large) net-positive when it comes to programming and how we think about programming language design.

My fear with AI tools in their current state is that it will slow down innovation in programming languages. Rust gained popularity because it brought things to the table that made writing safe, performant, and correct (thinking about the strong, expressive, static type checking) software much easier than it had been with the old incumbents (in certain domains).

But, if Rust were released today or in the near future, would it take off? If we could, hypothetically, get to a point where an AI tool could spit out C or C++ code and push it through some memory sanitzers, Valgrind, etc and just iterate with itself until it was very likely to be free of memory safety bugs, why would we need a new language to fix those things? I guess we wouldn't. And it wouldn't really matter if the code that gets generated is totally inscrutable. But, it saddens me to think that we might be nearing the end of human-readable programming language research and design.

It will be harder for new languages and frameworks. The AI Its exasperates the bootstrapping problem.

An interesting example is perl which is essentially static at this point (perl 6 was renamed and never got traction).

I know from experience running pipelines that those old perl scripts almost always work, where if I come across an old python script (2x) I will have to go in and make some adjustments. Maybe a library has changed too…

People like new shinny things though. Maybe the new languages will try to train the ai and release there own models, but that’s a huge lift.

Might be easier than you think. If DeepSeek can train a model cheaply, so you can you. Probably more cheaply as the technology and models get better.

People used to be worried that AI performance was going to degenerate if models are trained on AI slop, but it's been found that synthetic data is the bee's knees for coding, reasoning and such, so it may well be that a new language comes with a large amount of synthetic examples which will not just be good for AI training but also for documentation, testing and all that.

All you're talking about there in the end would be another compilation step.

I'm highly bearish on the concept of anything like that ever being possible (and near perfectly reliable) with llms, but if it were then it'd make sense as just another processing phase in compilation.

I'm also going to argue that Rust is a less AI-friendly language than, say, Go.

GC languages have many benefits that come from 'you don't have to think about memory allocation'. For instance you can just smack an arbitrary library into a Java program with maven and not think at all about whether the library or the application is responsible for freeing an object. The global problem of memory allocation is handled by a globally scoped garbage collector.

LLMs are great at superficial/local translation processes (e.g. medium-quality translation of 中文 to English doesn't require constraint solving any more than remembering which of the many indexing schemes is the right one for 'how do I look up this row/column/whatever in pandas') But fighting with the borrow checker (getting global invariants right) is entirely outside the realm of LLM competence.

a 5% _slower_ javascript framework
Surely 50% slower, compounding each year?

Jokes aside, I find it curious what does and doesn't gain traction in tech. The slowness of the IPv6 was already an embarrassment when I learned about it in university… 21 years ago, and therefore before the people currently learning about it in university had been conceived.

What actually took hold? A growing pantheon of software architecture styles and patterns, and enough layers of abstraction to make jokes about Java class names from 2011 (and earlier) seem tame in comparison: https://news.ycombinator.com/item?id=3215736

The way all of us seem to approach code, the certainty of what the best way to write it looks like, the degree to which a lone developer can still build fantastic products and keep up with an entire team… we're less like engineers, more like poets arguing over a preferred form and structure of the words, of which metaphors and simile work best — and all the while, the audience is asking us to rhyme "orange" or "purple"

The slowness to adopt IPv6 is because it's not a great design.

Going from 32-bits to 128-bits is complete overengineering. We will never need 128-bits of network address space as long as we are confined to this solar system, and the resulting addresses are extremely cumbersome to use. (Can you tell someone your IPv6 address over the phone? Can you see it on one monitor and type it into a different computer? Can you remember it for the 10 seconds it takes to walk over to the other terminal?)

48-bit addresses would have been sufficient, and at worst they could have gone with 64-bit addresses. This is already too cumbersome (9-12 base36 digits), but maybe with area-code like segmentation it could be rotated into manageable. 128-bits is just not workable.

> extremely cumbersome to use. (Can you tell someone your IPv6 address over the phone? Can you see it on one monitor and type it into a different computer? Can you remember it for the 10 seconds it takes to walk over to the other terminal?)

That's your idea of "extremely cumbersome"?

128 bit is exactly as hard as four groups of 32 bit.

Yes, it makes it untenable. There is no way I can remember 4 IP addresses at once.
Maybe I'm overlooking something but just looking at the past decade or so, a lot of new technologies and practices have been adopted. I assume most people would call these changes progress. So with this in mind, if in 10 years, we're by and large using the same technologies with AI injected in it, I feel that we would be missing something, as this article points out.

It's kind of sad to think that there may never be new technologies like Rust that break out and gain a critical traction. I'm hoping I'm wrong.

>5% faster javascript framework

Now you are bullshitting

You are correct he meant 1% faster
1% faster and 100% more complicated
More like 30% slower, 10% easier for trivial stuff and 70% more complicated for nontrivial stuff
I guess it makes sense to differentiate technological areas where we want progress at "any possible pace" vs "wait and see pace". I don't know if pathologists or other medical professionals feel the same about their field.

On a related note, are there any techniques for facilitating tech adoption and bringing all users up to speed?

Web development isn't your thing either i see ;)
Tiktok does not favour new creators, its users do. And only because it's a new generation of consumer for the most part, who want to consume content from their chosen platform. The same thing will happen with alpha.
Netflix and YouTube push new content/favor new creators or directors.

They don’t push the classics like music platforms do, and I don’t think it’s just streamers’ tastes.

(comment deleted)
>Pathologists as a specialty has been grousing about this for several years, at least since 2021 when the College of American Pathologists established the AI Committee.

This sounds like Moral Coating for what is otherwise protection of the Status Quo.

High paid doctors do not want to be replaced by AI. They will use every excuse to keep their high paying job.

> The entire music community has been complaining about how old music gets more recommendations on streaming platforms, necessarily making it harder for new music to break out.

Compared to what though? Compared to Limeware/Kaazaa back in the day, or compared to buying records in a store?

Personally, I find it easier than ever to find brand new music, mostly because Spotify still surfaces new things with ease for me (and always have been, since I started using it in 2008), and platforms like Bandcamp makes it trivial to find new artists that basically started uploading music yesterday.

Or compared to the days of radio, having labels decide what's on the mainstream and the indie college stations doing the unpaid work (and giving listeners the gift) of discovering "lost" hits.
Compared to Myspace. The difference for anyone who lived through it is night and day.
> Compared to what though? Compared to Limeware/Kaazaa back in the day, or compared to buying records in a store?

Compared to curation by other humans. Be it music labels, magazines, radio DJs, or a person sharing their playlist or giving you a mixtape.

In this model, tastes never overlap perfectly, so you're exposed to unfamiliar music fairly regularly, often in some emotional context that makes you more likely to accept something new.

Algorithms don't really do that. They could, but no one is designing them that way. If I listen predominantly to female vocalists on Spotify for a week, I'm only getting female vocalists from now on.

I don't get that. Its recommender has been great for me. And there are lots of playlist if I want to try something completely different.
> The entire music community has been complaining about how old music gets more recommendations on streaming platforms, necessarily making it harder for new music to break out.

I can understand other issues, but this has nothing to do with that. Models don't have to be re-trained to recommend new music. That's not how recommendation systems work.

This is the fault of the regulators. There’s no reason that new discoveries are not put in a queue to train a new AI and when there are enough to make it worth the run, you do the run and then you give the doctors old model and new model and they run both and compare the results.
> The entire music community has been complaining about how old music gets more recommendations on streaming platforms, necessarily making it harder for new music to break out.

Why does music continually entrench the older stuff (will we ever stop playing classic rock bands) whereas video streaming platforms like Netflix and YouTube try to hide/get rid of the old stuff?

I wonder if shows like The Office, Parks and Rec, Seinfeld, etc end up becoming the "classic rock" of streaming.
The main issue with AI, and ironically the reason why ChatGPT is the best one, is whom it works for.

AI doesn't work for the user. It couldn't care less if the user is happy or not. AI is designed first and foremost to make more money for the company. Its metrics are increased engagement and time on site, more sales, sales with better margins. Consequently, the user often has no choice or control over what the AI recommends for them. The AI is recommending what makes more sense for the company, so the user input is unnecessary.

Think of AI not as your assistant, but as a salesman.

One interesting consequence of this situation I found was that Youtube published a video "explaining" to creators why their videos don't have reach in the algorithm, where they essentially said a bunch of nothing. They throw some data at the AI, and the AI figures it out. Most importantly, they disclosed that one of the key metrics driving their algorithm is "happiness" or "satisfaction" partially gathered through surveys, which (although they didn't explicitly say this) isn't a metric that they provide creators with, thus it's possible for Youtube to optimize for this metric, but not for creators to optimize for it. That's because the AI works for Youtube. It doesn't work for creators, just as it doesn't work for users.

People are complex creatures, so any attempt at guessing what someone wants at a specific time without any input from them seems just flawed at a conceptual level. If Youtube wanted to help users, they would just fix their search, or incorporate AI in the search box. That's a place where LLMs could work, I think.

When you look at things this way, the reason why Netflix/Youtube get rid of old stuff has nothing to do with users, but with some business strategy that they have that differs from the music industry.

Working in Zed I'm full of joy when I see how well Claude can help me code. But when I ask Claude about how to use Zed it's worse than useless, because it's training data is old compared to Zed, and it freely hallucinates answers. So for that I switch over to Perplexity calling OpenAI and get far better answers. I don't know if it's more recent training or RAG, but OpenAI knows about recent Zed github issues where Claude doesn't.

As long as the AI is pulling in the most recent changes it wouldn't seem to be stiflling.

I tried uses chatgpt 4o to write a simple website that used the chagpt api. It always generated code for their deprecated API. I’d paste the error about using old calls, and it would recognize its error and.. generate old calls again.

Couldn’t ever use it owns api.

This makes me fear less for web development jobs being lost to AI, to be honest. Look, we can create new frameworks faster than they can train new models. If we all agree to churn as much as possible the AIs will never be able to keep up.
Until the AI groks the principle of deviation and starts generating frameworks faster than we do.
Sadly, as a person who used write AVX in C for real time imaging systems: don't care shipped.

I love dingling around with Cursor/Claude/qwen to get a 300 line prototype going in about 3-5 minutes with a framework I don't know. It's an amazing time to be small, I would hate to be working at a megacorp where you have to wait two months to get approval to use only GitHub copilot (terrible), in a time of so many interesting tools and more powerful models every month.

For new people, you still have to put the work in and learn if you want to transcend. That's always been there in this industry and I say that as a 20y vet, C, perl, java, rails, python, R, all the bash bits, every part matters just keep at it.

I feel like a lot of this is the js frontend committee running headlong into their first sea change in the industry.

Sonnet + Tailwind is something of a force multiplier though -- backend engineers now have a fast/reliable way of making frontend changes that are understandable and without relying on someone else -- you can even give 4o a whiteboard drawing of a layout and get the tailwind back in seconds

On the wider points, I do think it is reducing time coders are thinking about strategic situation as they're too busy advancing smaller tactical areas which AI is great at assisting -- and agree there is a recency issue looming, once these models have heavy weightings baked in, how does new knowledge get to the front quickly -- where is that new knowledge now people don't use Stackoverflow?

Maybe Grok becomes important purely because it has access to developers and researchers talking in realtime even if they are not posting code there

I worry the speed that this is happening results in younger developers not spending weeks or months thinking about something -- so they get some kind of code ADHD and never develop the skills to take on the big picture stuff later which could be quite a way off AI taking on

> backend engineers now have a fast/reliable way of making frontend changes that are understandable and without relying on someone else

backend engineers in this context could learn JS.

One thing that is interesting is that this was anticipated by the OpenAI Codex paper (which led to GitHub Copilot) all the way back in 2021:

> Users might be more inclined to accept the Codex answer under the assumption that the package it suggests is the one with which Codex will be more helpful. As a result, certain players might become more entrenched in the package market and Codex might not be aware of new packages developed after the training data was originally gathered. Further, for already existing packages, the model may make suggestions for deprecated methods. This could increase open-source developers’ incentive to maintain backward compatibility, which could pose challenges given that open-source projects are often under-resourced (Eghbal, 2020; Trinkenreich et al., 2021).

https://arxiv.org/pdf/2107.03374 (Appendix H.4)

ChatGPT and Gemini default to create-react-app, which has been considered poor practice for 2 years at least.
What's considered better practice? NextJS?!
Sure, or something like create-vite. Dealers choice.
Literally yes? https://react.dev/learn/creating-a-react-app lists several options because different people will click with different solutions. Or find one of the many other ones yourself of course, they're not hard to find, but cutting your teeth on an official recommendation before moving on to greener pastures is always a good idea.

The first one is always to learn from and then throw away.

It's also officially dead now (finally, lol).
>Consider a developer working with a cutting-edge JavaScript framework released just months ago. When they turn to AI coding assistants for help, they find these tools unable to provide meaningful guidance because their training data predates the framework’s release. [... This] incentivises them to use something [older].

That sounds great to me, actually. A world where e.g. Django and React are considered as obvious choices for backend and frontend as git is for version control sounds like a world where high quality web apps become much cheaper to build.

What if it happened just before React, and you therefore got stuck with angular? Should we now be stuck with React forever just because it's okay-ish, never allowing future better framework to emerge?
>What if it happened just before React, and you therefore got stuck with angular?

Still a good thing. :) The massive bump in developer market liquidity is far more valuable in my eyes than any inherent DevEx advantages. You'd still have much cheaper high quality web apps, although, if React truly has a technical advantage over Angular (doubtful), maybe not as much cheaper, but still much cheaper than pre-LLM.

If you truly want to figure out where I think the equation sign flips, it's probably like, pre-Smalltalk somewhere.

Or we ended up writing jqueryscript forever!

    $(function() {
      // yay
    })
I’m all for boring technologies but can we please at least use compiled languages with types
> A world where e.g. Django and React are considered as obvious choices for backend and frontend as git is for version control sounds like a world where high quality web apps become much cheaper to build.

Imagine you saying this twenty years ago. Would you still want to be writing your back-end in JavaBeans, your front end in VisualBASIC, and storing your data in Subversion?

VisualBASIC made much nicer (and more responsive) UIs than Electron, so this isn't the slam dunk you think it is.
We dunk on Electron too, don't worry.
The underlying tech choice only matters at the moment because as software developers we are used to that choice being important. We see it as important because we currently are the ones that have to use it.

As more and more software is generated and the prompt becomes how we define software rather than code i.e. we shift up an abstraction level, how it is implemented will become less and less interesting to people. In the same way that product owners now do not care about technology, they just want a working solution that meets their requirements. Similarly I don't care how the assembly language produced by a compiler looks most of the time.