Ask HN: Why there are no actual studies that show AI is more productive?

If AI makes people so much more productive, why aren't there much more apps on the Apple store? Mobile apps involve a lot of dirty, boring scaffolding work which AI automated first thing, 2 years ago easily. It should've been the very first place where productivity boost should've been evident, a year ago at least. But it's just not there. Why not?

AI can build systems based on static assumptions that the orchestrator (you) gives it. But proper engineering (which is what matters economically much more) is the process of the system's assumptions & requirements changing over time to ensure you have a reliable and consistent service - and that's not something that AI excels at (yet).

Some people prefer evidence before investing large amounts of money and labor. That is not an indication of irrational behavior even if challenging your emotionally invested opinion or result.

There are a few studies that show perceived increases in productivity (all of them show negativ or almost no real increase, but I don't that is relevant to snake oils salesman).

We've had the AI tools for maybe two years, and they have only gotten really good in the past half a year or so. For fuck's sake, adopting electricity took like 50 years, why would you expect to see any kind of effect from the AI so quickly? The tools are still developing - rapidly - and people are still figuring out the best usage patterns for it.

> Why there are no actual studies that show AI is more productive?

Beats me. With "AI" being so good at faking stuff, there should by now be ton of such studies :)

These sort of things are really hard to study. Combine that with the fact that the AI landscape is so varied and fast moving... It's easy to see why there aren't many studies on it.

There are a mountain of things that we reasonably know to be true but haven't done studies on. Is it beneficial for programming languages to support comments? Are regexes error-prone? Does static typing improve productivity on large projects? Is distributed version control better than centralised (lock based)? Etc.

Also you can't just say "AI improves productivity". What kind of AI? What are you using it for? If you're making static landing pages... yeah obviously it's going to help. Writing device drivers in Ada? Not so much.

Dora released a report last year: https://dora.dev/research/2025/dora-report/

The gains are ~17% increase in individual effectiveness, but a ~9% of extra instability.

In my experience using AI assisted coding for a bit longer than 2 years, the benefit is close to what Dora reported (maybe a bit higher around 25%). Nothing close to an average of 2x, 5x, 10x. There's a 10x in some very specific tasks, but also a negative factor in others as seemingly trivial, but high impact bugs get to production that would have normally be caught very early in development on in code reviews.

Obviously depends what one does. Using AI to build a UI to share cat pictures has a different risk appetite than building a payments backend.

I think most major efficiency improvements involve more adaptation costs than expected.

Those that can “see” the potential push through the adaptation period, even when longer than expected.

Depending on how forward looking a group is, the adaptation costs are a problem, a dilemma, or a completely obvious win.

Yet, external measurements don't distinguish between accumulating, accelerating, flat or fading intermediate value.

--

Avoidance of necessary adaptation, even with no immediate impact, becomes the dual. Technical, strategic, or capability debt.

Does that hidden anti-productivity ever get accounted for? When maladaptive firms take their anti-productivity into a hole as they fade/demise?

A company can operate with high margins while its sales fall off a cliff. Is that just "decreasing quantities" of uniformly "high productivity"?

So... you want a study to prove your ready made hypothesis?

It might also depend on how the tools are used. In practice a lot of value seems to come from reducing small bits of friction rather than dramatically increasing output.

because you can just look at the commit log

Just trust the vibe, bro. One trillion market cap cannot be wrong.

Ask HN: Why are there no actual studies that show the sky is green and the earth is at the centre of the universe?

I would have included the flatness of earth, but the flat earthers have some excellent studies (reviewed by their flat earth peers) on the subject.

Because we are incapable of measuring developer productivity.

GitHub has their own study using Copilot but given the obvious conflict of interest I would discount it.

Self-reported productivity does not equate to actual productivity. People have all sorts of biases that make such assessments fairly pointless. They only gauge how you feel about your productivity, which is not necessarily a bad thing, but it doesn't mean you're actually more productive.

> Many won't care unless you show them an actual study

Why are the pro AI people so obsessed with proving the AI skeptics wrong.

Is AI is working for you? Great. Go make great things. Isn't that the point after all? Who cares who believes you if the results speak for themselves?

The code was never the bottleneck. It’s always the org around it.

Because the data is private and often such studies are not measuring solely the part that AI makes more productive. And measuring productivity in general is a very hard problem so the results of whatever study often are meaningless in practice. Pair this with studies today still being based off ancient models like GPT-4o and it's even more meaningless.

If you are familiar with AI it's obvious how it increases productivity. When bugs get fixed with 0 human time it's plain as day that it was productive compared to a human making the fix.

How do you know you're more productive? Humans are excellent at fooling themselves, and absent a metric (or multiple metrics) by which you can measure your productivity, you can't be sure you're actually being more productive.

Why are we even discussing this before the theft problem has been solved? Or the energy consumption?

If anything, there needs to be studies done on

- the drop in creative, novel output from actual people (due to theft and loss of jobs)

- the energy cost per pax in relevant industries, pre/post LLMs being adopted

Surely, current openclaw has show AI's productive. More and more common person use it to change their lives, amazing

I believe that individual productivity in most areas peaked long ago. Industrial production is still scaling up, and this is the model that applies to AI, or as it realy is, automation of "management", but as this is NOT a linear mechanical process,(almost, oh! so almost mechanical), it is not quite working.For exactly the same reason that industry can not make you one ,lets say,car, that is green on one side, but orange on the other, and has six headlights, but only one seat, industry cant scale down, minimum order is 250000 units, it will take 3 years, pay us now! I deal with this every week, something small,(smol), breaks, in a large corporate environment, they work in millions, they have teams, and departments, but the little handle thing on a set of automated front doors facing a main street in a significant asset, has failed, and watching the whole corporate aparatus convulse as they try and figure out how to pay for something smaller than a rounding error to a company that barely exists, and needs to be passed higher and higher to be approved as there is no button, just like a major corporate deal. People cant figure this out, AI never will. And I am exploring just how to exploit this scaling problem to my advantage.