237 comments

[ 3.4 ms ] story [ 234 ms ] thread
Interesting

are there any other good reads on the Economic impact of AI that is not just hype or marketing but more considered analysis of data / indicators?

I ve stopped believing any of this PR from AI companies that are trying to justify their huge valuations with some nebulous societal impacts that we have yet to see any trace of. Stop talking about the damn thing and show me the thing
the entire premise of this economic index is that they're showing you actual usage and insights from millions of anonymized claude conversations.
It's pretty great that the AI bros have gotten quieter now. It was frankly exhausting.

Instead of getting robots that do the laundry and clean the kitchen we got robots that do token work in a showroom at a BMW factory.

All the knowledge surfaced through LLMs was already mostly available online, they just make it more cohesive. It is better search.

Devs have figured out that creating a login page over and over is not a job, and that is now somewhat automatable.

Also everyone hates the name Devin now.

(comment deleted)
> Overall, we saw a slight lean towards augmentation, with 57% of tasks being augmented and 43% of tasks being automated.

I'd like to see a comparison to the data 6 months ago, before Sonnet 3.5. I suspect the automation rate will track up over time, but that may mostly be captured by API usage which isn't in the dataset.

AI companies: But don't worry, AI chatbots will create a UBI utopia where we will not do any work and we will have a future that will give us more time back and would never have to pay for work or food again! /s

Here's the reality: You are getting displaced.

Companies like Anthropic and OpenAI screaming about AGI are repeatedly lying to you as they raise more money while Meta (who are laying off staff today), Salesforce (announced layoffs as well) [0], Klarna (not hiring), etc are admitting this in front of us (and laughing at all of us).

Do you get it now? I'm giving you a 5 year head start of their plan before it becomes a complete catastrophe for the market. [1]

[0] https://news.ycombinator.com/item?id=42975813

[1] https://www.weforum.org/publications/the-future-of-jobs-repo...

I don't think I can eat the output of some AI chatbot...
Pretty sure things are gonna go far beyond chatbots into robots
Probably robotic avatars like those Tesla bots You can be a servant for the billionaire class without leaving your home! Actually your makeshift hobble, you can't afford housing.
It’s time to take up socialism, folks.
Yup. If you disagree that's fine, I'm 99% sure you just disagree about the words -- it's time to take up whatever "real" democracy means to you! A democracy where increased productivity doesn't make people worry that they'll paradoxically have less. A democracy where power is neither hereditary nor selfish, but rather a civic sacrifice. A democracy where human flourishing is the goal, not a side hustle.
I wouldn't expect Anthropic to have any special knowledge/insight in this area aside from the data they have on usage of their own tools. As such, I'd be much more interested to see some "Google Trends"-like data about Anthropic usage. Unfortunately, since AI is dynamic and competitive industry, I don't think anyone will be sharing information like that for a long time.
Honestly, simply releasing a graph showing the trendline of their own metrics on e.g. inference would speak volumes. They have this data already, 100%, its on some grafana dashboard the on-calls are watching every day. My suspicion is that some AI providers (OAI/Google) are watching these metrics go up-and-to-the-right quite consistently, but Anthropic's isn't doing that.
IMO it's a mistake to get too caught up in the (admittedly self-described) goal of modelling AI's economic impacts.

Instead, this is a super rare and valuable look into who/what/how folks are doing with Claude across millions of conversations, nicely categorized by function and task.

The economic impact data (i.e. wages) that they might overlay onto that usage data is a separate thing that -- of course -- is more subjective and likely to be part of some PR machinery about the public value of AI etc.

But as to sharing the raw usage data itself - we should applaud it! What a useful window into how this stuff is being used in the real-world.

Will OpenAI release similar data? Why or why not? I hope they will. It elevates the discussion for everyone, and frankly would be 'good business' if it gets people thinking about who/how AI could be used at their organization with more granularity.

To what degree can you really trust these data? They obviously have a financial interest in playing with the numbers.
I would say it is trust worthy because if it were found to be gamed then Anthropic’s reputation would crater.

But, we found out that OpenAI is/was gaming benchmarks (https://news.ycombinator.com/item?id=42761648) and that seems to be forgotten history now - so I don’t know.

> I would say it is trust worthy because if it were found to be gamed then Anthropic’s reputation would crater.

But on the other hand, how would we found out that they've gamed the numbers, if they were gamed? Unless you work at Anthropic and have abnormally high ethics/morals, or otherwise private insight into their business, sounds like we wouldn't be able to find out regardless.

Have you noticed that the numbers are all relative?

Unless they are trying to mislead competitors (who don't look at their own numbers...), they have no reason at all to game those numbers there.

I wish Anthropic luck.

The company seems to be operating in a classic failure mode: being more concerned with its industry than its competitors and customers.

See the first few points here: https://brief.bismarckanalysis.com/p/27-insights-from-three-...

Where I could be wrong is the CEO is technical, however most of what I hear from them is about industry and social impact instead of product.

> however most of what I hear from them is about industry and social impact instead of product

Have you considered that, since they are a public benefit corporation staffed with people who left OpenAI in part due to more capitalistic pursuits, this is by design?

I am saying they are choosing to operate inside a failure mode, not that they are doing so accidentally.
(comment deleted)
Seems like Anthropic has too much money on their hands and are looking for ways to spend it. It’s surprising to see lean AI startups accumulate fat so quickly. Usually this sort of wheel spinning is reserved for large corporations.

And it’s not just them. To me this trend screams “valuations are too high”, and maybe hints at “progress might start to stagnate soon”.

I don't see it. This is just an analysis of how Anthropic customers are using the product and what investment areas seem most promising in the future - why wouldn't they want that?
It's clearly more than an interesting tech blog post written by one of the data guys in their spare time. It's an "initiative".

That said, this doesn't seem like completely superfluous "fat" like what Mozilla does. It seems very much targeted at generating interesting bits of content marketing and headlines, which should contribute to increasing Anthropic's household brand name recognition vs. other players OpenAI, as well as making them seem like a serious, trustworthy institution, rather than a rapacious startup that has no interest in playing nice with the rest of society. That is: it's a good marketing tool.

My guess is that they developed it internally for market research, and realized that the results would make them look good if published. Expect it to be "sunset" if another AI winter approaches.

Even on the contrary, this is very important information to have, in order to understand your customer base and how sticky you are with them, what features you need to focus on, etc etc
The exact opposite. Relative to ChatGPT Anthropic has an enormous "brand problem." What they should be doing is exclusive deals like this, but with deals with large publishers on a recurring basis and figure out how to teach consumers who they are and how to use them best. For like 99% of the use cases all these products are parody and the real business gains are finding a way into consumers lives.

Semi-relevant sidenote: ChatGPT, spent $8m on a super bowl commercial yesterday just to show cool visualizations instead of any emotional product use case to an ultra majority audience has never had a direct experience with the product.

These companies would be best served building a marketing arm away from the main campus in a place like LA or NY to separate the gen pop story from that of the technology.

I disagree. I think Anthropic, like the other big players, is trying to get some of that government money. Releasing policy-adjacent papers seems like a way to alert government officials that Anthropic ought to be in the room when the money starts changing hands.
I am inclined to agree. If you’re at the precipice of automating or transforming knowledge work and the value for being the first is nearly infinite (due to “flywheel effects”), why would you dedicate any energy to studying the impact of AI on jobs directly? The thesis is everything changes.

I think AI in its current iteration is going to settle into being like a slightly worse version of Wikipedia morphed with a slightly better version of stackoverflow.

I think that strongly underestimates the impact LLMs, especially reasoning models, have on how code is written today.
Educate me. I find them useful but they are less so when you try to do something novel. To me, it seems like fancy regurgitation with some novel pattern matching but not quite intuition/reasoning per se.

At the base of LLM reasoning and knowledge is a whole corpus of reasoning and knowledge. I am not quite convinced that LLMs will breach the confines of that corpus and the logical implications of the data there. No “eureka” discovery, just applying what we already have laying around.

> find them useful but they are less so when you try to do something novel.

Well over 90% of work out there is not novel. It just needs someone to do it.

I think the trap people fall in is that LLMs don't need to be novel or reason as well as a human to revolutionize society.

Plenty of value is already added just by converting unstructured data to structured data. If that is all LLMs did they would be still be a revolution in programming and human development. So much manual entry and development work has essentially evaporated overnight.

If there was never a chat based LLM "agent" LLMs just converting arbitrary text to structured JSON schema would be the biggest advancement in comp sci since the internet. There is nothing equivalent that existed before except for manual extraction or rule based hard coding.

Judging LLMs based on some criteria of creativity or intuition from a chat is missing the forest for the trees.

Let's say I can't fully disclose the details because it is an area I am actively working on, but I had an algorithmical problem that was already solved in an ancient paper, but after a few hours of research I could find no open implementation of it anywhere. I thus spent quite some time re-implementing this algorithm from scratch, but it kept failing in quite a few edge cases that should have been covered by the original design.

Just to try it out, I uploaded the paper to DeepSeek-R1 and wrote a paragraph on the desired algorithm, that it should code it in Python and that the code should be as simple as possible while still working in exactly the way as described in the paper. About ten minutes later (quite a long reasoning time, but inspecting the chain of thought, it did almost no overthinking, but only reasoned about ideas I had or should have considered) it generated a perfect implementation that worked for every single test case. I uploaded my own attempt, and it correctly found two errors in my code that were actually attributable to naming inconsistencies in the original paper that the model was able to spot and fix on the fly. (The model did not output this, this I had to figure out myself.) I would have never expected AI to do that in my lifetime just two years ago.

I don't know whether that counts as "novel" to you, but before DeepSeek, I also thought that Copilot-like AI would not be able to really disrupt programming. But this one experience completely changed my view. It might be the case the model was trained on similar examples, but I find it unlikely just because the concrete algorithm cannot be found online except for the paper.

clearly NOT novel as you so clearly explained, "an algorithmical problem that was already solved in an ancient paper"
Well, of course. Realistically, I would not expect AI systems like this to be very useful for novel cutting-edge scientific results, proving mathematical theorems etc. in the next few years.

But this is not the majority of what software developers are doing and working on today. Most have a set of features or goals to implement using code satisfying certain constraints, which is what current reasoning AI models seem to be able to do very well. Of course, this test was not rigorous in any meaningful way, but it really changed my mind on the pace of this technology.

This fits my experience. When the information is encoded somehow already, LLM’s excel at translating to another medium.

Combined with the old “nothing new under the Sun” maxim, in that most ideas are re-hashes or new combinations of existing ideas, and you’ve got a changed landscape.

Because that research helps you understand your market and where the value generation is. This can expose where to better invest.
A lot of assumptions there. Why isn't Ford the only motor company?

And if the flywheel is that AI begets AI exponentially in an infinite loop then those share certificates you own probably won't be worth much. The AI won.

Coincidentally, Anthropic's mission is AI safety.

Anthropic is a Public Benefit Corporation whose governance is very different from a typical company in that it doesn’t put shareholder ROI above all else. A majority of its board seats are reserved for people who hold no equity whatsoever and whose explicit mandate is to look out for humanity.

https://www.anthropic.com/news/the-long-term-benefit-trust

https://time.com/6983420/anthropic-structure-openai-incentiv...

Put this and 'dont be evil' and 5 dollars in my hand and I'll give you a cup of coffee.
Coffee for $5? That's a steal in this economy!
The coffee is made with the assistance of AI, which means some nonzero portion will be something other than coffee, but at least it means every sip is an adventure.
This is one of the funniest takes on ai I've read, it could've been out of a videogame like the outer worlds with its absurd takes on crapitalism

It's not the best choice, it's spacer's choice!

Isn't there an SCP where occasionally it spits out liquid magma or strange matter or something?
(comment deleted)
This is why I cancelled my chatgpt subscription and moved to claude. Its kinda silly, but I feel like the products are about equivalent for my use case so I'd rather do business with a company that is acting in good (better?) faith.
Don't think that's silly at all.
Hope not - I haven't purchased a Nestle brand in years for this exact reason.
In the case they don't get high salaries from this activity, there is also a solution. The next step in ~10 years could be to offer their services to governments to offer "automated court decisions".

Then the people who funded / trained this "justice" out of their good heart, would actually have leverage, in terms of concrete power.

It's a much more subtle way to capture power, if you can replace the judges with your software.

I guess they can get them to rewrite the US Constitution to remove that pesky "fair trial" bit and, since they would control the narrative, delete 1000+ years of common law.

Brave new world, indeed...

Anthropic pays their engineers pretty well. They're doing just fine, at least for as long as people are pouring money into their company. But that's everyone in this space, isn't it?
Thanks but no thanks.
That isn't silly, that's one of the only ways to exercise agency under hypercapitalism. I recently cancelled my Amazon Prime membership and got a Costco membership for the same reason. I don't get every product I want, but I'm also okay with that.
This has to be a meme. Costco is peak hypercapitalism lol.
Could you say more?
It's a 500B company that undercuts everyone else with incredible efficiency, just like Amazon. It's an example of how capitalism can be great. If you really want to get of out of capitalism, you can just buy directly from farmers or grow your own food.

The whole thing about no ethical consumption under capitalism is a just a way to enjoy the conveniences of capitalism on a moral high ground. It's totally doable, you just might not enjoy it haha.

I guess the angle I was coming at it from is that they pay their employees a living wage. I need to buy toilet paper from somewhere, and between Amazon and Costco I would much rather give my money to Costco.
The secret is buying a bidet so you dont need to buy from either ever again!
Hell, just buy from Wallyworld where you get low, low prices and pseudo-socialism with their employees on the food stamps.

The camel's gotta get its nose in the tent somehow.

I'm not sure if you are being sarcastic or not, but the practical upshot of this new "Public Benefit Corporation" thing, with or without a trust or non-profit attached, is that you can tell both the public and your investors to fuck off. The reason why all the big AI startups suddenly want to use it is because they can. Normally no sane investor would actually invest in such a structure, but right now the fear that you might be left out of the race for humanity's "last invention" is so acute that they do it anyway. But if Dario Amodei actually cared about humanity any more than Sam Altman, that would be the surprise of the year to me.
Can you imagine a hypothetical AI company that did care about humanity, and if so, how would it look different from Anthropic?
It wouldn't be doing this: https://investors.palantir.com/news-details/2024/Anthropic-a...

It wouldn't specifically brag about doing it, while leaving out that they were specifically dealing with Palantir, because they know what they're doing is unethical: https://www.anthropic.com/news/expanding-access-to-claude-fo...

Being available for use by militaries is incredibly irresponsible, regardless of what scope is specifically claimed, because of the inherent gravity of the situation when a military is wrong. The US military maintains a good deal of infrastructure in the US; putting into their hands an unreliable, incompetent calculator puts lives at risk.

It would be structured as a non-profit (there are no teeth to a PBC; the structure is entirely to avoid liability, and if you have no trust in the executive body of an organization, it has zero meaningful signal).

It would have a different leadership team.

It would have a leader who could steelman his own position competently. Machines of Loving Grace was less redeeming than Lenat's old stump speeches for his position, despite Amodei starting up in an industry significantly more geared for what he had to say, and Lenat having an incredibly flexible sense of morality. Its leader would not have a history working for Chinese companies and jingoistically begin advocating for export controls.

It would have different employees than the people I know who are working there, who have a history of picking the most unethical employers they can find, in a fashion not dissimilar to how Illumination Entertainment's "Minions" select employers.

You seem to misunderstand benefit corporations. They remain committed to profit and are just as subject to their board and officers as any other corporation.

There are sane investors that prefer investing in companies that adopt these corporate structures. Based on data, those investors see public benefit corporations as more profitable and resilient. They are able to attract employees and customers that would otherwise not be interested or might be less interested.

The attempt is commendable, but the agency problem is well understood and none of these alternative structures have really solve for it.
> the agency problem is well understood

What is "the agency problem"?

Very generally it is (https://en.wikipedia.org/wiki/Principal%E2%80%93agent_proble... ) about the conflict of interest between agent( people taking action) and principal (the entity or person on behalf of whom action is taken)

In modern management compensation theory (https://saylordotorg.github.io/text_introduction-to-economic... ) this is key to why executive compensation has increased much faster than workers in the last 50 years.

Stock based compensation mix evolved from this thesis, and quite common in the valley and why almost all OpenAI staff wanted Sam Altman back even though the non profit board did not.

Aligning key talent's compensation to enterprise value is only viable in unrestricted for profit entities any other structure with limits (capped profit, public benefit corporation, non profit, trust, 501c's etc) does not work as well.

Talent will then leave to a for-profit entity who can offer better compensation than a restricted entity can because they share a % of their enterprise value which restricted ones either cannot or not have same liquidity/value [1] etc.

---

[1]This is why public companies are more valuable for RSU/options than private companies, and why cash flow positive companies like Stripe still raise private money to just give liquidity to employees .

Understating who is using your product is wheel spinning?
idk, the models themselves are quickly becoming a commodity. it makes sense to spend money figuring out go to market rather than just improve the models themselves.
And yet they don't have the resources to let job applicants know when their application was unsuccessful. You just get an email after you applied saying: "We may not reach out unless we think you are a strong fit for the role you applied to. In the meantime, we truly appreciate your patience throughout our hiring process." They also tell you not to use AI in the application.
They only have like 500 employees. And you could argue this is part of their stated mission.
I would argue this is within their overall objective. It’s not like Stripe creating a publisher (??)
We live in a world where there's a lot of talk about how AI might impact societies and economies - but little actual data. To me it seems very worthwhile to try to add 'any' data to that discussion and track how things change over time. Are reports of economic or labour trends pointless? Should companies not track how people use their products? I don't think it costs Anthropic much to do this - it's work for a couple of people to analyze their database.
(comment deleted)
Potentially a good source of startup ideas: Look where AI is being underutilized vs its economic potential and go there.
Pretty clear the answer is the hi-tech industry.
Not really any surprises here, if you’ve been following this stuff. I’d be much more interested in understanding what is holding up penetration of Claude and the rest into medicine, finance, and law.

This may just be my ignorance, but it seems that distributed version control is a highly valuable technology which hasn’t penetrated that well into law. If this is true—my evidence is only anecdotal, talking with lawyers—then it should provide partial insight that translates into the problem of LLM adoption.

Its a public benefit research company. Expecting it to behave like a normal Corporation is missing the point...
It's nice that their public benefit charter terms are actually public unlike Blue Sky's, where bsky can have anything in there even just minor benefits stuff and still be pretty much a normal corporation.
Feels like a page of graphs where Anthropic team discovers that Claude is the best coding model and is used mostly by devs. And they have no penetration in general population compared to OpenAI
Read it as that as well, I think this mixes the "there is no penetration in that market" with "we have not been able to get these people to use our tools".
Does OpenAI have similar data available to compare to?
I laughed out loud at their chart showing that Sonnet had a higher share of coding questions, whereas Opus had more writing.

If they would just look at their product, they'd see that it literally says it in the model description that Opus is better for writing. If you advertise one of your models as geared for task X, the insight that people use it more for task X isn't really an insight.

Everyone in the comments seem to not like the article or see it as a waste of time. I just don't think we are the audience they wanted for this, I think they want to show the average business owner the realistic potential and public (journalists that will distill this later) they are aware of the impacts and what to expect.

I don't read it as fear AI, I read change is happening because of AI.

They also call out how it's more used for augmentation rather than full automation which will address some concerns the public has.
Tools empower those with knowledge further than those without knowledge. The fact that people were concerned layman were simply going to be able to take on experienced programmers at their day jobs was farcical.
I'm not a dev but I thought the issue from a senior dev perspective was AI on AI, not layman on AI?
It seems like you get downvoted, but I think you touch on something important. IMO, right now there are two limitations with AI replacing experienced developers: a) It's not good enough at programming. It sometimes goes down rabbit holes and cannot get out. In other cases it comes up with ridiculously complicated solutions that could be solved much simpler. b) Making assumptions instead of gathering requirements.

I suspect that a) will get better over time. I also suspect that b) can be addressed by a pre-programmed prompt-flow that uses a LLM to gather requirements from a PM and ask probing questions to get a well-defined scope and agree on how edge cases should be handled. It doesn't seem far-fetched that a AI also would be able to call out small requirement changes that might allow for much simpler/faster solutions.

That is also what I think will happen, I mean these tools right now are just capturing market share with funding rounds and hacks surly? wrapping any foundational modal and trying to scope it down is always going to suck, but once people start to truly nail just training in only the information required to do that job (I described 03-high-mini or whatever it's called as a dumb finance bro with no depth to my wife) and then couch the task LLMs with orchestration LLMs, surly things improve?
I honestly don’t know who the audience could be, other than “people who like to tell others they’re in the know because they read AI companies’ press releases.”

At no point do I see an actual elevator pitch/tl;dr/summary of what the frak this index actually is, except that it’s part of some effort to track AI adoption. It just rains down figures about which industries are using how much AI without first grounding the new concept they’re introducing.

When you say you have a new economic index, you need to give me a number, how I should interpret that number, and where it comes from. I don’t see that.

GDP: measure if a country’s total economic output by adding up end product purchases.

CPI: general price level by taking a weighted average of prices throughout the economy

Big Mac index: how expensive goods are in a country relative to the US by reference to the local cost of a Big Mac, converted through the exchange rate.

Here I expect something like “the economic output-weighted fraction of production taken over by AI”, but instead it’s just a list of AI adoption by industry.

Why introduce an index and not headline with a definition of an index? Which audience prefers that?

Independent of whatever this org does, it makes perfect sense for economists to start to track when wage labor can be gotten rid of (in line with the AI Hype).
Awesome that there are releasing this paper and the associated data. I hope they'll do this regularly, so that changes can be tracked.

One thing I hope they'll correct going forward is inclusion of API usage. Anecdotally, I only use Anthropic models via Cursor. So none of that usage shows up in here. I'd expect that specialized tools/interfaces like Cursor will grow and thus more usage will shift to API. It would be a shame to miss out on that in the data set.

haven't they committed to not training on / using data submitted through API?
Tools like cursor have to be huge chunk of their traffic. I mean Cursor and Sonnet are like two peas in a pod.

Even if they don’t train on the data they could break it down by user agent / API client ID and infer something about cursor traffic.

I expect more 'automation' to happen through the API than 'augmentation'.
(comment deleted)
(comment deleted)
Incredible how low usage is among lawyers. Does anyone have any intuition on why?
Lawyers are selected for critical thinking skills and they aren't vulnerable to AI hype the way relatively poorly educated computer guys are.
Because LLMs make things up and the lawyer is liable for using that made up information.
My guess is bc hallucinations in a legal context can be fatal to a case, possibly even career endu g — there’s been some high profile cases where judges have ripped into lawyers pretty destructively.
Part of it is selection bias, Claude is much less general-audiences than ChatGPT. But any lawyers using LLMs in 2025 deserve to be disbarred:

"A Major Law Firm's ChatGPT Fail" https://davidlat.substack.com/p/morgan-and-morgan-order-to-s...

"Lawyer cites six cases made up by ChatGPT" https://arstechnica.com/tech-policy/2023/05/lawyer-cited-6-f...

"AI 'hallucinations' by ChatGPT end up costing B.C. lawyer" https://www.msn.com/en-ca/news/world/ai-hallucinations-creat...

The list goes on and on. Maybe there's a bespoke RAG solution that works...maybe.

> But any lawyers using LLMs in 2025 deserve to be disbarred

In what year would you think it will be acceptable and why?

LLMs are tools, I don't see anything wrong with using them in any occupation as long as the user is aware of the limitations.

no - some Judge wrote to his family member recently.. " I am seeing all these great briefs now " followed by a novice discussion of AI use. This is anecdotal (recent), but it says to me that non-lawyers, with care, are writing their own legal papers across the USA and doing it well. This fits with other anecdotes here in coastal California for ordinary law uses.
i think they're especially likely to hallucinate when asked to cite sources, as in they're mostly prone to making up sources, and a lot of the work my lawyer friend have asked of chatgpt or claude requires it to cite stuff, and my friend has said it has just made up case law that isn't real. so while it's useful as a launching point and can in fact be helpful and find real case law, you still have to double check every single thing it says with a fine tooth comb, so its productivity impact is much lower than code where you can clearly see whether the output works immediately
Assuming that AI adoption is more prevalent in the technical domain, could this be one of the reasons why it is leaning towards computer and technical usage?
I’m not surprised most of their usage is for coding tasks, but I wonder if that reflects a kind of selection bias.

HN and programming subreddits rave about Claude for coding, so it’s possible that a lot of developers use Claude for coding, but the average AI use case may weight differently on ChatGPT or Grok.

In my experience, if ChatGPT can’t solve a coding problem, I try again on Claude. Although this happens less frequently since upgrading to o1-pro and o3-mini-high. And I haven’t used Claude for anything else.