I could care less about bot sitting (haven’t we always written our own automation?), but it’s botsitting the unverified slop that people send you that fuels frustration. I thought I worked with competent people who respected me
Your coworkers haven't changed. What changed is that people can hand off work they never had to think through themselves. So you don't know what they checked and you don't know what you need to. You just have to read the whole thing.
Understanding what is going on with AI productivity is … frustrating to say the least.
The best I can say is that genAI is a self reported a 20% efficiency boost, and for a very (very) small group of people, it’s maybe a 2-3x boost. (And if you are at a frontier lab, you go fly into the big bucket of exceptions)
At this point, for most use cases, AI productivity is either the equivalent of giving people 3D printers, and seeing little benefit, or signing up for an outsourcing service, just without the development of human capital anywhere.
>Understanding what is going on with AI productivity is … frustrating to say the least.
Agreed. I think one of the hardest things about it is that productivity != value. You can push all the code you want, but if it's not driving revenue up or cost down, it doesn't matter economically.
Here is the best data I've been able to find. An observational study of 4000 teams over 2 years across many different organizations. Data gathered from their task management, version control, and CI/CD tooling. Critically - this is not survey data. It's much more direct measurement.
6 hours a week is low, unless its the average spread across industries. I think I spend more time in Claude Code via the CLI versus any other app I have on my laptop.
Like others said, the frustration is when it gets something so wrong you just think "wow, how'd you mess that up?" but when it gets it right its kind of nice. I also dont like that I basically tell Claude what to do, and then either go to busy work or waste time on the internet.
Working with AI is trying to reduce the probability it'll pick undesirable paths. It's an exercise in trying to avoid what you DON'T want.
I suppose it's the same as asking someone else to take care of a feature and hoping they understand what you have in mind. The difference is that there's a lot of context that's shared between you and a human developer that is simply absent with AI.
I wonder how many hours of old stye non-overhead that is. And by overhead I mean mostly meetings. Then again maybe generating paperwork is included so that eats bit into it.
I've found that setting good guardrails, and running in a sandbox so that the agent doesn't keep asking tedious permission questions, makes things go a LOT smoother.
Generally, I spend anywhere between 15 mins and an hour setting things up (depending on how well the project is set up for AI work), and then set the agent going, coming back in a half-hour to an hour to check its progress. Generally, the tooling keeps it honest (for golang, forbidigo is AWESOME). 80% of the questions the agent asks me require a lot of thought. 20% of what it does needs correction.
The other thing to remember with LLMs is that they are NOT human, and won't react in a human way. So you'll see strikes of "brilliance" followed by the absolutely bizarre. But good guardrails keep that to a minimum.
> sandbox so that the agent doesn't keep asking tedious permission questions
> 80% of the questions the agent asks me require a lot of thought. 20% of what it does needs correction.
I've found even the permissions questions give me veto power over fruitless lines of exploration, especially in planning mode. For instance, it wants to use tools I don't have installed to access information that I have made available elsewhere? I get a chance to override this decision by declining the permissions check and redirecting it. Feels tedious, but helps me understand what information sources are influencing it. I head off a lot of bugs this way.
i've seen a number of articles claiming things like "devs self report they'er +x% more productive with AI, but actually they're -y% LESS efficient!". and i think that this is explanation for why.
as a boss (or researcher) i'm going to measure productivity based on amount of output per hour that i'm paying you; as a workers, i'm going to measure productivity based on amount of output relative to the amount of effort i'm putting in.
so what may be happening is that bosses see that output is at 80% (productivity down!) but workers see that they can give that 80% output with 40% effort (productivity up!).
I spend at least 6 hours a week arguing with bots owned by other teams, as I’m unable to reach a human before I bypass their bot. 10k person company, clients are paying for my time.
In some cases, workers are also being asked to automate the parts of their jobs they enjoy most, Hinds said on the podcast, pointing to customer-service employees who enjoy building relationships but are increasingly expected to supervise AI agents instead.
"That's what gives you joy and meaning at work," she said. "That is very dangerous."
What's a 20% productivity gain if I constantly feel deflated by work that used to energize me? That's going to give back the productivity gain and more, while also decreasing my quality of life.
I don't see a lot of talk about how AI development breaks the old feedback loop of write code, watch it run, change it, repeat. I really hate sitting around waiting for the agent to get done planning, reading the plan, then waiting for the agent to get done coding. It's those 5-10 minute windows when its working that really sap my patience and suck all the fun out of our jobs. Writing code by hand is just more fun.
My challenge has been trying to manage my higher-level context. I've gotten a pretty good setup where I have project-level orchestrator agents that can spin up workers to implement tasks with minimal oversight, and the resulting work is usually quite good (especially after I give it the mandatory "make the comments less verbose" refining, etc.). But that means I'm doing even more context-switching. I've gotten to the point where I have a half-dozen draft PRs that just need my review before I tag my colleagues, and trying to dig up the context from all of those tasks can be paralyzing.
This kind of reminds me of an article that I saw on HN ages back, there's like a subset of office workers who automated their Excel jobs, and just show up to work, read books, and do literally anything, while Excel does their work for them, and they collect their paycheck.
I just started using Claude Code for my work as a sysadmin. For my work, it's great. I don't need to wrestle with MySQL joins, claude gets even the most complex ones right WAY faster than I would. Same with new Terraform stuff. Things that would have taken me a day are cut to less than an hour.
So for my work, it's made me much better at my job. Much faster and more accurate.
For me, AI can sometimes create a false sense of productivity. It's similar to how in the past, people would spend time creating the perfect setup with notion templates, pomodoro timers and productivity tools, or tweaking their environment for maximum productivity, instead of actually doing productive work.
But now it's happening at the company level: "We're going to add a chatbot to increase productivity! Now MCP tools! Then agentic workflows! We’ll add skills, and now productivity will go up! Maybe loops will do it?"
You pay per token, even on subscription models the limit is tokens.
If I was valued at 1 trillion dollars, and I was in the hole enough to sink a couple small countries' GDP, maybe I would slowly start to optimize to maximize token usage.
I want to sell tokens, how do I sell more tokens? Not by doing the same work in less tokens, that's for sure.
This is like if you pay me by the hour and then excitedly tell me that you keep paying 10k a month and it's great. I will most certainly not work faster, in this hypothetical, if you tell me you love spending money because it gives you a dopamine rush. I would probably spend a couple more hours REALLY thinking about the task, maybe writing some docs nobody will read, maybe considering multiple options, doing benchmarks, doing research, and then later maybe ill do the actual task as well.
Im not saying these AI companies are scamming us, but the incentives are there and extremely clear. The only thing currently holding it back is that there is some vague kind of competition.
I don't know what they're complaining about. AI has freed us from the drudgery of craftsmanship, letting us focus on the important stuff—managerial and administrative work!
(There's a reason why I call it the MBA's stone. It transmutes all knowledge work into a problem of management.)
50 comments
[ 0.20 ms ] story [ 61.6 ms ] threadThe best I can say is that genAI is a self reported a 20% efficiency boost, and for a very (very) small group of people, it’s maybe a 2-3x boost. (And if you are at a frontier lab, you go fly into the big bucket of exceptions)
At this point, for most use cases, AI productivity is either the equivalent of giving people 3D printers, and seeing little benefit, or signing up for an outsourcing service, just without the development of human capital anywhere.
Agreed. I think one of the hardest things about it is that productivity != value. You can push all the code you want, but if it's not driving revenue up or cost down, it doesn't matter economically.
Here is the best data I've been able to find. An observational study of 4000 teams over 2 years across many different organizations. Data gathered from their task management, version control, and CI/CD tooling. Critically - this is not survey data. It's much more direct measurement.
https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways
Faros argues that teams are seeing about a 16% throughput improvement (PR merge rate) with heavy AI use.
I argue here that their data actually indicates negative absolute impact on throughput.
https://unessays.substack.com/p/talk-is-cheap
I’ve been told before.
Like others said, the frustration is when it gets something so wrong you just think "wow, how'd you mess that up?" but when it gets it right its kind of nice. I also dont like that I basically tell Claude what to do, and then either go to busy work or waste time on the internet.
I suppose it's the same as asking someone else to take care of a feature and hoping they understand what you have in mind. The difference is that there's a lot of context that's shared between you and a human developer that is simply absent with AI.
Welcome to the factory!
Generally, I spend anywhere between 15 mins and an hour setting things up (depending on how well the project is set up for AI work), and then set the agent going, coming back in a half-hour to an hour to check its progress. Generally, the tooling keeps it honest (for golang, forbidigo is AWESOME). 80% of the questions the agent asks me require a lot of thought. 20% of what it does needs correction.
The other thing to remember with LLMs is that they are NOT human, and won't react in a human way. So you'll see strikes of "brilliance" followed by the absolutely bizarre. But good guardrails keep that to a minimum.
> 80% of the questions the agent asks me require a lot of thought. 20% of what it does needs correction.
I've found even the permissions questions give me veto power over fruitless lines of exploration, especially in planning mode. For instance, it wants to use tools I don't have installed to access information that I have made available elsewhere? I get a chance to override this decision by declining the permissions check and redirecting it. Feels tedious, but helps me understand what information sources are influencing it. I head off a lot of bugs this way.
as a boss (or researcher) i'm going to measure productivity based on amount of output per hour that i'm paying you; as a workers, i'm going to measure productivity based on amount of output relative to the amount of effort i'm putting in.
so what may be happening is that bosses see that output is at 80% (productivity down!) but workers see that they can give that 80% output with 40% effort (productivity up!).
In some cases, workers are also being asked to automate the parts of their jobs they enjoy most, Hinds said on the podcast, pointing to customer-service employees who enjoy building relationships but are increasingly expected to supervise AI agents instead.
"That's what gives you joy and meaning at work," she said. "That is very dangerous."
What's a 20% productivity gain if I constantly feel deflated by work that used to energize me? That's going to give back the productivity gain and more, while also decreasing my quality of life.
This is all normal. It’s also well worth the time spent learning
So for my work, it's made me much better at my job. Much faster and more accurate.
But now it's happening at the company level: "We're going to add a chatbot to increase productivity! Now MCP tools! Then agentic workflows! We’ll add skills, and now productivity will go up! Maybe loops will do it?"
If I was valued at 1 trillion dollars, and I was in the hole enough to sink a couple small countries' GDP, maybe I would slowly start to optimize to maximize token usage.
I want to sell tokens, how do I sell more tokens? Not by doing the same work in less tokens, that's for sure.
This is like if you pay me by the hour and then excitedly tell me that you keep paying 10k a month and it's great. I will most certainly not work faster, in this hypothetical, if you tell me you love spending money because it gives you a dopamine rush. I would probably spend a couple more hours REALLY thinking about the task, maybe writing some docs nobody will read, maybe considering multiple options, doing benchmarks, doing research, and then later maybe ill do the actual task as well.
Im not saying these AI companies are scamming us, but the incentives are there and extremely clear. The only thing currently holding it back is that there is some vague kind of competition.
(There's a reason why I call it the MBA's stone. It transmutes all knowledge work into a problem of management.)