When everything is “estimates”, anything sounds amazing.
The impressive thing here is that Asana has such a poorly run engineering org that replacing their testing framework was estimated to take 5 years and $6 million.
1. I am not sure what your profession is, but I spend most of my reading time at work, with code, emails and so on. Unfortunately I don't have much time to read for recreation these days.
I would love to know the responsible people at Asana who approved this joint PR. Like, do you not get the blatant second degree negative implications of what’s stated here, albeit manipulated and hyperbolic, for a project management software?
People may think it’s an overstatement. But I think it’s not.
This kind of projects are where AI is most helpful. Long tedious and highly testable projects like ports or legacy system replacements where humans have to grind through millions of lines of code without really thinking are the perfect target for AI.
I once did a C to C# port of several math libraries, and while I was able to automate most of it, it still required a ton of manual work. I bet if I had Claude, I could do what I did in 3 months in a day or two.
This is what I was thinking too. Ports are probably where AI has the one of the biggest time savings over humans.
Although also agree, that even if human's did it, they probably would try to automate as much as possible and speed up development, and maybe that original 5 year estimate was if it was done completely manually.
Comparisons such as 5 years and $6M are meaningless. Instead, they should compare with other AI, such as how much it would have cost using self-hosted open-weight models, or using a API that offers open-weight models etc.
The latter sounds about as attractive to me as the former, actually more because my customers are mostly hoping for their bugs to be fixed , not for the latest novel feature.
These conclusion can be obtained by any individual adult with a command of the English language and average written text comprehension abilities within 2 minutes. That’s how long it takes to read and understand the case study.
So, why did you surrender your cognition to an LLM? Do you have some form of mental impairment?
The five year project likely represents a bunch of engineers that are stuck in meetings and coordination for 75% of their time, then you staff the project with more people (because you need four times as many) then they all need more meetings to coordinate too. And then they need to write a lot more documentation in the process because in that five years the people working on the project will all turn over at least once if not 2-3 times. Never mind critical people getting pulled off the project to work on other more important things on a monthly basis as is common at large firms.
It's entirely possible for a 2-3 month project if one senior person was left the hell alone to do it to take a dozen people five years.
Unfortunately, this is the type of statements we can't verify. I'm not sure why these types of news are still coming out when we all have AI at work.
Whenever someone does such a huge drastic change like this, it's by ignoring a large chunk of code that most people where afraid to touch for good reasons. Now, that code is gone, AI is celebrated, things will break, people will very hard in the background to fix it, with no fanfare.
Unit testing can get you some of the way, but its not a full-on all-case guarantee. Sometimes the code is encapsulating some particularly complex system/behaviour. Sometimes the reason is interop/compatilibity issues or some kind of politics.
P.S. you managed to introduce a typo in your quote (were –> where)
> I'm not sure why these types of news are still coming out when we all have AI at work.
Because OpenAI is burning $15 billion/year, and outrageous stories like those get parroted in the media. It's free marketing for a company desperate to get middle management to believe that a $500/mo subscription is absolutely crucial for every single employee.
ugh just recently I saw a news article gushing about how AI had helped with some health/medicine study, and various patient organizations were all like "oh yeah this is a Good use of AI" and then I click the link to read the study and it's decision trees and clustering on a tiny dataset that you could analyze with a ten year old laptop.
I mean, sure at some point decision trees and clustering were called "AI", but the way the article was written you'd think OpenAI and Anthropic were responsible for the advancement of medicine.
yeah but you'd need skills to operate the 10 year old laptop. probably have to copy and paste and understand excel and maybe whatever the hell a confidence interval is
this allows said middle managers to do the analysis themselves and feel mostly confident in the results.
I kind of agree, though I find "AI" to be an almost useless term. But the point here wasn't whether this or that form of ML is AI. The term "AI" was almost never used in mainstream media until a few years ago, when it suddenly it was all over the place
https://trends.google.com/trends/explore?date=all&q=AI,Biden...
and nearly without exception referring to very large generative text/image models. So when a news article in 2026 talks about AI without specifying that it's nothing to do with ChatGPT etc. (and in fact using a very different method requiring no pre-trained model and no dependence on large companies), I find it highly misleading.
I agree it's misleading and it's very frustrating. I blame the lay press and social media that as you say became very suddenly aware of the results of AI research. This happened at different times, e.g. in 2016 with AlphaGo, then in 2017 with OpenAIFive etc etc. And lately again with LLMs.
I don't agree that it's a useless term. AI, as a research area, is well-defined by its journals and conferences and the grants that support that research. It's not the fault of the AI research community that OpenAI's app went viral and now there's so much confusion in mainstream discourse about the term. I try to set the record straight whenever I can but it does feel like fighting a pointless rearguard action, even in this community that is better informed than most.
It’s just sales claims.
Do you see that “contact sales” button?
You’re not the target of that.
Deranged AI-psychotic C-suites with a two-digit IQ and too much confidence are the target.
Exactly. You can simply close all open tickets with <WONTFIX> and claim that you've cleared 5 years of engineering work in 5 minutes. It doesn't mean anything.
> But at the rate we were going, we were still roughly five years from finishing.
Everyone has seen how this sort of thing comes about: it's meaningful work for engineering but never business/product critical so it just drags along.
Seems like this time around someone just went "I wonder if we could do it this way" and it worked. Perfect example of ditching sunk-cost and starting from scratch. Great outcome for them.
This doesn't "verify" anything. I can't see the code before, I can't see the code after. I can't verify they do the same thing. It won't be updated if there's a long churn of issues and breakage coming from this work plaguing the team for years. The only thing it verifies is that Asana did indeed make the claim, which I don't think anyone was doubting.
> Back in 2022, we set out to migrate Asana's frontend test suite off Enzyme, our aging testing library, and onto React Testing Library (RTL).
Your telling me they had a full team of engineers at Asana, doing nothing but rewriting tests for 4 years until AI came along and did the last year in a couple days? I'm extremely doubtful. I don't doubt for a minute they were one-track to take 5 years, but not because it was 5 years of engineering effort for humans.
Far more likely they finally cleared up some tech debt they had been plugging at off and on for 4 years, then a PR flack got ahold of it and it became a breathless "AI did 5 years of work in a couple days".
How is it bad faith to question corporate marketing blog posts? That's not bad faith, that's table stakes for critical thinking.
We absolutely should be skeptical when an AI company makes big claims. The fact that the company the AI company is talking about also claims the same thing doesn't change that. OpenAI is getting awareness and marketing out of this, and I'm sure Asana is getting something out of it too.
And I'm not even saying OpenAI or Asana are necessarily lying. Asana might not find out for months or years that some tests had been rewritten poorly, and don't sufficiently test the thing they were supposed to test anymore. For example. If they truly had 5 years of work, then I find it hard to believe that in two weeks of the LLM churning, they had the time to review all the new tests. They spot-checked, at best.
Maybe everything is great. Maybe the LLM did a wonderful job, and this was awesome for Asana. But we have no idea, and we're unlikely to ever find out. Unless, of course, it's in Asana's interest from a marketing perspective to tell us.
(Not sure what the URLs you posted in your comment are supposed to prove. They're unrelated to the issue at hand.)
I'm not sure I agree with the extremely easy to verify part. How do you validate that your new test suite covers the exact same edge cases as your old?
But yes, LLMs are great at tests. I'm not doubting that an LLM migrated some legacy tests, or that it was a task taking a long time. I am doubting the way it was presented in the article, that this was taking a full team of engineers dedicated to nothing but this, 5 years to accomplish (and presumably already spent 4 million working on this, since the total estimate was 6 million, and they've been at it for 4 year).
I think some PR flack got ahold of the fact that an LLM wrapped up migrating some legacy tests that a team had been slowly chipping away at for 4 years, between their normal feature work, and were on track to finish in 5, then wrote it up like it was that teams entire focus instead of a piece of tech debt.
That makes far more sense to me than spending millions for a team of engineers dedicated to nothing but rewriting an existing test suite.
> Migrating a test suite is exactly the kind of work that LLM's excel at beause it's extremely easy to verify
I don't think that's true. All you see is all the test pass. You don't know that the tests still cover everything that they used to.
LLMs excel where there is an excellent test suite, and you ask them to modify the thing that the test suite tests, and forbid them from changing the tests.
> An engineer checked progress twice a day and reviewed every proposed change.
It doesn’t sound like that much changed. If one engineer can really review all the changes in under two weeks, the original estimate of 5 years of engineering work has to be waaaaaaay off.
Phrased this way, my aspiring product leader hat activates: that 5 years of work cannot be of much value to a company worth billions because a 5 year estimate is the lowest possible qualification you can make. Its a “will never happen” shelving.
all work is stack ranked against other opportunities. no matter how many teams help to parallelize, leadership basically comes down to ranking the stack correctly. Valuable work will be at the top by any means necessary. “mvp”, probes, task forces, code yellows, and so on.
edit: it’s even a backhanded compliment to agentic coding. Asana a public company considers a $12k outlay for something they’d never spend real resources on worth a case study =|
Apparently the task was to remove dead weight. Haven't read what they replaced "Enzyme" with.
I did subagent based removal tasks a few times. These were the ones that required the least amount of input or thinking from me, because the requirements are abundantly clear. "Remove this part of the code without breaking any other part or by porting the tests done with it onto another system."
In these situations the code acts as the bookkeeping ledger itself, and coordination complexity is almost a no-brainer.
either it was a hail mary 5 year change-the-market offering, or it was something that staff can "get around to" whenever they have the cycles, and would, at current rates, take like 3-5 years to do.
was the tool worth 12k? if it's not a line in capex or opex budgets it's value is $0
Is this the react test framework enzyme? To me, $6 million and 5 years seems like a very high estimate for this even without any automation (AI or otherwise) for even relatively large applications. How could they possibly have a test suite this large?
There are countless years man work awaiting in the issues of abandoned projects. Nobody is getting fired because of them. But new people won't be hired if managers believe in marketing stunts like this.
> For comparison: the previous plan was expected to take at least five years and estimated to cost roughly $6M.
If that's an estimate from an LLM, those have always been way off for me. I'm constantly amazed whenever something that an agent estimates would take weeks ends up being completed in an hour.
The five-sentence prompt beating an elaborate setup fits what I keep seeing with these tools. The trick is carving off well-scoped migrations, not handing over open-ended design work.
If you ask Sol or Claude how much time it will take to implement a plan they just came up with, they usually advise a timeframe in the weeks or months - assuming, I suppose, that human programmers will be building it. And then you ask the model to just "do it" and it takes an hour or two. I always find this entertaining.
It does, assuming "world" is made up of humans (which it is, in its training data). It does however not have any model whatsoever of itself, apart from some superficial "I am a helpful assistant built by $X".
I find that the models have increasingly started throwing out made up amounts of time around, as if they wouldn't be aware that the user is already using Claude Code or Codex.
It's like every time they make a plan, there's something about things taking "a week or two", "month of focused work", or whatever.
This is something that would've been RL'd out a long time ago if it wasn't great for business.
The British government cleared a large chunk of 5 years of the NHS (hospital) waiting list by, in part, just removing people from it if they'd been waiting too long.
85 comments
[ 0.25 ms ] story [ 19.4 ms ] threadThe impressive thing here is that Asana has such a poorly run engineering org that replacing their testing framework was estimated to take 5 years and $6 million.
― Woody Allen
You see: normally one reads for recreation or enlightenment.
If that same one would approach their job as a recreational activity, or solely for enlightenment, you might not have that job for very long.
2. It's a joke! Relax.
"Don't be snarky."
https://news.ycombinator.com/newsguidelines.html
This kind of projects are where AI is most helpful. Long tedious and highly testable projects like ports or legacy system replacements where humans have to grind through millions of lines of code without really thinking are the perfect target for AI.
I once did a C to C# port of several math libraries, and while I was able to automate most of it, it still required a ton of manual work. I bet if I had Claude, I could do what I did in 3 months in a day or two.
Although also agree, that even if human's did it, they probably would try to automate as much as possible and speed up development, and maybe that original 5 year estimate was if it was done completely manually.
Summary: The Takeaway Did an AI generate 5 years' worth of highly creative, novel software features from scratch in two weeks? No.
Did an AI complete 5 years' worth of tedious, widespread code-migration technical debt in two weeks? Yes.
We engineers, had the power to dictate how our days will look, and we were able to give ballooned timelines to give us room to braeth.
This has changed. The power is no longer in our hands, for good or bad.
I suspect it as a two month project without agents. Max.
It's entirely possible for a 2-3 month project if one senior person was left the hell alone to do it to take a dozen people five years.
Whenever someone does such a huge drastic change like this, it's by ignoring a large chunk of code that most people where afraid to touch for good reasons. Now, that code is gone, AI is celebrated, things will break, people will very hard in the background to fix it, with no fanfare.
This appears entirely unreasonable.
Normally the reason is not good. The reason is that unit testing is missing or that downstream effects are not entirely mapped out.
Exactly activities that traditional software developers are loathing because they are boring and mentally straining.
Seems a bit strong.
Unit testing can get you some of the way, but its not a full-on all-case guarantee. Sometimes the code is encapsulating some particularly complex system/behaviour. Sometimes the reason is interop/compatilibity issues or some kind of politics.
P.S. you managed to introduce a typo in your quote (were –> where)
Because OpenAI is burning $15 billion/year, and outrageous stories like those get parroted in the media. It's free marketing for a company desperate to get middle management to believe that a $500/mo subscription is absolutely crucial for every single employee.
ugh just recently I saw a news article gushing about how AI had helped with some health/medicine study, and various patient organizations were all like "oh yeah this is a Good use of AI" and then I click the link to read the study and it's decision trees and clustering on a tiny dataset that you could analyze with a ten year old laptop.
I mean, sure at some point decision trees and clustering were called "AI", but the way the article was written you'd think OpenAI and Anthropic were responsible for the advancement of medicine.
this allows said middle managers to do the analysis themselves and feel mostly confident in the results.
I kind of agree, though I find "AI" to be an almost useless term. But the point here wasn't whether this or that form of ML is AI. The term "AI" was almost never used in mainstream media until a few years ago, when it suddenly it was all over the place https://trends.google.com/trends/explore?date=all&q=AI,Biden... and nearly without exception referring to very large generative text/image models. So when a news article in 2026 talks about AI without specifying that it's nothing to do with ChatGPT etc. (and in fact using a very different method requiring no pre-trained model and no dependence on large companies), I find it highly misleading.
I don't agree that it's a useless term. AI, as a research area, is well-defined by its journals and conferences and the grants that support that research. It's not the fault of the AI research community that OpenAI's app went viral and now there's so much confusion in mainstream discourse about the term. I try to set the record straight whenever I can but it does feel like fighting a pointless rearguard action, even in this community that is better informed than most.
This. It looks like AI companies have managed to use this for their advantage. Can't really blame them.
Key statement:
> But at the rate we were going, we were still roughly five years from finishing.
Everyone has seen how this sort of thing comes about: it's meaningful work for engineering but never business/product critical so it just drags along.
Seems like this time around someone just went "I wonder if we could do it this way" and it worked. Perfect example of ditching sunk-cost and starting from scratch. Great outcome for them.
> Back in 2022, we set out to migrate Asana's frontend test suite off Enzyme, our aging testing library, and onto React Testing Library (RTL).
Your telling me they had a full team of engineers at Asana, doing nothing but rewriting tests for 4 years until AI came along and did the last year in a couple days? I'm extremely doubtful. I don't doubt for a minute they were one-track to take 5 years, but not because it was 5 years of engineering effort for humans.
Far more likely they finally cleared up some tech debt they had been plugging at off and on for 4 years, then a PR flack got ahold of it and it became a breathless "AI did 5 years of work in a couple days".
The ultimate bad faith interpretation. "Unless I can verify the results that contradict my worldview, I don't acknowledge them."
> But you _can_ verify if you had bothered
I think pointing out I can't is indeed fair.
We absolutely should be skeptical when an AI company makes big claims. The fact that the company the AI company is talking about also claims the same thing doesn't change that. OpenAI is getting awareness and marketing out of this, and I'm sure Asana is getting something out of it too.
And I'm not even saying OpenAI or Asana are necessarily lying. Asana might not find out for months or years that some tests had been rewritten poorly, and don't sufficiently test the thing they were supposed to test anymore. For example. If they truly had 5 years of work, then I find it hard to believe that in two weeks of the LLM churning, they had the time to review all the new tests. They spot-checked, at best.
Maybe everything is great. Maybe the LLM did a wonderful job, and this was awesome for Asana. But we have no idea, and we're unlikely to ever find out. Unless, of course, it's in Asana's interest from a marketing perspective to tell us.
(Not sure what the URLs you posted in your comment are supposed to prove. They're unrelated to the issue at hand.)
With enough budget this seems a rather reasonable and fun task.
But yes, LLMs are great at tests. I'm not doubting that an LLM migrated some legacy tests, or that it was a task taking a long time. I am doubting the way it was presented in the article, that this was taking a full team of engineers dedicated to nothing but this, 5 years to accomplish (and presumably already spent 4 million working on this, since the total estimate was 6 million, and they've been at it for 4 year).
I think some PR flack got ahold of the fact that an LLM wrapped up migrating some legacy tests that a team had been slowly chipping away at for 4 years, between their normal feature work, and were on track to finish in 5, then wrote it up like it was that teams entire focus instead of a piece of tech debt.
That makes far more sense to me than spending millions for a team of engineers dedicated to nothing but rewriting an existing test suite.
I don't think that's true. All you see is all the test pass. You don't know that the tests still cover everything that they used to.
LLMs excel where there is an excellent test suite, and you ask them to modify the thing that the test suite tests, and forbid them from changing the tests.
It doesn’t sound like that much changed. If one engineer can really review all the changes in under two weeks, the original estimate of 5 years of engineering work has to be waaaaaaay off.
all work is stack ranked against other opportunities. no matter how many teams help to parallelize, leadership basically comes down to ranking the stack correctly. Valuable work will be at the top by any means necessary. “mvp”, probes, task forces, code yellows, and so on.
edit: it’s even a backhanded compliment to agentic coding. Asana a public company considers a $12k outlay for something they’d never spend real resources on worth a case study =|
I did subagent based removal tasks a few times. These were the ones that required the least amount of input or thinking from me, because the requirements are abundantly clear. "Remove this part of the code without breaking any other part or by porting the tests done with it onto another system."
In these situations the code acts as the bookkeeping ledger itself, and coordination complexity is almost a no-brainer.
either it was a hail mary 5 year change-the-market offering, or it was something that staff can "get around to" whenever they have the cycles, and would, at current rates, take like 3-5 years to do.
was the tool worth 12k? if it's not a line in capex or opex budgets it's value is $0
Non prod code, good use case.
> For comparison: the previous plan was expected to take at least five years and estimated to cost roughly $6M.
If that's an estimate from an LLM, those have always been way off for me. I'm constantly amazed whenever something that an agent estimates would take weeks ends up being completed in an hour.
It's like every time they make a plan, there's something about things taking "a week or two", "month of focused work", or whatever.
This is something that would've been RL'd out a long time ago if it wasn't great for business.
Oh dear, poor Asana customers...
The headlines feel similar