This appears to be another take on a previously discussed study

One discussion - https://news.ycombinator.com/item?id=44549436

At this point I use AI primarily to write docstrings and to do annoying refactors. The value add in other situations is decent but not great, and IMO most of the agentic things are trying to iterate and implement way too much at one time. I prefer aider for this reason because it does very little.

Blogspam, dupe.

Original research: https://news.ycombinator.com/item?id=44522772 (750 points, 478 comments)

Another large thread: https://news.ycombinator.com/item?id=44526912 (276 points, 272 comments)

Do yourself a favor and skip what Gary Marcus has to say on this and read the METR study it's self

Gary Marcus is just a guy whose job it is to say AI doesn't work, regardless of whether or not is actually is working in any particular situation.

(Ed Zitron is another such guy, except Ed doesn't actually know anything about AI and refuses to learn anything either.)

I really love it when people tell me that my own lived experience is invalid and wrong.