They conflate domain expertise with coding expertise, and then assess that people with domain expertise demonstrate great success at coding tasks, which suggests coding agents are so good at writing code that domain experts can now cut software engineering experts out of the loop entirely. Yet if you look at their classifier, it classifies user expertise almost exclusively according to standards that measure expertise in coding. No wonder it predicts success at coding tasks. This just in: people who know how to develop software are better at developing software. What a fucking joke.
Also, in one of their points they argue that, in the case of non-programmers, the more domain expertise one brings to the table, the higher the chances of a successful outcome. Well sure, but what's the quality of the output? Someone with little to no coding ability cannot meaningully judge a piece of source code. It might not matter when we're talking about one-off scripts and such, but for anything more serious, somebody really needs to be in the driver's seat, 'cause this ain't a Waymo yet.
> They conflate domain expertise with coding expertise, and then assess that people with domain expertise demonstrate great success at coding tasks
I didn't read it as such - I read that people with expertise have more success in reaching the goal of the session. Still your point stands = how is this news?
What baffles me, is that expertise of writing code is not important and is hand waved away = non-technical person can reach their goal and the session will be deemed successful. They don't conflate that domain expert = developer, and they dismiss that expertise (ie. updating legal rules matters, not how they are implemented) so i'm confused. Shouldn't both matter?
this seems to be a load bearing claim in the article: "success is determined by how well a person understands the problem they are trying to solve, not whether they’re trained in coding."
I would normally assume that my understanding of the problem I'm trying to solve is a blend of domain knowledge and system knowledge; that is, some of the what and some of the how. And just as important as bringing domain knowledge to the table is bringing the ability to articulate it with conceptual clarity in a way that bridges to the software. I have met many very deep domain experts who don't have that skill.
My personal interest of late is in capturing domain knowledge and managing it like source code, and using it to share with both the humans (in visual form) and the agents (in plain text form) some of that foundational understanding to make them both better at both the what and the how. This seems aligned with the article when it says "the more understanding a worker brings to an agent, the more quality work the agent is able to do" but I would also say that the more understanding we can bring to future humans new to the domain has pretty good ROI as well.
5 comments
[ 3.0 ms ] story [ 18.2 ms ] threadA more useful headline (that fits in our limit) might have been the takeaway they want you to have:
The more domain expertise brought to Claude sessions, the more end in success
They conflate domain expertise with coding expertise, and then assess that people with domain expertise demonstrate great success at coding tasks, which suggests coding agents are so good at writing code that domain experts can now cut software engineering experts out of the loop entirely. Yet if you look at their classifier, it classifies user expertise almost exclusively according to standards that measure expertise in coding. No wonder it predicts success at coding tasks. This just in: people who know how to develop software are better at developing software. What a fucking joke.
I didn't read it as such - I read that people with expertise have more success in reaching the goal of the session. Still your point stands = how is this news?
What baffles me, is that expertise of writing code is not important and is hand waved away = non-technical person can reach their goal and the session will be deemed successful. They don't conflate that domain expert = developer, and they dismiss that expertise (ie. updating legal rules matters, not how they are implemented) so i'm confused. Shouldn't both matter?
I would normally assume that my understanding of the problem I'm trying to solve is a blend of domain knowledge and system knowledge; that is, some of the what and some of the how. And just as important as bringing domain knowledge to the table is bringing the ability to articulate it with conceptual clarity in a way that bridges to the software. I have met many very deep domain experts who don't have that skill.
My personal interest of late is in capturing domain knowledge and managing it like source code, and using it to share with both the humans (in visual form) and the agents (in plain text form) some of that foundational understanding to make them both better at both the what and the how. This seems aligned with the article when it says "the more understanding a worker brings to an agent, the more quality work the agent is able to do" but I would also say that the more understanding we can bring to future humans new to the domain has pretty good ROI as well.