No. Prompting is Project Management. You’re setting expectations for what you want and waiting to see what the model provides. You may be learning to articulate things well enough for the specific model you’ve chosen to produce the output you want, but it’s not programming. Only when you manipulate the code is it programming.
It's pair programming. It's definitely not project management because, to build anything useful, I need to know everything I needed to know as a software engineer before AI hit the mainstream. A PM who isn't an experienced engineer could probably come up with the bones of an app, but the app would suck.
Programming as I remember it pre-AI was essentially just toil and iteration. Programming with AI is nearly zero toil and iteration. The first prompt is like installing a framework. The next 200 prompts are programming.
Yeah, kinda, I guess. It's different from Project Management though, because a lot of that was also managing people and expectations (both the stakeholders and the team).
Writing a really good skill, or a spec document, feels more like programming than Project Management.
I’ll buy this. I get a lot more out of the code when I ask it to do things in steps.
Though I absolutely still have to go through it robustly to protect against Claudisms, comments only Claude understands, function names that make sense only with maximum context and a dictionary of that same context, etc. and it still overlaps function responsibility.
This really resonated with me. My co-founder is a really smart man who cannot get good results out of LLMs. I haven't really gone into it with him in any detail, but his experience is like the academic in TFA; it just doesn't do what he wants it to do.
I'll send him the article and see if a crash-course in "this is how programmers think about problems" helps.
The reason it's not programming is because the AI has several layers of indirection that take ambiguous descriptions and iterates on them in a similar way a human would before writing the code itself. These layers of indirection are not computer specifications.
Programming entails specific instructions and a high level of control, a.k.a. specifications. Of course, there are other kinds of specs, like business specs, but here we're talking about specs in the literal sense, where you are being specific of how the machine should work in a predictive sense. We don't do this with prompts, and we expect the AI to do this for us in the same way we expect a human engineer to take ambiguous directions and turn it into the actual computer spec.
The fact that domain knowledge makes for better prompts is a shortcoming of AI instead of a feature of prompting.
These are pretty nice examples, as both SQL and Prolog can only be used well for anything non-trivial when you're aware of how the result you're asking for is going to be figured out.
SQL and Prolog are still deterministic mappings, as in the same syntax will yield the same results every time.
AI has an architecture that needs to be trained, and then it works at such large scales with so many other points of indirections like mixture of experts, chain of thought, that it's just way past the point it could be called a mapping or an abstraction.
Every sentence in English is either a function call (action), a database query of sorts, an observation about time/place/people or a logical proposition. Even continuations / process control are a part of natural language.
Domain knowledge is basically having an idea about the rules and procedures of the said domain. A glossary of a domain is a library.
I find my prompting these days increasingly become more and more imperative and resemble logo somewhat. It is better I suppose from assembling code from blogs, books, github, stack overflow and 20+ google searches.
1. The verifiable-ness of software is overstated. Compiling successfully is not verification. Passing unit tests is not verification. Only verification against the specification is verification. Engineers are good at AI coding because we are better at writing the spec (which is just a pre-defined rubric for verification), and also actually verifying the program for adherence against it (as well as characteristics of the program that are not codified in the spec).
2. You still need to be "tech-savvy" to use LLMs effectively. If you'd struggle with the settings menu of a smart TV, you're probably gonna struggle with LLMs. I don't know what makes people "tech-savvy" (maybe just IQ plus broad familiarity with technology) but most software engineers definitely have it.
3. I think domain expertise is paradoxically less of a moat with LLMs. I have a Socratic teaching-machine at my fingertips.
15 comments
[ 0.22 ms ] story [ 22.6 ms ] threadProgramming as I remember it pre-AI was essentially just toil and iteration. Programming with AI is nearly zero toil and iteration. The first prompt is like installing a framework. The next 200 prompts are programming.
Writing a really good skill, or a spec document, feels more like programming than Project Management.
Though I absolutely still have to go through it robustly to protect against Claudisms, comments only Claude understands, function names that make sense only with maximum context and a dictionary of that same context, etc. and it still overlaps function responsibility.
I’m not out of a job yet! Hah.
I'll send him the article and see if a crash-course in "this is how programmers think about problems" helps.
Programming entails specific instructions and a high level of control, a.k.a. specifications. Of course, there are other kinds of specs, like business specs, but here we're talking about specs in the literal sense, where you are being specific of how the machine should work in a predictive sense. We don't do this with prompts, and we expect the AI to do this for us in the same way we expect a human engineer to take ambiguous directions and turn it into the actual computer spec.
The fact that domain knowledge makes for better prompts is a shortcoming of AI instead of a feature of prompting.
I’m not saying you’re wrong to separate it and say there’s a difference, but I don’t think the line where you put it.
AI has an architecture that needs to be trained, and then it works at such large scales with so many other points of indirections like mixture of experts, chain of thought, that it's just way past the point it could be called a mapping or an abstraction.
Every sentence in English is either a function call (action), a database query of sorts, an observation about time/place/people or a logical proposition. Even continuations / process control are a part of natural language.
Domain knowledge is basically having an idea about the rules and procedures of the said domain. A glossary of a domain is a library.
I find my prompting these days increasingly become more and more imperative and resemble logo somewhat. It is better I suppose from assembling code from blogs, books, github, stack overflow and 20+ google searches.
2. You still need to be "tech-savvy" to use LLMs effectively. If you'd struggle with the settings menu of a smart TV, you're probably gonna struggle with LLMs. I don't know what makes people "tech-savvy" (maybe just IQ plus broad familiarity with technology) but most software engineers definitely have it.
3. I think domain expertise is paradoxically less of a moat with LLMs. I have a Socratic teaching-machine at my fingertips.