This is written for the 3 models (Sonnet, Haiku, Opus 3). While some lessons will be relevant today, others will not be useful or necessary on smarter, RL’d models like Sonnet 4.5.
> Note: This tutorial uses our smallest, fastest, and cheapest model, Claude 3 Haiku. Anthropic has two other models, Claude 3 Sonnet and Claude 3 Opus, which are more intelligent than Haiku, with Opus being the most intelligent.
The big unlock for me reading this is to think about the order of the output. As in, ask it to produce evidence and indicators before answering a question. Obviously I knew LLMs are a probabilistic auto complete. For some reason, I didn't think to use this for priming.
I find the word "engineering" used in this context extremely annoying. There is no "engineering" here. Engineering is about applying knowledge, laws of physics, and rules learned over many years to predictably design and build things. This is throwing stuff at the wall to see if it sticks.
(They're not an LLM fan; also: I directionally agree about "prompt" engineering, but the argument proves too much if it disqualifies "context" engineering, which is absolutely a normal CS development problem).
Unless you are going into a legal definition, where there's a global enumeration of the tasks it does, "engineering" means building stuff. Mostly stuff that is not "art", but sometimes even it.
Building a prompt is "prompt engineering". You could also call it "prompt crafting", or "prompt casting", but any of those would do.
Also, engineering also had a strong connotation of messing with stuff you don't understand until it works reliably. Your idea of it is very new, and doesn't even apply to all areas that are officially named that way.
> Engineering is about applying knowledge, laws of physics, and rules learned over many years to predictably design and build things. This is throwing stuff at the wall to see if it sticks.
There’s one other type of “engineering” that this reminds me of…
I found Hillel Wayne's series of articles about the relationship between software and other engineering disciplines from a few years fairly insightful [1]. It's not _exactly_ the same topic but a lot of overlap in defining wht is "real engineering".
I really struggle to feel the AGI when I read such things. I understand this is all of year old. And that we have superhuman results in mathematics, basic science, game playing, and other well-defined fields. But why is it difficult to impossible for LLMs to intuit and deeply comprehend what it is we are trying to coax from them?
Reminds me of a time that I found I could speed up by 30% an Algo in a benchmark set if I seed the random number generator with the number 7. Not 8. Not 6. 7.
Agree with the other commenters here that this doesn't feel like engineering.
However, Anthropic has done some cool work on model interpretability [0]. If that tool was exposed through the public API, then we could at least start to get a feedback loop going where we could compare the internal states of the model with different prompts, and try and tune them systematically.
"Engineering" here seems rhetorically designed to convince people they're not just writing sentences. With respect "prompt writing" probably sounds bad to the same type of person who thinks there are "soft" skills.
Here's my best advice of prompt engineering for hard problems. Always funnel out and then funnel in. Let me explain.
State your concrete problem and context. Then we funnel out by asking the AI to do a thorough analysis and investigate all the possible options and approaches for solving the issue. Ask it to go search the web for all possible relevant information. And now we start funneling in again by asking it to list the pros and cons of each approach. Finally we asked it to choose which one or two solutions are the most relevant to our problem at hand.
For easy problems you can just skip all of this and just ask directly because it'll know and it'll answer.
The issue with harder problems is that if you just ask it directly to come up with a solution then it'll just make something up and it will make up reasons for why it'll work. You need to ground it in reality first.
So you do: contrete context and problem, thorough analysis of options, list pros and cons, and pick a winner.
My workflow has gotten pretty lax around prompts since the models have gotten better. Especially with Claude 4.5 (and 4 before it) once they have a bit of context loaded about the task at hand.
I keep it short and conversational, but I do supervise it. If it goes off the rails just smash esc and give it a course correction.
And then if you're coming from no context: I throw a bit more detail in at the start and usually start by ending the initial prompt with a question asking it if it can see what I'm talking about in the code; or if it's going to be big: I use planning mode.
So we've taught this thing how to do what we did and now we need to be taught how to get it to do the things we taught it to do. If this didn't have the entire US economy behind it, it would catch fire like a hot balloon.
First it says that if you ask a logic problem without any system prompt it will not get it. But if you add a system prompt that "You are a logic bot designed to answer complex logic problems.", it will get it right.
It gets "a married person (Jack) is looking at an unmarried person (George)" which is simply incorrect!
They also add "Although notably not for all the right reasons", is it to justify the logic?
Yesterday I was trying to make a small quantized model work, but it just refused to follow all my instructions. I tried to use all the tricks I could remember, but fixing instruction-following for one rule would always break another.
Then I had an idea: do I really want to be a "prompt engineer" and waste time on this, when the latest SOTA models probably already have knowledge of how to make good prompts in their training data?
Five minutes and a few back-and-forths with GPT-5 later, I had a working prompt that made the model follow all my instructions. I did it manually, but I'm sure you can automate this "prompt calibration" with two LLMs: a prompt rewriter and a judge in a loop.
Are there any project-based up-to-date guides to Agentic coding w/ VS Code? Including all the latest features they keep adding from Cursor and other IDEs. Since we can just bring our own free models there.
30 comments
[ 3.0 ms ] story [ 43.3 ms ] thread> Note: This tutorial uses our smallest, fastest, and cheapest model, Claude 3 Haiku. Anthropic has two other models, Claude 3 Sonnet and Claude 3 Opus, which are more intelligent than Haiku, with Opus being the most intelligent.
https://news.ycombinator.com/item?id=44978319
(They're not an LLM fan; also: I directionally agree about "prompt" engineering, but the argument proves too much if it disqualifies "context" engineering, which is absolutely a normal CS development problem).
Building a prompt is "prompt engineering". You could also call it "prompt crafting", or "prompt casting", but any of those would do.
Also, engineering also had a strong connotation of messing with stuff you don't understand until it works reliably. Your idea of it is very new, and doesn't even apply to all areas that are officially named that way.
There’s one other type of “engineering” that this reminds me of…
[1] https://hillelwayne.com/post/are-we-really-engineers/
Prompt engineering is honestly closer to the work of an o.g. engineer. Monitor the dials. Tweak the inputs. Keep the train on time.
Maybe Blahgineering?
"The words "blah blah blah" increase AI accuracy" - https://news.ycombinator.com/item?id=45524413
Reminds me of a time that I found I could speed up by 30% an Algo in a benchmark set if I seed the random number generator with the number 7. Not 8. Not 6. 7.
However, Anthropic has done some cool work on model interpretability [0]. If that tool was exposed through the public API, then we could at least start to get a feedback loop going where we could compare the internal states of the model with different prompts, and try and tune them systematically.
[0] https://www.anthropic.com/research/tracing-thoughts-language...
State your concrete problem and context. Then we funnel out by asking the AI to do a thorough analysis and investigate all the possible options and approaches for solving the issue. Ask it to go search the web for all possible relevant information. And now we start funneling in again by asking it to list the pros and cons of each approach. Finally we asked it to choose which one or two solutions are the most relevant to our problem at hand.
For easy problems you can just skip all of this and just ask directly because it'll know and it'll answer.
The issue with harder problems is that if you just ask it directly to come up with a solution then it'll just make something up and it will make up reasons for why it'll work. You need to ground it in reality first.
So you do: contrete context and problem, thorough analysis of options, list pros and cons, and pick a winner.
I keep it short and conversational, but I do supervise it. If it goes off the rails just smash esc and give it a course correction.
And then if you're coming from no context: I throw a bit more detail in at the start and usually start by ending the initial prompt with a question asking it if it can see what I'm talking about in the code; or if it's going to be big: I use planning mode.
https://docs.google.com/spreadsheets/d/1jIxjzUWG-6xBVIa2ay6y...
First it says that if you ask a logic problem without any system prompt it will not get it. But if you add a system prompt that "You are a logic bot designed to answer complex logic problems.", it will get it right.
It gets "a married person (Jack) is looking at an unmarried person (George)" which is simply incorrect!
They also add "Although notably not for all the right reasons", is it to justify the logic?
Then I had an idea: do I really want to be a "prompt engineer" and waste time on this, when the latest SOTA models probably already have knowledge of how to make good prompts in their training data?
Five minutes and a few back-and-forths with GPT-5 later, I had a working prompt that made the model follow all my instructions. I did it manually, but I'm sure you can automate this "prompt calibration" with two LLMs: a prompt rewriter and a judge in a loop.