I hate approximately everything about this article, but I'm glad that I took a second look through because this is a decent framework for the-thing-which-he-swears-isnt-prompt-engineering:
---
* diagnosis
* decomposition
* reframing
* constraint design.
Diagnosis is discovering the problem that AI can solve. This is the human part of knowing that a problem exists. Learning to ask the right questions, look at the different ways that the problem can be seen.
Decomposition is about splitting the big problems into bite-sized ones. Take the problem apart, examine it, and let AI help you determine your findings since it handles data so well. Instead of tackling the biggest problem, take it apart and work on the smaller parts to achieve small successes.
Reframing is about shifting your perspective and seeking new interpretations. Extrapolating and recombining the parts of the problem in order to identify the meta components. Perhaps a new way of looking at the problem may find a solution hidden in plain sight.
Constraint design is about setting boundaries for the solution. Knowing what to accomplish, and when to know it is done. Setting the length, style, and description of the audience can help AI understand its mission. But we have to know that first in order to instruct.
---
As someone who asks GPT and junior developers for lots of things, there are a lot of similarities. I don't imagine that is going away, at least until we wire LLMs up to a huge amount of rapidly changing, cross-silo context so it could understand "Fix the monitoring that slowed our recognition of yesterday's bug". So being thoughtful isn't going away. The author agrees with that (see above), but doesn't make clear where he draws the boundary between "being thoughful" and "prompt engineering"
That's just systems analysis though. To call rediscovering that prompt engineering just because you're dealing with AI rather than programmers siloed away from business processes feels a little silly and just being justified to keep the title when as originally formulated by the people who first started using it, it was about learning the "magic words" for a specific LLM
See I always saw prompt engineering as the ability to author a prompt that elicits a specific behavior by the LLM, not simply the incantation to make to it say penis or talk like a sailor. The “make the AI be lewd” was a parlor trick, but the art of making it behave the way you want and to get the answers you want at the depth, breadth, and form you need is a skill and one I think isn’t going away in our lifetime. I’d further note that writing to influence humans to your intended response and understanding is a widely studied field by every study with multiple PhD and beyond disciplines associated. We even elect such people to lead us largely through their ability to prompt engineer humans.
IMO - Folks seem to be reacting to the term “engineer” in the same way traditional engineers with engineering exams do to software engineers claiming to be engineers despite often graduating from a LAS school or not at all - I have a bunch of PE EE in my family and they treat my claim to be an engineer with extreme scorn.
This article’s argument is interesting because it relies on two very questionable principles:
1) That as tools get better, operator specialization is less useful. I understand how this feels like it makes sense, because for for binary tasks like driving it’s true, but for creative tasks, I struggle to think of a single example where this is the case. Which leads to
2) Humans won’t figure out how to use these tools in increasingly complex/weird ways to create increasingly complex/weird outputs.
I think in general, everyone has been so conditioned by the idea of singularity (which to be frank is a completely tangential concept to contemporary LLMs) that they refuse to see these things for what they are: tools built by humans to serve humans when operated by humans.
Add as many layers of “self-prompting” as you want there, but a human still set the original intention and they will be the ones to judge the ultimate outputs.
i feel like people that get really into prompt engineering
seem to forget that other people are working on building models that are even better. Do they really think that the tricks they developed to make gpt3.5 work well will still be necessary on gpt9 (or whatever new model replaces gpt)?
if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff.
I don't agree. There's a huge need for people to figure out how to integrate AI into larger things, and to establish ways we'll work with it - and at every step, including GPT 3.5 prompt engineering. There's a lot you can come up with if you just assume that good generative AI exists, and will probably get better. Different iterations of AI might as well be pluggable modules.
I tend to think that as long as these models remain tools (rather than AI that is self aware, etc.), we will have careers available in both “mechanics and engineers” so to speak. Some will design, others will build, still others will integrate and maintain.
Is that not true to some extent? In the early days of computer science, when there were so many new developments, it would serve you well to be the person working on writing the programs and building computer hardware, rather than the person using computers to get stuff done. The person using computers was still miles ahead of the person not using computers but where you really wanted to be was building languages and inventing new tools.
learning a programming language confers useful skills that you can generalize to new versions of that language or different languages. we can make these generalization becasue we know how the langauges work and people can draw connections from one to the other (i think people here would all agree that learning your second language is far easier than leraning your first).
this is different from LLMs because they don't have explicitly constructed abilities that we can compare across models. so we need to step back and approach every model as brand new and figure out what they're capable of. just because you have a prompt that works amazingly in one model, there's no guarantee that that prompt will continue to work in bigger "better" models.
so rather than working to devise way to trick a particular model into doing your task, it would probably be a better use of your time to learn how to train/modify models to explicitly solve the problem you care about.
The culture war surrounding prompt engineering is so dumb. My eyes roll hard into the back of my head every time I read a comment like this.
Prompt engineering is not the same as programming. In some regards, it is “better”. In other regards, it is “worse”. They’re two different yet similar disciplines, each with their own strengths and weaknesses, but both equally legitimate.
Programming is simply the skill of being able to communicate with a compiler/interpreter effectively.
Prompt engineering is simply the skill of being able to communicate with an LLM effectively.
I wish we weren’t so divided over an incredible new technology.
Anyone using a computer is "just" clicking on specific pixels and triggering keyboard events.. it still could be "actual engineering" imo if one is building a system following engineering methods, solving problems etc.
Working on new LLM model is like doing genetic engineering trying to come up with a new animal species. Prompt engineering is like feeding said animal and shoveling its manure.
Prompt engineering is a reality. No reasons denying. Better prompt is needed for better results.
But, even today prompts and rules of writing are not transferable even between existing LLMs. Future LLMs will have different architectures and requirements. I suspect today's prompt will be split into data and prompt. Probably just references, keywords to actual data.
In other words current state is transitory, next will be very different.
> if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff.
Back in the day, when those new fangled relationship database things came on the scene, do you think people would have been well advised to try and find work on the actual database engine itself, instead of the more frivolous work of using the new technology to, say, solve actual business problems?
"prompt engineering" is a self-destructing field. If you use any rigorous approach to optimizing the prompt, you end up with essentially supervised machine -learning: models can (and do) learn the optimal prompt once there is a yardstick for the goodness of the model's response. That's a classical for a data-scientist, but the skill set has little to do with prompts.
If you are not rigorous, then what you are doing is essentially "black art". It may work for some tasks ad-hoc, but with the rapid pace of model improvement your skill will likely become irrelevant/not needed quickly.
I don't have much experience with GPT but with image generation.
You need some amount of experimentation to get the best results but in my experience what works for one model does nothing or worsens the output in others. Adding loras and different types of images into the equation makes this so variable that I would never consider it useful besides keeping a few key words I used to get x good result on y model and experimenting with those when I start a new project.
Focusing on specific incantations, yes. Focusing on how to decomposing a problem, probably not, but then you get very close to designing systems / data design and analysis methodologies more than "prompt engineering", so I guess I mostly agree with you in as much as the relevant field is not really about AI as it is about picking up more structured design and analysis practices (again).
I'm not so sure - while I agree, prompt engineering will go away, there's probably a huge market for pre-built integrations in every sector. Bind this AI platform to this industry specific tooling in a polished way, etc.
I feel like people who write stuff like this don't understand the difference between software and software that delivers value.
I mean, you think those graphics optimizations we pour millions of dollars into before releasing AAA games will matter when the GTX 8020 outperforms a 4090?
Hint: Delivering value for actual people is rarely the result of sitting on your hands and waiting for the next big platform, or even rolling up your sleeves and trying to learn how to build the next big thing.
You have this idea that by investing energy in something that will be obsoleted you're losing out, but spoiler, that's how 99% of software that delivers actual value works.
The cutting edge of tech almost always ends up being PaaS/SaaS serving itself:
Your mail gets to you because someone is working on software with limitations we solved decades ago.
Your paycheck ends up in your bank account because people invest a ton of time in codebases subject to problems we solved long ago.
Your anti-lock brakes aren't built on a Rust codebase, but some horrible memory unsafe mess running on a processor that's a decade out of date.
—
The reality is: 99% of the effort that goes into trying to build the next big thing goes nowhere. The expected value of you trying to learn "the actual AI stuff" to the greater world is near 0 compared to you "just prompt engineering" and putting out something that solves a pain-point nicely with GPT 3.5.
At the end of the day most of the value that gets delivered to actual users comes from engineers who went deeper into extracting value from the current thing.
We still need people to work on the next big thing so that the 1% of effort that isn't wasted can actually materialize... but in my experience the most successful engineers in that regard are still able to realize the delusion it requires, without being paralyzed by the cognitive dissonance that realization invites.
prompt engineering can definitely deliver value for a customer, but i don't think it will deliver value to a developer in terms of building a sustainable career.
a different analogy that gets at my original concern: becoming an expert prompt engineer for an particular LLM is like becoming a power user for a piece of proprietary software that isn't getting any more updates.
The first point doesn't follow the second: there will always be a sustainable career in building software that delivers value.
In tech we take it for granted that just because there's some new hotness everyone wants to jump on it day 1. GPT 5 could drop tomorrow and if your tool delivers value using 3.5, it's not going to magically stop delivering value, and in most verticals people will prefer your battle tested 3.5 to some brand new 5.
And if 5 does simplify prompts for your use case and there'll still only be two options:
- making a the same thing as what you made with 3.5 is now trivial... in which case you still have the mindshare and the distribution solved to a degree your newly enabled clones don't.
- making a better version of what you made is now trivial... in which case you can just as trivially improve your version and already have the mindshare and distribution solved.
At the end of the day software developers often struggle to fit software into the larger ecosystem it slots into before it becomes something valuable, and to be GPT has been an amazing case study in the fact.
I think the use of the word engineering did half the damage, and I think people confusing the twitter memes with the interesting attempts at prompt engineering (via ReACT, Gorilla, etc) did the other half, but at the end of the day I think a lot of people will be left kicking themselves when "misguided prompt engineers" end up solving real useful problems in ways they didn't think were possible well before we reach the arbitrary goalposts for foundation models that people keep setting up.
I think a lot of prompt engineering is magic thinking. It apparently worked once so you keep iterating on it without knowing if it has a positive or negative effect on any continued use. Some folks at work have got really into it and have some elaborate prompts to use before asking code related questions. I cannot see any different in their output vs mine when I just straight ask gpt4 questions.
The writer tears down prompt engineering while extolling the virtues of well-written prompts. Also GPT-4 isn't "learning new things all the time" in that way...it's getting updates, sure, but not constantly or in real time. These articles are pure nonsense.
everyone thinking Prompt Engineering will go away dont understand how close Prompt Engineering is to management or executive communications. until BCI is perfect, we'll never be done trying to serialize our intent into text for others to consume, whether AI or human.
As someone who’s spent an unreasonable amount of time working with prompting for both text and text-to-image, I can confidently say prompt engineering does work for any nontrivial use cases (e.g generation with constraints). I don’t like that prompt engineering works, but it does, and proper use of it does result in objective improvements in generation quality.
The latest ChatGPT model, gpt-3.5-turbo-0613 has better system prompt steerability, and with some prompt engineering I can get GPT-4 quality results out of it at a fraction of the cost.
Ultrapopular tools like LangChain and AutoGPT are essentially just prompt engineering under the hood.
Even if these models become AGI someday, there's still sort of "prompt engineering", because there is still social engineering. What and how you say something is always going to have an effect on the response.
So, the new thing is that we're going to call everyone an engineer?
Carpenter -> Wood engineer
Tailor -> Fabric engineer
Ceramist -> Clay engineer
See how dumb it looks? And maybe the field is too new to have a specific word for it, but calling it "engineer" diminishes the level of actual engineers.
Calling carpenters wood engineers is pretty much warranted. There should be no engineering police that determines which jobs are allowed to carry the prestigious engineer title. Software engineering was also ridiculed for the longest time by “real” mechanical engineers.
As a former mechanical engineer I’m not sure how common that was. Computer Scientist was/is a common alternative but that’s somewhat of an artifact of CS often being associated with math departments.
I don't see why we have to take and muddle someone else's word, "engineer" doesn't have inherently prestigious syllables, the word sounds good when someone says it at a party because the job itself is actually hard or whatever.
If software development (or even "programming"!) is the same, the same thing will happen. If not, any other words we steal will lose their shine too and engineers will start calling themselves something else to get it back.
A Profession Engineer’s authority comes from their education and experience. When they stamp a design or plans, they are putting their reputation and livelihood on the line. If they are negligent, they can be sued for malpractice and may be barred from working as an Engineer in their field. If their employer asks them to do something unethical, they have a duty to refuse.
I personally think it’s worth reserving certain titles for qualified practitioners. If you want to call yourself a Doctor, Lawyer, or Engineer, I think that should mean something.
In many countries it's more a title associated to education and specific degrees from specific schools than a license to practice some professional organisation could revoke.
While, from my limited knowledge, doctors and lawyers both seem to have some sort of license and controlling body in most, if not all, countries.
I haven't counted many this page where there is a regulating body that could take away a license to practice as an engineer.
The UK seems to discriminate on a per discipline basis. Canada apparently is ambiguous, with self-regulating bodies but courts dismissing cases regarding job titles. Germany has one, but only for civil engineers (still according to the wikipedia page).
Still only relying on this wikipedia page, there are on the other hand many countries where although the title is protected, it simply requires one to have studied a certain number of years (Poland), to have completed a specific degree (Brazil, Chile, Germany), or a specific degree in one of a few select higher-education schools (France, Turkey).
This is just how language evolves, stop gatekeeping it. Most engineers these days have never been near an actual engine, and nor do they have any need to.
Sorry, this was already on its way out from the very beginning with the term "software engineer".
Outside of tech, being an actual engineer involves universal standards, certifications and the onus and responsibility of failures that the deregulated libertarian paradise of the tech world is impossible to implement.
Actual engineers go through rigorous testing, certification and have universal standards that uphold them to meet these standards regardless of business pressure.
Until the tech and software world have anywhere near that level of scrutiny by public institutions (good luck, all you will hear is the screams of "communism!") then frankly its already looked dumb ever since developers were even called "engineers" to begin with.
The joke about self-aggrandizing "X engineer" titles is very old. On "Mama's Family," the main character is essentially a homemaker, but in one episode she calls herself a "domestic engineer" to make it sound more impressive. That episode is from 1987.
Prompts are programs. They take input and produce an expected output. You can parameterize them and they can be used as functions and chained together. In that way, prompt engineering is software engineering.
>> Prompts are programs. They take input and produce an expected output.
I think one key difference is that (most) programs that people are used to writing tend to be deterministic (when not intentionally random...) whereas LLM prompts pretty much always end up with an nondeterministic output.
You are not the only one, but also your experience is not the only experience with the tool.
I have used several of 'the big' SaaS GPTs, and have gotten great use from them.
There is absolutely a use for these tools, but they do take some small amount of skill to get good results.
It all comes down to the context you can provide to steer the answer to what you really need. The better you can describe your current state and what you want from the answer, the better the answer will be.
When working on a project, context is everything. I feel like it'll only really make a difference once it's able to read the thousands of files in my project's folder.
My only use so far is get inspiration for type naming, and very simple scripts that i'm too lazy to write myself.
I'm using it for hobby development, which is more or less serious. Most useful cases I found:
1. API, concepts, etc docs with explanation. Much faster then googling then scrolling through tons of texts with blinking adds
2. Write simple things that I don't want to think about. Like in Python process all files in a directory, follow the links. Saves time.
3. Try things I don't know how to do. The recent was checking if user pressed a key without blocking in Python. Nontrivial, but possible. We went through several options till found the one which works on Ubuntu.
So, it's useful, no regrets about subscribing. Funny thing I'm using it working on toy GPT
I wonder what happened to those people who got 5-year "Knowledge Engineering" degrees from Stanford in the 1980s. They were trained on how to write rules for "expert systems", which turned out to not be very useful.
I'm going to link my own snarky titled by actually okay-ish take on prompt engineering, emphasis on the actual engineering, to show that the author doesn't even know what prompt engineering is - https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...
Prompt engineering is not designing a cool prompt. It's when you start applying genuine techniques to do things that are not possible with tokens alone. For example:
"What's the definition of {apple|orange}" where {apple|orange} is the mathematical average of those two words. This is prompt engineering. Right now, Prompt Engineering is basically in Stable Diffusion through Automatic1111, it's in libraries like microsoft Guidance or LMQL, and not a whole lot else.
This is a weird article not understanding what prompt engineering is. Basically with GPT-3.5 one can say bye bye to fine-tuning in most cases as natural language directives (prompting) combined with chain of thought can now beat most fine-tunings. That only works with extremely large models though. One can use prompt engineering for things like "the person I like talks this way: ... Please talk like this person". Or one can use prompt engineering to simulate one-shot NER and collect all the data appearing in the conversation without any super complex ML model pipelines. I fail to see how this goes away once GPT-274745 can write its own prompts.
I don't think prompt engineering is the problem. I find myself with minimal education but a lot of trial and error being able to "engineer" a good prompt eventually if I know the problem and can validate answers.
I think the big problem is actually finding problems you could use LLMs in. I think anyone that's played with them tends to have a good guess at whether they could or couldn't do something (although you need to really test to be sure, do some prompt engineering etc), but actually finding problems to work on that are within the realm of being solvable and useful is the hardest part imo.
Personally, I look at prompt engineering the same way I see "software engineering" and think it's a bit of a mockery of the trend.
Other industries have proper certifications for "engineering" that I.T. absolutely doesn't, yet someone who figured out how to center a div calls themselves a software engineer.
I reckon if you're going to give devs the right to call themselves engineers just because they write code to solve business problems, you don't have much of a leg to stand to judge people whom write natural language into a software solution, and receive output that solves business problems.
The style of this article is all arched-eyebrows and “keep up with me I see the future” BS.
Clear away the faff and ignore the “AI is improving sooo fast ohh what a magic” remarks. What you’re left with is a tepid article about prompt engineering.
75 comments
[ 2.2 ms ] story [ 74.1 ms ] thread---
* diagnosis
* decomposition
* reframing
* constraint design.
Diagnosis is discovering the problem that AI can solve. This is the human part of knowing that a problem exists. Learning to ask the right questions, look at the different ways that the problem can be seen.
Decomposition is about splitting the big problems into bite-sized ones. Take the problem apart, examine it, and let AI help you determine your findings since it handles data so well. Instead of tackling the biggest problem, take it apart and work on the smaller parts to achieve small successes.
Reframing is about shifting your perspective and seeking new interpretations. Extrapolating and recombining the parts of the problem in order to identify the meta components. Perhaps a new way of looking at the problem may find a solution hidden in plain sight.
Constraint design is about setting boundaries for the solution. Knowing what to accomplish, and when to know it is done. Setting the length, style, and description of the audience can help AI understand its mission. But we have to know that first in order to instruct.
---
As someone who asks GPT and junior developers for lots of things, there are a lot of similarities. I don't imagine that is going away, at least until we wire LLMs up to a huge amount of rapidly changing, cross-silo context so it could understand "Fix the monitoring that slowed our recognition of yesterday's bug". So being thoughtful isn't going away. The author agrees with that (see above), but doesn't make clear where he draws the boundary between "being thoughful" and "prompt engineering"
IMO - Folks seem to be reacting to the term “engineer” in the same way traditional engineers with engineering exams do to software engineers claiming to be engineers despite often graduating from a LAS school or not at all - I have a bunch of PE EE in my family and they treat my claim to be an engineer with extreme scorn.
1) That as tools get better, operator specialization is less useful. I understand how this feels like it makes sense, because for for binary tasks like driving it’s true, but for creative tasks, I struggle to think of a single example where this is the case. Which leads to
2) Humans won’t figure out how to use these tools in increasingly complex/weird ways to create increasingly complex/weird outputs.
I think in general, everyone has been so conditioned by the idea of singularity (which to be frank is a completely tangential concept to contemporary LLMs) that they refuse to see these things for what they are: tools built by humans to serve humans when operated by humans.
Add as many layers of “self-prompting” as you want there, but a human still set the original intention and they will be the ones to judge the ultimate outputs.
if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff.
I think this is more for people, though, that want to maximize the use of AI in their own field, isn't it? The "knowledge worker enhancer"?
I am not sure they really need to work on the actual AI stuff...
The latter latches on to the surface level of any new tech but proceeds with swagger befitting a Nobel laureate.
- a C developer that writes modem drivers at a telecom company in the 90s about html/perl developers building first interactive websites
Is probably what someone said many decades ago.
Building computers, operating systems and compilers was always a niche.
So if you're a computer scientist or an electrical engineer, sure.
But if you're not, there was, is and will be "power users" who make the most of it.
this is different from LLMs because they don't have explicitly constructed abilities that we can compare across models. so we need to step back and approach every model as brand new and figure out what they're capable of. just because you have a prompt that works amazingly in one model, there's no guarantee that that prompt will continue to work in bigger "better" models.
so rather than working to devise way to trick a particular model into doing your task, it would probably be a better use of your time to learn how to train/modify models to explicitly solve the problem you care about.
Prompt engineering is not the same as programming. In some regards, it is “better”. In other regards, it is “worse”. They’re two different yet similar disciplines, each with their own strengths and weaknesses, but both equally legitimate.
Programming is simply the skill of being able to communicate with a compiler/interpreter effectively.
Prompt engineering is simply the skill of being able to communicate with an LLM effectively.
I wish we weren’t so divided over an incredible new technology.
https://www.reddit.com/r/datascience/comments/14nbwfv/where_...
But, even today prompts and rules of writing are not transferable even between existing LLMs. Future LLMs will have different architectures and requirements. I suspect today's prompt will be split into data and prompt. Probably just references, keywords to actual data.
In other words current state is transitory, next will be very different.
Back in the day, when those new fangled relationship database things came on the scene, do you think people would have been well advised to try and find work on the actual database engine itself, instead of the more frivolous work of using the new technology to, say, solve actual business problems?
I don’t think LLM offer standardized API like that ?
If you are not rigorous, then what you are doing is essentially "black art". It may work for some tasks ad-hoc, but with the rapid pace of model improvement your skill will likely become irrelevant/not needed quickly.
You need some amount of experimentation to get the best results but in my experience what works for one model does nothing or worsens the output in others. Adding loras and different types of images into the equation makes this so variable that I would never consider it useful besides keeping a few key words I used to get x good result on y model and experimenting with those when I start a new project.
Calling it "prompt engineering" seems odd.
I mean, you think those graphics optimizations we pour millions of dollars into before releasing AAA games will matter when the GTX 8020 outperforms a 4090?
Hint: Delivering value for actual people is rarely the result of sitting on your hands and waiting for the next big platform, or even rolling up your sleeves and trying to learn how to build the next big thing.
You have this idea that by investing energy in something that will be obsoleted you're losing out, but spoiler, that's how 99% of software that delivers actual value works.
The cutting edge of tech almost always ends up being PaaS/SaaS serving itself:
Your mail gets to you because someone is working on software with limitations we solved decades ago.
Your paycheck ends up in your bank account because people invest a ton of time in codebases subject to problems we solved long ago.
Your anti-lock brakes aren't built on a Rust codebase, but some horrible memory unsafe mess running on a processor that's a decade out of date.
—
The reality is: 99% of the effort that goes into trying to build the next big thing goes nowhere. The expected value of you trying to learn "the actual AI stuff" to the greater world is near 0 compared to you "just prompt engineering" and putting out something that solves a pain-point nicely with GPT 3.5.
At the end of the day most of the value that gets delivered to actual users comes from engineers who went deeper into extracting value from the current thing.
We still need people to work on the next big thing so that the 1% of effort that isn't wasted can actually materialize... but in my experience the most successful engineers in that regard are still able to realize the delusion it requires, without being paralyzed by the cognitive dissonance that realization invites.
a different analogy that gets at my original concern: becoming an expert prompt engineer for an particular LLM is like becoming a power user for a piece of proprietary software that isn't getting any more updates.
In tech we take it for granted that just because there's some new hotness everyone wants to jump on it day 1. GPT 5 could drop tomorrow and if your tool delivers value using 3.5, it's not going to magically stop delivering value, and in most verticals people will prefer your battle tested 3.5 to some brand new 5.
And if 5 does simplify prompts for your use case and there'll still only be two options:
- making a the same thing as what you made with 3.5 is now trivial... in which case you still have the mindshare and the distribution solved to a degree your newly enabled clones don't.
- making a better version of what you made is now trivial... in which case you can just as trivially improve your version and already have the mindshare and distribution solved.
At the end of the day software developers often struggle to fit software into the larger ecosystem it slots into before it becomes something valuable, and to be GPT has been an amazing case study in the fact.
I think the use of the word engineering did half the damage, and I think people confusing the twitter memes with the interesting attempts at prompt engineering (via ReACT, Gorilla, etc) did the other half, but at the end of the day I think a lot of people will be left kicking themselves when "misguided prompt engineers" end up solving real useful problems in ways they didn't think were possible well before we reach the arbitrary goalposts for foundation models that people keep setting up.
Where I DO think Prompt Engineering goes away is because codegen and code orchestration of LLMs rises up to take its place. Hence Prompt Engineer -> AI Engineer https://www.latent.space/i/131896365/the-role-of-code-in-the...
The latest ChatGPT model, gpt-3.5-turbo-0613 has better system prompt steerability, and with some prompt engineering I can get GPT-4 quality results out of it at a fraction of the cost.
Ultrapopular tools like LangChain and AutoGPT are essentially just prompt engineering under the hood.
Carpenter -> Wood engineer
Tailor -> Fabric engineer
Ceramist -> Clay engineer
See how dumb it looks? And maybe the field is too new to have a specific word for it, but calling it "engineer" diminishes the level of actual engineers.
If software development (or even "programming"!) is the same, the same thing will happen. If not, any other words we steal will lose their shine too and engineers will start calling themselves something else to get it back.
It is very rare I think for completely new words to emerge nowadays.
Prompt engineer is descriptive and you can guess its meaning.
When you say diminished, are you are talking from a societal point of view?
Prompt Engineering is a fun title in a fast moving and interesting field. Everyone's taking it far too seriously.
I'm going off to become a Clay Engineer, that sounds fun!
I personally think it’s worth reserving certain titles for qualified practitioners. If you want to call yourself a Doctor, Lawyer, or Engineer, I think that should mean something.
In many countries it's more a title associated to education and specific degrees from specific schools than a license to practice some professional organisation could revoke.
While, from my limited knowledge, doctors and lawyers both seem to have some sort of license and controlling body in most, if not all, countries.
There are quite a few jurisdictions where some Engineer title is protected:
https://en.wikipedia.org/wiki/Regulation_and_licensure_in_en...
The UK seems to discriminate on a per discipline basis. Canada apparently is ambiguous, with self-regulating bodies but courts dismissing cases regarding job titles. Germany has one, but only for civil engineers (still according to the wikipedia page).
Still only relying on this wikipedia page, there are on the other hand many countries where although the title is protected, it simply requires one to have studied a certain number of years (Poland), to have completed a specific degree (Brazil, Chile, Germany), or a specific degree in one of a few select higher-education schools (France, Turkey).
Go this way, instead:
Engine-maker Road-maker Trend-maker Fabric-maker Clay-maner Wood-maker Makeup-maker Book-maker Law-maker
Each school of making shall also have ranks: Vice Chief of Book-making, Novice Fabric-maker, Treasurer of Makeup-making et cetera
Outside of tech, being an actual engineer involves universal standards, certifications and the onus and responsibility of failures that the deregulated libertarian paradise of the tech world is impossible to implement.
Actual engineers go through rigorous testing, certification and have universal standards that uphold them to meet these standards regardless of business pressure.
Until the tech and software world have anywhere near that level of scrutiny by public institutions (good luck, all you will hear is the screams of "communism!") then frankly its already looked dumb ever since developers were even called "engineers" to begin with.
I think one key difference is that (most) programs that people are used to writing tend to be deterministic (when not intentionally random...) whereas LLM prompts pretty much always end up with an nondeterministic output.
Reading that article makes me wonder if we're even talking about the same thing.
I have used several of 'the big' SaaS GPTs, and have gotten great use from them.
There is absolutely a use for these tools, but they do take some small amount of skill to get good results.
It all comes down to the context you can provide to steer the answer to what you really need. The better you can describe your current state and what you want from the answer, the better the answer will be.
My only use so far is get inspiration for type naming, and very simple scripts that i'm too lazy to write myself.
But this happens less than once a month.
1. API, concepts, etc docs with explanation. Much faster then googling then scrolling through tons of texts with blinking adds
2. Write simple things that I don't want to think about. Like in Python process all files in a directory, follow the links. Saves time.
3. Try things I don't know how to do. The recent was checking if user pressed a key without blocking in Python. Nontrivial, but possible. We went through several options till found the one which works on Ubuntu.
So, it's useful, no regrets about subscribing. Funny thing I'm using it working on toy GPT
Where are they now?
In a couple of years, perhaps it will come to be known as “machine psychology”?
I'm going to link my own snarky titled by actually okay-ish take on prompt engineering, emphasis on the actual engineering, to show that the author doesn't even know what prompt engineering is - https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...
Prompt engineering is not designing a cool prompt. It's when you start applying genuine techniques to do things that are not possible with tokens alone. For example:
"What's the definition of {apple|orange}" where {apple|orange} is the mathematical average of those two words. This is prompt engineering. Right now, Prompt Engineering is basically in Stable Diffusion through Automatic1111, it's in libraries like microsoft Guidance or LMQL, and not a whole lot else.
I think the big problem is actually finding problems you could use LLMs in. I think anyone that's played with them tends to have a good guess at whether they could or couldn't do something (although you need to really test to be sure, do some prompt engineering etc), but actually finding problems to work on that are within the realm of being solvable and useful is the hardest part imo.
Other industries have proper certifications for "engineering" that I.T. absolutely doesn't, yet someone who figured out how to center a div calls themselves a software engineer.
https://www.theatlantic.com/technology/archive/2015/11/progr... <- related article.
I reckon if you're going to give devs the right to call themselves engineers just because they write code to solve business problems, you don't have much of a leg to stand to judge people whom write natural language into a software solution, and receive output that solves business problems.
Clear away the faff and ignore the “AI is improving sooo fast ohh what a magic” remarks. What you’re left with is a tepid article about prompt engineering.