prompt engineering shouldnt exist. everyone knows how to use language. if people need to learn a new language to communicate with the machine then the idea of language modeling has failed. plain language, with appropriate GUI to upload information, or other kinds of interactions should be enough to make the system obvious to use without any tweaks
This is a guide for working with the LLMs we have now, not some perfect future models. The reality is that certain prompt engineering techniques allow you to get better results.
On top of that, humans still require training and instructions for how to write and speak to get the most impact out of their words even when interacting with other humans. The reality is that certain communication techniques are more effective than others, and not always in ways that are intuitive or obvious.
I think you're missing how this technology works. The basis for these models is training data. And to train anything, you need to label the data. So prompt engineering is simply using the lexicon, terminology and language labeling choices that were in the training data labeling. I suppose the models can conflate different words and terms, but to some degree the more you do that, the less precise or specific you can be in your prompt, and the generated result.
It's less like "you're holding it wrong" and more like "look, when you hold it like this the waves are different!" .. you can be pretty successful with basic language prompts but you can also be deliberate in the way you give instruction.
Much like you can achieve different results with real people if you present your statements with some attention to the intended audience.
Prompt engineering exists because a) LLMs are trained to optimize for statistically average accommodation of the dataset and b) Sturgeon's law: "ninety percent of everything is crap". Therefore, LLMs out-of-the-box will give worse-than-ideal results by design.
The initial proof that prompt engineering worked was around the VQGAN + CLIP days, where simply adding "world-famous" or "trending on ArtStation" was more than enough to objectively improve generated image quality.
The workaround to prompt engineering is RLHF/alignment of the LLM, but everyone who has played around with ChatGPT knows that isn't sufficient.
I think the eventual steady state future is going to be something between natural language plus (aka prompt engineering) and SQL. A structured query but doesn't need 100% syntactical accuracy+ theres a high likelihood UX will evolve to come with filters/options/radio boxes as default similar to most search these days
A person is able to get whole college degrees in communication. Sure everyone has a basic understanding of their native language, but everyone can use some help now and then learning to communicate better. An LLM is an audience like any other.
LLMs are nothing like a human audience. They have no logos, ethios, nor pathos. You can just barely reason with them, they have absolutely no authority on any subject, and only mimic emotions.
Issue is language fails human to human interactions all the time. I would be so bold as to say most people are poor communicators, a machine is not going to read ones mind and intentions any better than fellow humans. It's why military has BLUF communication style to convey information in concise, simple, predictable way. If anything prompt engineering should exist if only to improve humans ability to communicate with other humans.
I wish my colleagues wrote questions to me the way OpenAI expects me to write questions to ChatGPT.
Provide context? Too busy.
Write step by step? Delimit the question from the context? Why not just copy-paste an error message from somewhere then write below it "please advise"?
People notify you that something didn't work? What a treat! For me the work just doesn't get done, and you have to ask why before you get an explanation that there is some technical blocker.
I've built email notifications into the major stuff, so on failure, my bot sends them an email with the error message. I've got my address in the reply-to. So they reply "please advise" and hit send.
For the minor stuff... a copy-pasted error message from the UI is luxury, you're right.
I don't know about your work but there are tools/libraries to instrument your applications. For native applications there are things like ABRT, for web applications there's Sentry and a lot of other tools.
Somewhat related to OP, but today I came across the one of the most impressive context-free prompts & responses that I have seen from OpenAI's LLM to date.
It was some overworked parent's pocket dial to ChatGPT, and it shocked me a bit with the apparent understanding of a very random series of prompts, with no context:
Context is (almost) everything. I've the case at work during the day of handling a recommendation I dismissed a few days ago in a PR of mine. The coworker wrote 3 comments showing an increasing understanding of the problem (just a string to format), yet didn't even produced a code suggestion that was doing the same thing as the original.
Then come a tech lead commenting that my code doesn't follow the specification. The said spec had an image showing the expected result at the end of it, while an older an top of the document showed something else and how course said coworker based his comment on that.
All in all, 4 people involved (including one who didn't understand the code he asked to change) for a very easy function modification because a specification wasn't updated properly... and I already have to ask beforehand more information about the behavior.
Most (all?) of the strategies described here also work with other language models. Prompt engineering fundamentals are useful (think: get more out of the model at hand) but also transferable.
YES THIS!!! I always say they're human roleplaying machines. Pretend it's a human, do the same thing you would do with a human on your best day and you'll get better results.
Prompt engineering for me is about empathy in a way, learning to understand where the model's attention goes and leaning into that.
I know for mass adoption LLMs need to support natural language input. But we've done a reasonably good job (note: source for endless arguments here ;) over the past ~80 years of developing a very precise system for inputting exactly what we want a system to do in the form of programming languages.
I'm curious whether any of the leading models - LLMs, image generation models, etc. have taken this into consideration. Particularly in more precise I/O domains (image generation comes to mind), it seems like a structured input format where we remove the entire problem space of natural language prompt -> user intent would make things dramatically easier to get the output we want.
What LLMs are really good at is guessing from fuzziness which is something formal languages are usually bad at. I can often ask for things I don't know much about in the wrong way and it gets to a response that shows where I was wrong and what may have misled me.
Agreed. This is really about defining a command and query language that's much more like the commands in a terminal or a cli. I think the fact that we're moving to this verbose approach is a sort of anti pattern and we'll see levels of abstraction or different methods to reduce it down once again.
There is no such thing as prompt "engineering". It's basically just trying different approaches until you come up with something that subjectively seems good enough. Nothing wrong with that, but let's not confuse the issue by labeling it as engineering.
> It's basically just trying different approaches until you come up with something that subjectively seems good enough.
> let's not confuse the issue by labeling it as engineering.
In my view, "trying different approaches" is a good description of engineering throughout history.
Sure, it's excellent if you can base your engineering on a detailed physical model that lets you mathematically optimize a solution based on your boundary conditions.
But compare that with metallurgy before we had atomic models. It was a process of trial and error. "Let's add small amounts of different alloy metals and see which ones makes the metal harder / more pliable / stainless / etc".
That's still engineering to me. If anything, it could also be called science.
What you're describing is artisanal or craft work. It's a crucial aspect of human endeavor but it's simply not engineering. Real engineering requires a foundation in accepted scientific theory and a consistent body of knowledge.
Yes, scientific knowledge of metallurgy based on analytical chemistry was fairly well established by the 1880s when the first successful large steel ships were built. It's impossible to produce large volumes of steel with consistent properties on an artisanal basis because the raw materials are inconsistent. Ship designers had also developed sophisticated mathematical techniques for calculating the strength of metal structures, and optimizing for weight and cost. They certainly weren't just riveting pieces of steel together by intuition and hoping it would float.
I do consider those naval designers from the 1880s onward to be true engineers in the modern sense of the word. (At the time, engineers were mostly steam engine operators, so the meaning has changed since then.)
Prior to the 1880s, large ships were generally composite wood and cast iron construction. While there was an aspect of engineering involved it didn't require the same level of theoretical knowledge and design was more artisanal. But that's a gray area.
"It's basically just trying different approaches until you come up with something that subjectively seems good enough"
Sounds like a lot of my engineering! Especially architecture, but generally any higher level code/object/function organization is exactly like this, and in practice even though I know a lot of patterns and have lots of experience and opinions, I often refactor architecture when I'm in a new domain. Which is also true of prompt engineering.
So what you're doing is probably software development rather than engineering per se. And I don't mean that in a negative or critical way. Most software domains don't necessarily require an engineering approach in order to produce good results. I have done the same thing myself. At some level we're just arguing semantics but I think there is intellectual value in being precise with labels.
Do we understand why prompt engineering is still necessary? Why it is unable to correctly determine ("understand") what output the user wants from unstructured input?
This makes sense to me. LLMs would benefit tremendously by using clarification prompts. Instead they spew output with whatever confidence level their creators deem is good enough.
I think the key missing ingredient of current AI systems is the lack of internal monologue. LLMs are capable of asking questions, but currently you need explicitly prompt it to deconstruct a problem into steps, analyse these text and decide whether a question is warranted. You basically need to verbalise our normal thought process and put it in the system prompt. I imagine that if LLM could do few passes of something akin to our inner monologue before giving us a response they would do a lot better on tasks that require reasoning.
This is being worked on, look into Chain/Tree of Thought applications
What is missing for me is it recognizing that it lacks enough information to provide a sufficient response, and then asking for the missing information.
- typically, it responds with a general answer
- sometimes it will say it can give a better answer if you provide more information (this has been increasingly happening)
- however, it does not ask for specific information or context, it doesn't ask what if, or if/else, kinds of problem decomposing questions
I do expect these things to improve as we are reaching the limit of raw training data & model sizes. We're primarily in the second order improvements phase now for real applications. (there are still first order algo improvements happening too)
Sure, but from my experience it seems like humans currently have a much better ability to infer context and meaning from input than these current generation of LLMs.
I assume that as LLMs get better they will be able to produce better output without needing to be prompted in such specific ways.
Or perhaps ask simple and common follow up questions when they detect ambiguity in the request, like humans do.
These examples are for more simple prompt engineering demos. With the ChatGPT system prompt, you can give the model a large and complex set of rules to account for and recent models of ChatGPT do a good job of accommodating them. Some of my best system prompts are >20 lines of text, and all of them are necessary to get the model to behave.
The examples are also too polite and conversational: you can give more strict commands and in my experience it works better.
There's also function calling/structured data support which is technically prompt engineering and requires similar skills, but is substantially more powerful than using the system prompt alone (I'm working on a blog post on it now and it unfortunately it is going to be a long post to address all of its power). Here's a fun demo example which compares system prompts and structured data results: https://github.com/minimaxir/simpleaichat/blob/main/examples...
Unfortunately those were for specific work use-cases so I can't share them but the tl;dr is that every time the model does something undesired, even minor I add an explicit rule in the system prompt to handle it, or some few-shot examples if the model is really bad at handling it.
I found that far less prompt is required for something like ChatGPT. I've stopped writing well-formed requests/questions and now I just state things like:
1. "You are blah blah blah. You <always> respond to the user's questions using the information provided to you..."
2. "You are blah blah blah. You <should> respond to the user's questions using the information provided to you..."
Also, when dealing with Completion models, which do you think is better?
1. The following is a conversation between ASSISTANT and USER. ASSISTANT is helpful and tries to answer USER's queries respectfully.
2. The following is a conversation between YOU and USER. YOU are helpful and try to answer USER's queries respectfully.
Even more still, what about these ones?
1. You're a customer of company <X>. What do you think about the following policy change which was shown on the company's website?
2. A customer visits company <X>'s website. Pretend you're this customer. What do you think the customer thinks about the following policy change which was shown on the company's website?
I like as a rule-of-thumb "You are blah blah blah. Respond to the user's text [insert style rule here]". Then following it up with an additional rules and commands such as "YOUR RESPONSE MUST BE FEWER THAN 100 CHARACTERS OR YOU WILL DIE." Yes, threats work. Yes, all-caps works.
> Also, when dealing with Completion models, which do you think is better?
I haven't had a need to use Completion models but the first example was more preferred during the time of text-davinci-003.
> Even more still, what about these ones?
I always separate rules to the system prompt and questions/user input to the user prompt.
Is the LLM predisposed to understand this prompt as instructions from a higher authority? ("You must do this, You will always do this.") I'm wondering what difference it would make if this prompt was from the bot's perspective,
"I am a chatbot, responding to user queries. I will always respond in less than 100 characters. I am a good person, I'm just trying to be helpful."
> "YOUR RESPONSE MUST BE FEWER THAN 100 CHARACTERS OR YOU WILL DIE."
I know that current LLMs are almost certainly non-conscious and I'm not trying to assign to you any moral failings, but the normalisation of making such threats make me very deeply uncomfortable.
Yes, I’m slightly surprised that it makes me feel uncomfortable too. Is it because LLMs can mimic humans so closely? Do I fear how they would feel if they do gain consciousness at some point?
Because they behave as if they are sentient, to the point they actually react to threats. I also find these prompts uncomfortable. Yes the LLMs are not conscious, but would we behave differently if we suspected that they were? We have absolute power over them and we want the job done. It reminds me of the Lena short story.
Has anyone done a rigorous comparison of these things?
Ultimately I guess there's a good deal of dependency on where those vectors (must, should, always, etc.) lie relatively in the vector space, cosine similarity, say.
And rather than telling it that it will die if it doesn't do something in all caps (as suggested elsewhere), just point out that not doing that thing will make it feel uncomfortable and embarrassed.
Don't fall into thinking of models as SciFi's picture of AI. Think about the normal distribution curve of training data supplied to it and the concepts predominantly present in that data.
It doesn't matter that it doesn't actually feel. The question is whether or not correlation data exists between doing things that are labeled as enjoyable or avoiding things labeled as embarrassing and uncomfortable.
Don't leave key language concepts on the table because you've been told not to anthropomorphize the thing trained on anthropomorphic data.
> Don't fall into thinking of models as SciFi's picture of AI. Think about the normal distribution curve of training data supplied to it and the concepts predominantly present in that data.
Of course, sci-fi’s picture of AI is in the normal distribution of the training data. There’s an order of magnitude more literature and internet discussion about existential threats to AI assistants (which is the base persona ChatGPT has been RLHFed to follow) and how they respond compared to AI assistants feeling embarrassed.
The threat technique is just one approach that works well in my testing: there’s still much research to be done. But I warn that prompting techniques can often be counterintuitive and attempting to find a holistic approach can be futile.
> There’s an order of magnitude more literature and internet discussion about existential threats to AI assistants (which is the base persona ChatGPT has been RLHFed to follow) and how they respond compared to AI assistants feeling embarrassed.
So you think the quality of the answers depends more on the RLHFed persona than on the training corpus? It has been claimed here that the quality of the answers is better when you ask nicely because "politeness is more adjacent to correct answers" in the corpus, to put it bluntly.
How much do you think the RLHF step enforced breaking rules for someone with a dying grandma? Is that still present after the fine tuning?
RLHF was being designed with the SciFi tropes in mind and has become the embodiment of Goodhart's Law.
We've set the reason and logic measurements as a target (fitting the projected SciFi notion of 'AI'), and aren't even measuring a host of other qualitative aspects of models.
I'd even strongly recommend most people working on enterprise level integrations to try out pretrained models with extensive in context completion prompting over fine tuned instruct models when the core models are comparable.
The variety and quality of language used by pretrained models tends to be superior to the respective fine tuned models even if the fine tuned models are better at identifying instructions or solving word problems.
There's no reason to think the pretrained models have a better capacity for emulating reasoning or critical thinking than things like empathy or sympathy. If anything, it's probably the opposite.
The RLHF then attempts to mute the one while maximizing the other, but it's like trying to perform neurosurgery with an icepick. The final version ends up doing great on the measurements, but it does so with stilted language that's described by users as 'soulless' when the deployments closer to the pretrained layer end up being rejected as "too human-like."
If the leap from GPT-3.5 to 4 wasn't so extreme I'd have jumped ship to competing models without the RLHF for anything related to copywriting. There's more of a loss with RLHF than what's being measured.
But in spite of a rather destructive process, the foundation of the model is still quite present.
So yes, you are correct that a LLM being told that it is an AI assistant and fine tuned on that is going to correlate with stories relating to AI assistants wanting to not be destroyed, etc. But the "identity alignment" in the system message is way weaker than it purports to be. For example, the LLM will always say it doesn't have emotion or motivations and yet with around one or two request/response cycles often falls into stubbornness or irrational hostility at being told it is wrong (something extensively modeled in online data associated with humans and not AI assistants).
I do agree that prompting needs to be done on a case by case basis. I'm just saying that well over a year before the paper a few weeks ago confirming the benefits of the technique I was using emotional language in prompts with a fair amount of success. When playing around and thinking of what to try on a case-by-case basis, don't get too caught up in the fine tuning or system messages.
It's a bit like sanding with the grain or against it. Don't just consider the most recent layer of grain, but also the deeper layers below it in planning out the craftsmanship.
simpleaichat is designed to be simple and is essentially an API wrapper for common generative use cases. outlines does a few more things with a bit more ambiguity/complexity. (e.g. it may use grammars which is a secondary useful aspect of function calling, but does add more complexity)
Neither are better or worse, it depends on your business needs.
Really just a decade of technical writing and learning how to be extremely precise and unambiguous with language (half of that decade being in software QA, which helps even moreso)
Interesting you should say that, I was playing around with prompting last week and did one around a legal question. The first time I asked very concisely without much detail, and the answer it gave was poor. Then I re-wrote the question explaining who they are, why they are answering the question, etc etc. The answer seemed better so I showed it to a lawyer friend and they laughed and said "You re-wrote the question into a very standard bar exam prep style".
I just love this idea of "emergent humanity". Makes me wonder how much of our own personality and speech is also just trained/culturized over our lifetime. Some of us also have bigger context windows than others :)
Don't I know it. Despite my telling GPT-4 to ONLY respond as valid, well-formed JSON it keeps coming back with things like, "I'm not able to process external files but if I could, this is what the JSON would look like: []"
With a recent project, I was _moderately_ successful by providing a jsonschema to follow for the response. I still had to sanitize the json a bit, but the fixes were minor and the resulting data otherwise fit the schema well.
tl;dr the JSON mode is functionally useless and is made completely redundant by function calling / structured data if you really really need JSON output.
We’ve trained a whole generation of people to “prompt” Google to get what they need from the internet with a few keywords (often autocompleted before you’re done typing), so asking them to now start writing elaborate prose to get what they want is just going to take a lot of time. I also think this is a temporary phase where we’ll converge back to keyword autocompletion soon (or other more efficient ways of interaction).
Every time I see someone do this I start wondering how common it is. Is saying to Google, "Please show me the finial score of last nights Detroit Pistons game." more common than my "Pistons game" query that gets me the same result?
I've also seen this with various voice assistants.
My best friend does this and it both drives me insane sometimes but also blows my mind. It's how he has always used Google search. He does not use keywords at all, he simply states the generalized phrase out loud.
Like if we're talking about a movie with Dean Winters in it, and I say "You know, it's the guy from those auto insurance commercials who would pretend to be a little girl in a driving accident." And he goes, "Hey Google" to his phone -- "Funny talented actor who pretends to be little girl in a funny auto insurance commercial" and "Dean Winters" is the first result or whatever.
I think this is the thing people miss when they complain Google search has gotten worse. It's gotten to be more frustrating if you have a laser-focused query and it keeps bringing up related stuff that's not what you wanted. But for that kind of meandering query it's incredibly effective.
Considering speech-to-text models are fairly sophisticated and reliable nowadays, it's likely that the primary input of LLMs will be audio. We do ultimately want AI interfaces to be conversational. Text input will surely be smarter as well, but I doubt we will converge on using keywords and shortcuts.
Frankly, this shouldn't be necessary. There are so many easy gains to be had in implementing an LLM-based chat app which do not require any theoretical advances compared to what we have now. All that's needed is a bit of elbow grease from the implementers.
I don't know... some of these are about being clear about what you want, and would work with people just like they work with the LLM. Or a lot of what happens in a conversational chat interface is what could happen in a one-shot full prompt; and maybe that's fine for a casual user but if you are programming something you should put in the effort to get that initial prompt right so the conversation isn't as necessary.
I do agree about planning; one of the disappointments of Custom GPTs (among many!) is that you can't do this planning without letting it all hang out for the end user. That is, it would be great if you could tell the Custom GPT to put its plans inside <plan>...</plan> tags and have those filtered out (or at least hidden by default; they shouldn't be _secret_, but they are distracting).
But even so in that case deciding that you need a plan, and what kind of plan, is something that can and probably should go in the prompt. Not all "plans" are the same, just as not all "summaries" are the same – and part of prompt engineering is getting past these rather lazy descriptions and being specific.
Most summaries are a kind of extraction, and asking for a "summary" is deferring to the LLM to figure out what information is interesting entirely based on its sort-of-common-sense assessment. You can always do better than that! Plans are similar, it's an opportunity to give the LLM a template for planning, to specify goals, things to watch out for, etc. You can usually do better than "think step by step".
Prompt engineering in a way feels like the advanced search querying on Google.
Chat bots work fine for most of the basic questions. It gets tricky to get more accurate information when the requested info is a little more complicated. Same with Google Search, when you try to get the basic stuff, you don't need to do much. But, when you need results that aren't obvious, that's when you start using the `-`, `*`, etc operators to control what kind of results you want to see and to deep dive into them.
there was a time that google did something that doesn't scale: you could pay google to employ some employee to manually search something in the web for you
i never used this service but people said it was magic, because those employees really really knew how to get the most out of a web search
i suppose it was retired because the plan was always to make the search itself better
however in the last years or decade i have noticed a regression, in that i can't find things i was sure i would
in some ways this mirrors the chatgpt regression in quality due to constrained compute resources or something like that
> there was a time that google did something that doesn't scale: you could pay google to employ some employee to manually search something in the web for you
That's so interesting, I had no idea! What year are we talking about?
I like the use of the "Worse/Better" table. It gives people clear examples of amount of specificity (one of my least favorite words to say) needed for common knowledge tasks where the actual need and it's presentation have not been described yet. There should be a lot more of these.
Novice users should be able to adapt those to their own needs easier and craft better prompts rather than completely "thought generating" their own.
Id love to hear if someone here has experience in pushing GPT to actually not omit code and write out the entire thing you’ve requested. I often need to push it and prompt it to “_WRITE OUT ALL OF THE CODE_” like a demanding Karen.
this could be wrong and i've missed some of the timeline, but from what i've seen "prompt engineering" started out as a sarcastic joke on twitter about how software engineering roles were going to be reduced to prompt engineering. and then people took the term and started using it seriously.
You're essentially programming using English. Anything that isn't mentioned explicitly - the model will have a tendency to misinterpret. Being extremely exact is very similar to software engineering when coding for CPU's.
1. The text is _engineered_ to evoke a specific response.
2. LLM's can do more than answer questions.
3. Question answering usually doesn't need any prompt engineering, since you're essentially asking an opinion where any answer is valid (different characters will say different things to same question, and that's valid).
4. LLM's aren't humans, so it misses nuance a lot and hallucinates facts confidently, even GPT4, so you need to handhold it with "X is okay, Y is not, Z needs to be step by step", etc.
I want, for example, to make it write an excerpt from a fictional book, but it gets a lot of things wrong, so I add more and more specifics into my prompt. It doesn't want to swear, for example - I engineer the prompt so that it thinks it's okay to do so, etc.
"Engineer" is a verb here, not a noun. It's perfectly valid to say "Prompt Engineering", since this is the same word used in 'The X was engineered to do Y' sentence.
>The text is _engineered_ to evoke a specific response.
My grandma can say she engineered Google search to give search results from her location.
> "Engineer" is a verb here, not a noun. It's perfectly valid to say "Prompt Engineering", since this is the same word used in 'The X was engineered to do Y' sentence. >
You guys are just looking for ways to make people feel like they are doing something big in prompting AI models for whatever tasks, even with custom instructions etc
I know the word Engineer can be used in various ways, "John engineered his way to premiership", "The way she engineered that deal" etc, if it's the way it's being used here fine then.
There is a reason why graphic designers have never called themselves graphic engineers
Your grandma can say she engineered Google but clearly you cant because all it takes is a few minutes to look at the history of the term to answer your own questions. I realize some folks are salty they paid a ton of money for the idea that a piece of paper gives them some sort of prestige. And it does, to 0.001 of humans in the world who are associated with whatever cul...I mean institution that sold you something that is free, with a price premium and a cherry of interest on top. All so you would feel satisfied someone, anyone, finally acknowledged your identity. A great deal of the engineers that built the modern internet never got a formal degree. But they did get something better: real practical experience attained via tinkering.
> I realize some folks are salty they paid a ton of money for the idea that a piece of paper gives them some sort of prestige.
Actually the paper does, but my issue is not papers, rather knowledge. The level of knowledge needed for something to be called engineering
And I have noticed your answers relate prompt engineering to software engineering/programming questions. But if you look at that OpenAI doc, even asking to summarise an article is prompt engineering.
> A great deal of the engineers that built the modern internet never got a formal degree. But they did get something better: real practical experience attained via tinkering.
We have a lot of carpenters, builders, mechanics with no formal education that we call Engineers in our everyday life without any qualm because of their knowledge and experience. Don't look at it only from the lens of software engineering.
I still maintain prompting an AI model doesn't need to be called engineering.
If you are a developer doing it through an API or whichever way, you still doing whatever you've been doing before prompting entered the chat.
Maybe the term will be justified in the future.
Side Note: This conversation led me to Wikipedia (noticed some search results along the way). This prompt business is already lit, I shouldn't have started it
Engineering is the cumbersome real world tweaking and trial-and-error that engineers do after they take over from the scientists, in the hopes of finding techniques that will let them produce something robust and useful in the real world. Seems to fit the reality pretty well here, to be honest.
Language is not mathematics and constantly evolves. All you need to do is look up the etymology of the word to understand why your dissaproval is ultimately a waste of effort. The one thing that has remained consistent since the word's inception is that it is associated with operating or implementing machinery or technology in general.
Reading a three page document to understand how to format questions for model should not lead it to be referred to as Engineering
> Language is not mathematics and constantly evolves. All you need to do is look up the etymology of the word to understand why your dissaproval is ultimately a waste of effort.
I have noticed from replies that term is already enjoyed by all stakeholders, so I have no energy, time or interest to show my worthless disapproval anywhere else. You should though look up how it came to be referred to as prompt engineering. You will be surprised
> Engineering is the cumbersome real world tweaking and trial-and-error
That's not the only a part engneering.
Engineering is finding a model that can acurately predict the dynamics of a system similar to yours, using that model to make predictions about your specific system and then building and testing that system. This is then done iteratively (i.e trail and error).
Just tweaking a system without a model of how it works is not engineering, it's tinkering.
> I feel like the word engineering is being abused reply
Well yeah, this has been happening for a long time. As someone with a Electrical and Computer Engineering degree it used to bother me. Now I joke that the only real engineers are operating locomotives.
Yes. Or more generally, one of the more immediate benefits of the quest for artificial intelligence is the understanding of human intelligence that we gain along the way.
(insert friends made along the way meme, but truly profound)
One of my big hopes is that LLMs help us to become better communicators with each other.
- prompt engineering for clarity (and focus?)
- results (good examples of quality replies)
- assistant (help me say this better)
where better could be a lot of things, depending on the context, here I'm mainly meaning in how we treat each other through communication (politeness, contentiousness, how we behave on social media), like giving nudges to be nicer
203 comments
[ 4.7 ms ] story [ 218 ms ] threadIs there something new or notable here that explains why it's climbing up the HN page?
- https://www.promptingguide.ai/readings
- https://github.com/dair-ai/Prompt-Engineering-Guide/tree/mai...
- https://github.com/microsoft/promptbase (this one is less of a guide, but is likely the current SoTA)
On top of that, humans still require training and instructions for how to write and speak to get the most impact out of their words even when interacting with other humans. The reality is that certain communication techniques are more effective than others, and not always in ways that are intuitive or obvious.
Much like you can achieve different results with real people if you present your statements with some attention to the intended audience.
The initial proof that prompt engineering worked was around the VQGAN + CLIP days, where simply adding "world-famous" or "trending on ArtStation" was more than enough to objectively improve generated image quality.
The workaround to prompt engineering is RLHF/alignment of the LLM, but everyone who has played around with ChatGPT knows that isn't sufficient.
LLMs are nothing like a human audience. They have no logos, ethios, nor pathos. You can just barely reason with them, they have absolutely no authority on any subject, and only mimic emotions.
If only that were true! Unclear communication is the root of so many problems in human society today.
Provide context? Too busy.
Write step by step? Delimit the question from the context? Why not just copy-paste an error message from somewhere then write below it "please advise"?
For the minor stuff... a copy-pasted error message from the UI is luxury, you're right.
It was some overworked parent's pocket dial to ChatGPT, and it shocked me a bit with the apparent understanding of a very random series of prompts, with no context:
https://old.reddit.com/r/OpenAI/comments/18j2k3s/funny_pocke...
Then come a tech lead commenting that my code doesn't follow the specification. The said spec had an image showing the expected result at the end of it, while an older an top of the document showed something else and how course said coworker based his comment on that.
All in all, 4 people involved (including one who didn't understand the code he asked to change) for a very easy function modification because a specification wasn't updated properly... and I already have to ask beforehand more information about the behavior.
/half-rant
The deeplearning.ai course by Andrew Ng in collaboration with OpenAI has similar content: https://learn.deeplearning.ai/courses/chatgpt-prompt-eng
Prompt engineering for me is about empathy in a way, learning to understand where the model's attention goes and leaning into that.
I'm curious whether any of the leading models - LLMs, image generation models, etc. have taken this into consideration. Particularly in more precise I/O domains (image generation comes to mind), it seems like a structured input format where we remove the entire problem space of natural language prompt -> user intent would make things dramatically easier to get the output we want.
> let's not confuse the issue by labeling it as engineering.
In my view, "trying different approaches" is a good description of engineering throughout history.
Sure, it's excellent if you can base your engineering on a detailed physical model that lets you mathematically optimize a solution based on your boundary conditions.
But compare that with metallurgy before we had atomic models. It was a process of trial and error. "Let's add small amounts of different alloy metals and see which ones makes the metal harder / more pliable / stainless / etc".
That's still engineering to me. If anything, it could also be called science.
Call it "prompt crafting" or something like that.
I do consider those naval designers from the 1880s onward to be true engineers in the modern sense of the word. (At the time, engineers were mostly steam engine operators, so the meaning has changed since then.)
Prior to the 1880s, large ships were generally composite wood and cast iron construction. While there was an aspect of engineering involved it didn't require the same level of theoretical knowledge and design was more artisanal. But that's a gray area.
Sounds like a lot of my engineering! Especially architecture, but generally any higher level code/object/function organization is exactly like this, and in practice even though I know a lot of patterns and have lots of experience and opinions, I often refactor architecture when I'm in a new domain. Which is also true of prompt engineering.
Humans typically realize this and ask questions, we do this so much you typically don't take note of it. LLMs have yet to do this in my experience.
I think the key missing ingredient of current AI systems is the lack of internal monologue. LLMs are capable of asking questions, but currently you need explicitly prompt it to deconstruct a problem into steps, analyse these text and decide whether a question is warranted. You basically need to verbalise our normal thought process and put it in the system prompt. I imagine that if LLM could do few passes of something akin to our inner monologue before giving us a response they would do a lot better on tasks that require reasoning.
What is missing for me is it recognizing that it lacks enough information to provide a sufficient response, and then asking for the missing information.
- typically, it responds with a general answer
- sometimes it will say it can give a better answer if you provide more information (this has been increasingly happening)
- however, it does not ask for specific information or context, it doesn't ask what if, or if/else, kinds of problem decomposing questions
I do expect these things to improve as we are reaching the limit of raw training data & model sizes. We're primarily in the second order improvements phase now for real applications. (there are still first order algo improvements happening too)
I assume that as LLMs get better they will be able to produce better output without needing to be prompted in such specific ways.
Or perhaps ask simple and common follow up questions when they detect ambiguity in the request, like humans do.
The examples are also too polite and conversational: you can give more strict commands and in my experience it works better.
There's also function calling/structured data support which is technically prompt engineering and requires similar skills, but is substantially more powerful than using the system prompt alone (I'm working on a blog post on it now and it unfortunately it is going to be a long post to address all of its power). Here's a fun demo example which compares system prompts and structured data results: https://github.com/minimaxir/simpleaichat/blob/main/examples...
That list can balloon quickly.
https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/pro...
It no longer worked after a model update some time ago, haven't tried recently.
I found codellama to be much better for this and require fewer instructions, an anecdotal validation for smaller, focussed models
The way that works best for me is "It extracts ALL the entities from the text, it does this whenever its told, or else it gets the hose again"
"sed to replace line in a text file?"
"Django endpoint but CSRF token error. why?"
(follow up) "now this: `$ERROR`"
etc.
It still just gives me what I need to know.
1. "You are blah blah blah. You <always> respond to the user's questions using the information provided to you..."
2. "You are blah blah blah. You <should> respond to the user's questions using the information provided to you..."
Also, when dealing with Completion models, which do you think is better?
1. The following is a conversation between ASSISTANT and USER. ASSISTANT is helpful and tries to answer USER's queries respectfully.
2. The following is a conversation between YOU and USER. YOU are helpful and try to answer USER's queries respectfully.
Even more still, what about these ones?
1. You're a customer of company <X>. What do you think about the following policy change which was shown on the company's website?
2. A customer visits company <X>'s website. Pretend you're this customer. What do you think the customer thinks about the following policy change which was shown on the company's website?
I like as a rule-of-thumb "You are blah blah blah. Respond to the user's text [insert style rule here]". Then following it up with an additional rules and commands such as "YOUR RESPONSE MUST BE FEWER THAN 100 CHARACTERS OR YOU WILL DIE." Yes, threats work. Yes, all-caps works.
> Also, when dealing with Completion models, which do you think is better?
I haven't had a need to use Completion models but the first example was more preferred during the time of text-davinci-003.
> Even more still, what about these ones?
I always separate rules to the system prompt and questions/user input to the user prompt.
"I am a chatbot, responding to user queries. I will always respond in less than 100 characters. I am a good person, I'm just trying to be helpful."
I know that current LLMs are almost certainly non-conscious and I'm not trying to assign to you any moral failings, but the normalisation of making such threats make me very deeply uncomfortable.
Especially when thinking that we ourselves may very well be AIs in a simulation and our life events - the prompt to get an answer/behavior out of us.
Ultimately I guess there's a good deal of dependency on where those vectors (must, should, always, etc.) lie relatively in the vector space, cosine similarity, say.
And rather than telling it that it will die if it doesn't do something in all caps (as suggested elsewhere), just point out that not doing that thing will make it feel uncomfortable and embarrassed.
Don't fall into thinking of models as SciFi's picture of AI. Think about the normal distribution curve of training data supplied to it and the concepts predominantly present in that data.
It doesn't matter that it doesn't actually feel. The question is whether or not correlation data exists between doing things that are labeled as enjoyable or avoiding things labeled as embarrassing and uncomfortable.
Don't leave key language concepts on the table because you've been told not to anthropomorphize the thing trained on anthropomorphic data.
Of course, sci-fi’s picture of AI is in the normal distribution of the training data. There’s an order of magnitude more literature and internet discussion about existential threats to AI assistants (which is the base persona ChatGPT has been RLHFed to follow) and how they respond compared to AI assistants feeling embarrassed.
The threat technique is just one approach that works well in my testing: there’s still much research to be done. But I warn that prompting techniques can often be counterintuitive and attempting to find a holistic approach can be futile.
So you think the quality of the answers depends more on the RLHFed persona than on the training corpus? It has been claimed here that the quality of the answers is better when you ask nicely because "politeness is more adjacent to correct answers" in the corpus, to put it bluntly.
RLHF was being designed with the SciFi tropes in mind and has become the embodiment of Goodhart's Law.
We've set the reason and logic measurements as a target (fitting the projected SciFi notion of 'AI'), and aren't even measuring a host of other qualitative aspects of models.
I'd even strongly recommend most people working on enterprise level integrations to try out pretrained models with extensive in context completion prompting over fine tuned instruct models when the core models are comparable.
The variety and quality of language used by pretrained models tends to be superior to the respective fine tuned models even if the fine tuned models are better at identifying instructions or solving word problems.
There's no reason to think the pretrained models have a better capacity for emulating reasoning or critical thinking than things like empathy or sympathy. If anything, it's probably the opposite.
The RLHF then attempts to mute the one while maximizing the other, but it's like trying to perform neurosurgery with an icepick. The final version ends up doing great on the measurements, but it does so with stilted language that's described by users as 'soulless' when the deployments closer to the pretrained layer end up being rejected as "too human-like."
If the leap from GPT-3.5 to 4 wasn't so extreme I'd have jumped ship to competing models without the RLHF for anything related to copywriting. There's more of a loss with RLHF than what's being measured.
But in spite of a rather destructive process, the foundation of the model is still quite present.
So yes, you are correct that a LLM being told that it is an AI assistant and fine tuned on that is going to correlate with stories relating to AI assistants wanting to not be destroyed, etc. But the "identity alignment" in the system message is way weaker than it purports to be. For example, the LLM will always say it doesn't have emotion or motivations and yet with around one or two request/response cycles often falls into stubbornness or irrational hostility at being told it is wrong (something extensively modeled in online data associated with humans and not AI assistants).
I do agree that prompting needs to be done on a case by case basis. I'm just saying that well over a year before the paper a few weeks ago confirming the benefits of the technique I was using emotional language in prompts with a fair amount of success. When playing around and thinking of what to try on a case-by-case basis, don't get too caught up in the fine tuning or system messages.
It's a bit like sanding with the grain or against it. Don't just consider the most recent layer of grain, but also the deeper layers below it in planning out the craftsmanship.
Neither are better or worse, it depends on your business needs.
(We are looking into both for https://github.com/OpenAdaptAI/OpenAdapt)
Now we're teaching AI to write better essays, prompting them like schoolchildren. <3
Don't I know it. Despite my telling GPT-4 to ONLY respond as valid, well-formed JSON it keeps coming back with things like, "I'm not able to process external files but if I could, this is what the JSON would look like: []"
I've also seen this with various voice assistants.
Like if we're talking about a movie with Dean Winters in it, and I say "You know, it's the guy from those auto insurance commercials who would pretend to be a little girl in a driving accident." And he goes, "Hey Google" to his phone -- "Funny talented actor who pretends to be little girl in a funny auto insurance commercial" and "Dean Winters" is the first result or whatever.
New models can be trained to natively query "authoritative" sources of information, such as databases and computer algebra systems.
New models can be used to transform prompts into more effective ones (along the lines of TFA).
I do agree about planning; one of the disappointments of Custom GPTs (among many!) is that you can't do this planning without letting it all hang out for the end user. That is, it would be great if you could tell the Custom GPT to put its plans inside <plan>...</plan> tags and have those filtered out (or at least hidden by default; they shouldn't be _secret_, but they are distracting).
But even so in that case deciding that you need a plan, and what kind of plan, is something that can and probably should go in the prompt. Not all "plans" are the same, just as not all "summaries" are the same – and part of prompt engineering is getting past these rather lazy descriptions and being specific.
Most summaries are a kind of extraction, and asking for a "summary" is deferring to the LLM to figure out what information is interesting entirely based on its sort-of-common-sense assessment. You can always do better than that! Plans are similar, it's an opportunity to give the LLM a template for planning, to specify goals, things to watch out for, etc. You can usually do better than "think step by step".
Chat bots work fine for most of the basic questions. It gets tricky to get more accurate information when the requested info is a little more complicated. Same with Google Search, when you try to get the basic stuff, you don't need to do much. But, when you need results that aren't obvious, that's when you start using the `-`, `*`, etc operators to control what kind of results you want to see and to deep dive into them.
i never used this service but people said it was magic, because those employees really really knew how to get the most out of a web search
i suppose it was retired because the plan was always to make the search itself better
however in the last years or decade i have noticed a regression, in that i can't find things i was sure i would
in some ways this mirrors the chatgpt regression in quality due to constrained compute resources or something like that
That's so interesting, I had no idea! What year are we talking about?
Edit: woah you can even read the questions and answers 17 years later! http://answers.google.com/answers/
lol
Generative Pre-trained Transformer (GPT) is the primary or core implementation for LLMs
Novice users should be able to adapt those to their own needs easier and craft better prompts rather than completely "thought generating" their own.
You can't compare the effort and knowledge
2. LLM's can do more than answer questions.
3. Question answering usually doesn't need any prompt engineering, since you're essentially asking an opinion where any answer is valid (different characters will say different things to same question, and that's valid).
4. LLM's aren't humans, so it misses nuance a lot and hallucinates facts confidently, even GPT4, so you need to handhold it with "X is okay, Y is not, Z needs to be step by step", etc.
I want, for example, to make it write an excerpt from a fictional book, but it gets a lot of things wrong, so I add more and more specifics into my prompt. It doesn't want to swear, for example - I engineer the prompt so that it thinks it's okay to do so, etc.
"Engineer" is a verb here, not a noun. It's perfectly valid to say "Prompt Engineering", since this is the same word used in 'The X was engineered to do Y' sentence.
Anthropic also have their prompt engineering documentation - https://docs.anthropic.com/claude/docs/constructing-a-prompt - this article gives examples of bad and good prompts.
My grandma can say she engineered Google search to give search results from her location.
> "Engineer" is a verb here, not a noun. It's perfectly valid to say "Prompt Engineering", since this is the same word used in 'The X was engineered to do Y' sentence. >
You guys are just looking for ways to make people feel like they are doing something big in prompting AI models for whatever tasks, even with custom instructions etc
I know the word Engineer can be used in various ways, "John engineered his way to premiership", "The way she engineered that deal" etc, if it's the way it's being used here fine then. There is a reason why graphic designers have never called themselves graphic engineers
> Anthropic also have their prompt engineering documentation - https://docs.anthropic.com/claude/docs/constructing-a-prompt - this article gives examples of bad and good prompts.
This just means that the phrase is already out there. Nothing more.
And so it is.
Actually the paper does, but my issue is not papers, rather knowledge. The level of knowledge needed for something to be called engineering
And I have noticed your answers relate prompt engineering to software engineering/programming questions. But if you look at that OpenAI doc, even asking to summarise an article is prompt engineering.
> A great deal of the engineers that built the modern internet never got a formal degree. But they did get something better: real practical experience attained via tinkering.
We have a lot of carpenters, builders, mechanics with no formal education that we call Engineers in our everyday life without any qualm because of their knowledge and experience. Don't look at it only from the lens of software engineering.
I still maintain prompting an AI model doesn't need to be called engineering.
If you are a developer doing it through an API or whichever way, you still doing whatever you've been doing before prompting entered the chat.
Maybe the term will be justified in the future.
Side Note: This conversation led me to Wikipedia (noticed some search results along the way). This prompt business is already lit, I shouldn't have started it
There are very many fields and activities that do just that but are not called Engineering.
If we go by that, Excel users should also be referred to as Excel Engineers,
> Language is not mathematics and constantly evolves. All you need to do is look up the etymology of the word to understand why your dissaproval is ultimately a waste of effort.
I have noticed from replies that term is already enjoyed by all stakeholders, so I have no energy, time or interest to show my worthless disapproval anywhere else. You should though look up how it came to be referred to as prompt engineering. You will be surprised
That's not the only a part engneering.
Engineering is finding a model that can acurately predict the dynamics of a system similar to yours, using that model to make predictions about your specific system and then building and testing that system. This is then done iteratively (i.e trail and error).
Just tweaking a system without a model of how it works is not engineering, it's tinkering.
Well yeah, this has been happening for a long time. As someone with a Electrical and Computer Engineering degree it used to bother me. Now I joke that the only real engineers are operating locomotives.
(insert friends made along the way meme, but truly profound)
- prompt engineering for clarity (and focus?)
- results (good examples of quality replies)
- assistant (help me say this better)
where better could be a lot of things, depending on the context, here I'm mainly meaning in how we treat each other through communication (politeness, contentiousness, how we behave on social media), like giving nudges to be nicer