Many think prompt engineering is just like being good at writing Google searches, where my job is to be good at knowing "how Google/GPT4 thinks"—good at writing a single query.
However, I think prompt engineering will evolve to be an actual technical role, akin to Data Engineering (the people who make the systems, pipelines, ETL jobs, etc for the data).
Prompt engineers will build systems that facilitate prompt generation. Meaning that prompts will be dynamically generated or at least partially generated with modifications or additions to the raw user prompt.
It's the difference to being able to write HTML vs being able to do all the backend work to dynamically generate the HTML for my Amazon homepage (including the performance benchmarks and other strategic requirements), for example.
I'm working on a prompt engineering product at a stealth mode startup. Would it be possible for me to pick your brain sometime? I'd like to understand more about your workflow, and how our product could fit into that.
I’ve been doing the same thing with a number of projects, building chains of prompts from one api call to another e.g. for ConjureUI (self-creating, iterable UIs that come into existence, get used, then disappear) https://youtu.be/xgi1YX6HQBw how it works to generate a full self-contained react component:
1. Take user task
2. Pass it to a prompt that requests a Product UI description of a component
3. Pass 1+2 to another that asks for which npm packages to use
4. Pass 1+2+3 to a templated prompt to write the code in a constrained manner
5. Run 4 in a sandbox to see if there are errors, if so pass it back to #4, looping
It’s currently quite slow, but that’s an implementation detail I think.
Yes, that wasn't a priority here, but I also don't think it's much of a concern with e.g. GPT-4's `system` vs `assistant` vs `user` roles. Would be another thing to work on, but nothing worth doom and gloom.
Although, 'script(/injection) kiddie' will be an interesting phenomenon in the future...
Crucially important that engineers who are building systems like that that work by concatenating prompts together have a very solid understanding of prompt injection attacks.
I don't think it will evolve at all, because models will just become better at understanding what people mean. There's no point in trying to be a better prompter as a competitive advantage.
Already, chatGPT and bing can both give great on-topic answers to 3 word queries. and the fact that you can infinitely refine it is great
OTOH i think there is space for developing GUIs for prompts. Makes them more engaging
> Already, chatGPT and bing can both give great on-topic answers to 3 word queries. and the fact that you can infinitely refine it is great
You’re completely ignoring the system prompts that OpenAI/Bing have already set up so your “3 word query” works as you intend. These system prompts are what prompt engineering is all about.
If I have an LLM in a video game that generates NPC dialog, I might want to feed more information to the prompt based on things my character has done in the game so the NPC dialogue is more relevant to me.
Maybe I want to inject the user’s location or the current weather in Sunnyvale, CA for the specific query. Or maybe I want to inject that the user is currently at Disneyland.
Maybe the user really likes a specific tv show and I want to let the model know that. Or maybe the specific TV show was DMCA’d by an IP owner and I need to put that in the prompt.
Do I need to detect that the user is below 13 yrs old and use a different prompt (or a different model altogether)?
Maybe I want the model to only ever respond with JSON without the user needing to specify. It needs to be clarified in the prompt with no way for the user to override it.
Models can be simplified to: input to output. Prompt engineering will be engineering the inputs.
Another example. Someone has yet another new programming language. Expected infrastructure was github, website with docs, editor/IDE integration, community repo, slack/mailinglist, etc. "Now" it's also a "chatoverflow" capable of answering "how do I ...?" and "translate this <code in popular language>". What does a MVP training set for this look like? Can translation be pushed to permit assimilating "batteries" from similar languages? What does the infrastructure for that look like? Does this mean a half century of agonizingly slow language evolution is about to state change?
Well, it depends on what you need from your prompt.
What engineering is required for Amazon to deliver me an HTML page?
Are the backend engineers just “injecting” HTML? Of course not. It’s not the same as generating the HTML for a personal blog.
As far as what skills are need, it depends on how any given team wants to modify the prompt. It’s up to the ingenuity of the engineers. Maybe they need to hit 5 other models before generating the prompt. IDK.
The prompt may change for any given scenario, which is why you need engineers to build the system.
It's macro substitution on crafted templatized prompts at best. Calling it engineering dilutes the term either further than it already is. (Looking at you Salesforce "Engineer").
These clearly demonstrate that there's engineering skills beyond "I want you to act as a Linux terminal."
Also note that there's a difference between the prompt that you use on ChatGPT and the one that you use in the GPT API or other LLMs.
In the first, you're already dealing with the prompt that openai supplied to the LLM in the first place.
I stopped reading your second link after seeing that the person teaching prompt engineering demonstrated a lack of understanding regarding proper sentence structure.
No. Of course that's not the reason. Please don't take your biases and make those sorts of statements about what you believe is in somebody else's thoughts.
The reason was what I stated, which has nothing to do with the provenance or capabilities of that person.
What’s funny? Try to write a chat bot built on an LLM that adapts to your personality over time and do it without EVER dynamically changing the prompt.
But why call it prompt engineer and not just data or, you know, software engineering? Prompt engineer means the prompting, and that’s already quite dead as it is: you can just ask gpt4 to fix the user input for a specific goal. We do this in our pipeline and it’s obviously (much) faster and cheaper than ‘prompt engineers’, but it is usually also simply better for what we need it for.
The stuff you say it will evolve in already exists and has names everyone uses.
I believe that a critical part of software engineering is the ability to trace and resolve an issue through many different systems and code paths (what most people call debugging)
There is no way to debug these models, there is no way to systematically fix your prompts. It is therefore completely guesswork on what words in what orders might produce results closer to what your stakeholders expect.
Software engineers, when presented with an issue, can dive down as deep as they want (hell, even to the machine code layer if it comes down to it) in order to fine-tune, fix bugs, or do whatever else a stakeholder might expect.
Two very different coins in two very different universes.
> There is no way to debug these models, there is no way to systematically fix your prompts. It is therefore completely guesswork on what words in what orders might produce results closer to what your stakeholders expect.
I think if you spend some more time with these models, or any similar model where you have X input > Y output, you notice that you can in fact "debug" those as well, without knowing 100% of the internals. Some fixes are better than others. There are better and more reliable ways to steer these models, compared to other ways. It is not 100% guesswork, some people are better at it than others.
But you can't get too far. It's like trying to design a pharmaceutical without using a microscope or chemistry to study the target system, just throwing drugs at a human model and seeing what happens.
The goal should be to be able to just talk your model without any engineering. If you as a normal user can not interact with an AI then the AI is just not smart.
I hope Andrew Ng was correct that a great many methods end up largely equivalent given the same amount of input because the current methods deliver a rather useless result.
With a google search you are getting citations and stuck finding the truly good ones that truly match, with chatgpt you are getting an answer from someone who read all those citations and treats them all as equally good.
I think the real problem was to get the most perfect citations given your specific question. We can forgive the error since it took the web a few years to discover that best is rarely a fresh bit of blog spam by a moron who has read everything and refuses to cite sources. So GPT is convincingly as good as a human blogger yet not as useful as a less human emulating tool.
When it comes to making ChatGPT solve technical problems, my observation is that you need to be good at writing (technical) requirements to be good at writing prompts.
When I assign a task to a (human) developer, the results depend on two things: First, how good the developer is, second, and more importantly, how well and clearly I am expressing the requirements. And this is also true for ChatGPT. With very precise requirements I get very good results.
So prompt engineering is like writing good requirements, and that also requires understanding the problem domain.
writing/reading/understanding good requirements is a really nice skill to have in this decade. I have a peer that just CAN NOT interpret what is going on so he always need to schedule a meeting and it pisses me off hard. "are you able to talk right now?" sigh
> You're throwing shit at the wall and hoping it sticks.
A more charitable description might be "You're employing the scientific method to extract value from GPT-like systems." Just like in science, with time you're developing intuition for how the underlying system works, but you still have to run the experiments.
This is an unhelpfully cynical take.
The job title has "engineer" in it, so a more charitable interpretation is more serious than "AI monkey".
Using LLMs to solve real problems is not easy. Making sure that you don't introduce regressions while making improvements is difficult, and requires building and evaluating a dataset, and the necessary pipelines. It may also include diversification of LLM providers, and creating the necessary abstractions. A fundamental understanding of how LLMs work, ability to compare different architectural approaches, along with typical data engineering and software development skills would be required.
What if you want to use the LLM for Question/Answer systems that requires working with embeddings? What if you want to find a way to process data locally without sending sensitive data to the LLM provider?
That is exactly why products like ChatGPT have been taking off as quick as they have.
What you're talking about is not using LLMs as a product but using them as a component within a broader system. And so of course that requires engineering skills.
That's why the word "engineer" is in the title and it isn't "prompt writer" or "ChatGPT user".
It's highly unlikely that anyone taking this effort seriously is copying and pasting from ChatGPT, rather than using the API and building pipelines as part of a broader system.
Instruction-fine-tuned LLMs like ChatGPT require creating, validating, and maintaining prompts. Finding ways to use them safely is also not easy - prompt injection and hallucination are just 2 potential pitfalls - there are many more.
Denigrating this effort as "AI monkey" is myopic at best, but really just comes across as a signal that someone is terrified of being replaced by this new tech. With that attitude, they will be.
Then you're a software engineer. Writing prompts isn't engineering. Building systems is engineering. Just because I use keyboards to program doesn't mean I'm a keyboard engineer, does it?
Writing prompts and engineering together = prompt engineer. The engineering depends on the prompt and the prompt depends on the engineering. Just like an ML engineer, or a QA engineer, or [anything] engineer. How specific the job title gets really depends on hiring criteria and daily job function.
Otherwise, the job title would be "prompt writer".
Your point is what, that existing engineering titles cover this effort? Sure, you can just call all of it software engineering, but sometimes it's useful to be more specific. The LLMs are so powerful now that this new, more specific title makes sense to me, and clearly those using this new title. We'll see how it pans out over the next few years.
This is demonstrably false for many use cases. For one broad example, LLMs have shown incredible performance on many NLU and NLP tasks, that are not currently possible using other techniques.
I'm convinced this will be a common job description for a few years, after which it will flow into and just become a part of any other job. Like Googling. I mean, we all know it does take some domain knowledge to be able to use it in your job. Also just like Googling.
We've started calling it LLMing (llemming).
Edit: Specifying prompts is leaning towards specification. I am not saying googling is that. I'm saying that, like googling, it will just be a part of the job in a not distant future.
Comment engineer. It’s a job that’s already been made redundant by prompt engineers. A prompt engineer can spin up hundreds of comments in seconds that would have taken dozens of comment engineering man hours. We should also be worried about legal engineers and legislation engineers.
Retort Engineering is mostly involved in the design stage, they can send a prototype, but you probably would be better off contacting the Technicians in “Snarky one-liners.”
Given the wild popularity of posts about prompt engineering "jobs" paying +$300k, it was only a matter of time for an indie hacker to create a job board specifically for this type of job.
I've also read the articles about prompt engineers with no experience or technical skills commanding six figures, or the one about these jobs paying +300k.
I have a BSc and Master's in Computer Science, 14 years of experience, I have slowly climbed the ladder, got a FAANG job 5 years ago, and only last year I managed to break 300k salary for the first time (working in ML of all things). Am I losing my sanity or these reports are greatly exaggerated?
> articles about prompt engineers with no experience or technical skills commanding six figure
Please share an example.
I've only seen articles claiming this might exist, based one attention-seeking job ad from one company, which didn't claim "no experience or technical skills".
OP here, I see lots of comments about prompt engineering thinking in the context of asking one question to get the answer.
In that perspective, I understand why many people think it is useless. However, if you tried to make a chain of functions/calls, or worked with a tool like LangChain [0] you will see its importance.
Ex: "Which stock had better performance in the last 6 months, Tesla or Microsoft?"
A question like this would check:
- Understanding this is a financial question.
- Get the stock ticker (symbol) for each one.
- Use an API to get their performance history in the last 6 months.
And, with that, we’re back to pre-LLM chatbot design: intent classification, entity extraction, business logic, return a result. Only the whole process rests on a more rickety foundation. It’s also bloated and slow, querying an LLM over and over for these things. I’m starting to see some parallels to modern JavaScript and SPAs. ;-)
It is a nightmare for me, and I do somewhat regret taking the contract even with the sizable hourly rate.
In my eyes, programming makes sense. Even if I introduce bugs, I can sooner or later track down the issue, facepalm, resolve it, and move on with my day, with some semblance of accomplishment and lessons learned.
My prompt engineering work offers no such rewards, and is a total time-suck. This is because, as another commenter wrote here, it is "throwing shit at the wall and hoping it sticks."
While you can treat it as a scientific endeavor, testing hypotheses against a black box, you will never find a prompt that works consistently, even with a low temperature, solely because these models were not built to give consistent results. There is no end-all solution, there is no "correct prompt."
Companies employing prompt engineers are looking for such consistency. Prompt engineers are therefore stuck in a system where they simply cannot succeed. They can hope and pray that the testing done by managers produces fruitful results upon every test, but the results are, for all intents and purposes, random.
In most cases prompt engineers are hired solely to achieve consistency, in order to reliably integrate the responses into other areas of their stacks which may only be able to handle certain kinds of inputs.
Let me give a real example (cant go into too much depth but this is generic enough). I was told to create a prompt that would make ChatGPT generate ten of the hottest real estate markets:
"You must always format this data in exactly the following format: ${index (as a single numeral)}: ${name of location} ${median housing price} ${YoY increase} ${link to internal system} ${number of available locations}"
You would think this would be a pretty easy task for ChatGPT/Davinci to handle, but like I said, consistency is completely missing.
- Sometimes it will say "Market ${index}" instead of just the number, even with the single numeral qualifier.
- Sometimes it will comma separate the data, even if there are explicit instructions not to do so
- Sometimes it will provide the prices/percentages/numbers as regular integers, sometimes it will format them, again, even if explicit instructions are provided
- Sometimes it will place an empty space in between the lines, even if there are explicit instructions not to do so
- Sometimes it will place random punctuation at the end, even if there are explicit instructions not to do so
It's entirely random, you could have a 4000 token prompt or a 40 token prompt, and the results will be of the same quality.
Usually, this is fine for content generation. But content formatting? Anything remotely specific? Forget about it. All of the models (I haven't been able to try ChatGPT 4 yet, however) will simply ignore things, even if (and I promise it's the last time I'll say this) there are explicit instructions not to do so.
Unfortunately this suffers from the same effect as the descriptions. I would say a solid 7 times out of 10 everything works as expected, but sometimes it seems the model goes down another path and ignores most instructions.
I would claim that using examples do improve its chances, but it is still too random to assume the edits were what improved the response, or whether it just happened to be a lucky roll of the dice.
Very not my field, but I've seen it suggested that attempting to highly constrain response format can degrade semantic quality, so one does separate report and reformatting prompts?
Perhaps ask for json, to simplify the output space?
I imagine in the near future we'll be able to prompt with a json schema, and have it enforced.
This exchange:
“I have a strong suspicion that “prompt engineering” is not going to be a big deal in the long-term & prompt engineer is not the job of the future. AI gets easier. You can already see in Midjourney how basic prompts went from complex in v3 to easy in v4. Same with ChatGPT to Bing. To the extent that prompt engineering remains a thing, it might be the Era of the Humanities Major.“
103 comments
[ 0.23 ms ] story [ 209 ms ] threadI want to be a Cannabis Prompt Engineer now https://www.xing.com/jobs/muenchen-remote-cannabis-ki-prompt...
I'm afraid these kind of job postings asking for experience longer than technology will always be there.
I have removed the post now.
They seem pretty determined to find that one candidate.
However, I think prompt engineering will evolve to be an actual technical role, akin to Data Engineering (the people who make the systems, pipelines, ETL jobs, etc for the data).
Prompt engineers will build systems that facilitate prompt generation. Meaning that prompts will be dynamically generated or at least partially generated with modifications or additions to the raw user prompt.
It's the difference to being able to write HTML vs being able to do all the backend work to dynamically generate the HTML for my Amazon homepage (including the performance benchmarks and other strategic requirements), for example.
No pressure. My email is in my profile.
1. Take user task
2. Pass it to a prompt that requests a Product UI description of a component
3. Pass 1+2 to another that asks for which npm packages to use
4. Pass 1+2+3 to a templated prompt to write the code in a constrained manner
5. Run 4 in a sandbox to see if there are errors, if so pass it back to #4, looping
It’s currently quite slow, but that’s an implementation detail I think.
I see a fresh new generation of supply chain attack, or more prompt engineering to hopefully filter out malicious packages
Although, 'script(/injection) kiddie' will be an interesting phenomenon in the future...
Already, chatGPT and bing can both give great on-topic answers to 3 word queries. and the fact that you can infinitely refine it is great
OTOH i think there is space for developing GUIs for prompts. Makes them more engaging
You’re completely ignoring the system prompts that OpenAI/Bing have already set up so your “3 word query” works as you intend. These system prompts are what prompt engineering is all about.
If I have an LLM in a video game that generates NPC dialog, I might want to feed more information to the prompt based on things my character has done in the game so the NPC dialogue is more relevant to me.
Maybe I want to inject the user’s location or the current weather in Sunnyvale, CA for the specific query. Or maybe I want to inject that the user is currently at Disneyland.
Maybe the user really likes a specific tv show and I want to let the model know that. Or maybe the specific TV show was DMCA’d by an IP owner and I need to put that in the prompt.
Do I need to detect that the user is below 13 yrs old and use a different prompt (or a different model altogether)?
Maybe I want the model to only ever respond with JSON without the user needing to specify. It needs to be clarified in the prompt with no way for the user to override it.
Models can be simplified to: input to output. Prompt engineering will be engineering the inputs.
Can you share something similar for "prompt engineering" ?
What engineering is required for Amazon to deliver me an HTML page? Are the backend engineers just “injecting” HTML? Of course not. It’s not the same as generating the HTML for a personal blog.
As far as what skills are need, it depends on how any given team wants to modify the prompt. It’s up to the ingenuity of the engineers. Maybe they need to hit 5 other models before generating the prompt. IDK. The prompt may change for any given scenario, which is why you need engineers to build the system.
https://www.promptingguide.ai/
https://lilianweng.github.io/posts/2023-03-15-prompt-enginee...
These clearly demonstrate that there's engineering skills beyond "I want you to act as a Linux terminal."
Also note that there's a difference between the prompt that you use on ChatGPT and the one that you use in the GPT API or other LLMs. In the first, you're already dealing with the prompt that openai supplied to the LLM in the first place.
Is that what you discount them for?
The reason was what I stated, which has nothing to do with the provenance or capabilities of that person.
[1]: https://en.wikipedia.org/wiki/Feature_engineering
The stuff you say it will evolve in already exists and has names everyone uses.
I believe that a critical part of software engineering is the ability to trace and resolve an issue through many different systems and code paths (what most people call debugging)
There is no way to debug these models, there is no way to systematically fix your prompts. It is therefore completely guesswork on what words in what orders might produce results closer to what your stakeholders expect.
Software engineers, when presented with an issue, can dive down as deep as they want (hell, even to the machine code layer if it comes down to it) in order to fine-tune, fix bugs, or do whatever else a stakeholder might expect.
Two very different coins in two very different universes.
I think if you spend some more time with these models, or any similar model where you have X input > Y output, you notice that you can in fact "debug" those as well, without knowing 100% of the internals. Some fixes are better than others. There are better and more reliable ways to steer these models, compared to other ways. It is not 100% guesswork, some people are better at it than others.
With a google search you are getting citations and stuck finding the truly good ones that truly match, with chatgpt you are getting an answer from someone who read all those citations and treats them all as equally good.
I think the real problem was to get the most perfect citations given your specific question. We can forgive the error since it took the web a few years to discover that best is rarely a fresh bit of blog spam by a moron who has read everything and refuses to cite sources. So GPT is convincingly as good as a human blogger yet not as useful as a less human emulating tool.
What does "smart" mean to you in this context?
When I assign a task to a (human) developer, the results depend on two things: First, how good the developer is, second, and more importantly, how well and clearly I am expressing the requirements. And this is also true for ChatGPT. With very precise requirements I get very good results.
So prompt engineering is like writing good requirements, and that also requires understanding the problem domain.
Software Engineering is kinda fake, especially in industry, but at least that's an actual discipline.
"AI monkey" is a better description
A more charitable description might be "You're employing the scientific method to extract value from GPT-like systems." Just like in science, with time you're developing intuition for how the underlying system works, but you still have to run the experiments.
Using LLMs to solve real problems is not easy. Making sure that you don't introduce regressions while making improvements is difficult, and requires building and evaluating a dataset, and the necessary pipelines. It may also include diversification of LLM providers, and creating the necessary abstractions. A fundamental understanding of how LLMs work, ability to compare different architectural approaches, along with typical data engineering and software development skills would be required.
What if you want to use the LLM for Question/Answer systems that requires working with embeddings? What if you want to find a way to process data locally without sending sensitive data to the LLM provider?
This requires real engineering skills.
Yes it is.
That is exactly why products like ChatGPT have been taking off as quick as they have.
What you're talking about is not using LLMs as a product but using them as a component within a broader system. And so of course that requires engineering skills.
It's highly unlikely that anyone taking this effort seriously is copying and pasting from ChatGPT, rather than using the API and building pipelines as part of a broader system.
Instruction-fine-tuned LLMs like ChatGPT require creating, validating, and maintaining prompts. Finding ways to use them safely is also not easy - prompt injection and hallucination are just 2 potential pitfalls - there are many more.
Denigrating this effort as "AI monkey" is myopic at best, but really just comes across as a signal that someone is terrified of being replaced by this new tech. With that attitude, they will be.
>What if you want to [do software engineering]?
Then you're a software engineer. Writing prompts isn't engineering. Building systems is engineering. Just because I use keyboards to program doesn't mean I'm a keyboard engineer, does it?
Otherwise, the job title would be "prompt writer".
Your point is what, that existing engineering titles cover this effort? Sure, you can just call all of it software engineering, but sometimes it's useful to be more specific. The LLMs are so powerful now that this new, more specific title makes sense to me, and clearly those using this new title. We'll see how it pans out over the next few years.
"Google for Dummies"
We've started calling it LLMing (llemming).
Edit: Specifying prompts is leaning towards specification. I am not saying googling is that. I'm saying that, like googling, it will just be a part of the job in a not distant future.
Given the wild popularity of posts about prompt engineering "jobs" paying +$300k, it was only a matter of time for an indie hacker to create a job board specifically for this type of job.
I have a BSc and Master's in Computer Science, 14 years of experience, I have slowly climbed the ladder, got a FAANG job 5 years ago, and only last year I managed to break 300k salary for the first time (working in ML of all things). Am I losing my sanity or these reports are greatly exaggerated?
Please share an example. I've only seen articles claiming this might exist, based one attention-seeking job ad from one company, which didn't claim "no experience or technical skills".
In that perspective, I understand why many people think it is useless. However, if you tried to make a chain of functions/calls, or worked with a tool like LangChain [0] you will see its importance.
Ex: "Which stock had better performance in the last 6 months, Tesla or Microsoft?"
A question like this would check:
- Understanding this is a financial question.
- Get the stock ticker (symbol) for each one.
- Use an API to get their performance history in the last 6 months.
- Compare.
- Return the answer.
[0] https://github.com/hwchase17/langchain
It is a nightmare for me, and I do somewhat regret taking the contract even with the sizable hourly rate.
In my eyes, programming makes sense. Even if I introduce bugs, I can sooner or later track down the issue, facepalm, resolve it, and move on with my day, with some semblance of accomplishment and lessons learned.
My prompt engineering work offers no such rewards, and is a total time-suck. This is because, as another commenter wrote here, it is "throwing shit at the wall and hoping it sticks."
While you can treat it as a scientific endeavor, testing hypotheses against a black box, you will never find a prompt that works consistently, even with a low temperature, solely because these models were not built to give consistent results. There is no end-all solution, there is no "correct prompt."
Companies employing prompt engineers are looking for such consistency. Prompt engineers are therefore stuck in a system where they simply cannot succeed. They can hope and pray that the testing done by managers produces fruitful results upon every test, but the results are, for all intents and purposes, random.
Are the companies enamored with the productivity gains of prompt-based systems sufficiently that they can ease up on the requirement for consistency?
Let me give a real example (cant go into too much depth but this is generic enough). I was told to create a prompt that would make ChatGPT generate ten of the hottest real estate markets:
"You must always format this data in exactly the following format: ${index (as a single numeral)}: ${name of location} ${median housing price} ${YoY increase} ${link to internal system} ${number of available locations}"
You would think this would be a pretty easy task for ChatGPT/Davinci to handle, but like I said, consistency is completely missing.
- Sometimes it will say "Market ${index}" instead of just the number, even with the single numeral qualifier.
- Sometimes it will comma separate the data, even if there are explicit instructions not to do so
- Sometimes it will provide the prices/percentages/numbers as regular integers, sometimes it will format them, again, even if explicit instructions are provided
- Sometimes it will place an empty space in between the lines, even if there are explicit instructions not to do so
- Sometimes it will place random punctuation at the end, even if there are explicit instructions not to do so
It's entirely random, you could have a 4000 token prompt or a 40 token prompt, and the results will be of the same quality.
Usually, this is fine for content generation. But content formatting? Anything remotely specific? Forget about it. All of the models (I haven't been able to try ChatGPT 4 yet, however) will simply ignore things, even if (and I promise it's the last time I'll say this) there are explicit instructions not to do so.
I would claim that using examples do improve its chances, but it is still too random to assume the edits were what improved the response, or whether it just happened to be a lucky roll of the dice.
Perhaps ask for json, to simplify the output space?
I imagine in the near future we'll be able to prompt with a json schema, and have it enforced.
https://twitter.com/emollick/status/1627804798224580608
I think Prompt Engineering Jobs could become popular
I don't speak french but it clearly says 20 years of experience. Talk about trolling.