Alternatively LLM’s will have gotten good enough that prompts can be general enough to not need tuned skills. I.e the average person can prompt well enough to get great results
For me, if it stays exactly the same only updated with the latest data ingested it will still be useful for me in 5 years - that is unless something even better replace it.
I can't really imagine it will be a job long term. It's more of a skill you use for some other task. Not that different from having good google search skills as a programmer.
It’s a real job for “raw” transformer models, where the model is just playing “follow the leader” to whatever kind of text you prompt it with – but depends a lot on how the model is trained and fined-tuned.
Calling it "engineering" is the idiotic part, not the actual act. It's obnoxious that 15 minute after ChatGPT was announced there were people marketing themselves for outrageous prices as "prompt engineers".
If people are willing to pay them for the value they bring, then no wonder they're marketing themselves that way. If it's effective and their goal is increased renumeration then they're smart people.
I agree, but it's actually perhaps not as bad as white goods OEMs sending out 'engineers' to fit replacement parts, or ISPs' 'engineers' connecting faceplates for a fibre installation, etc. that we've had for years.
An an electrical engineer, I often scoff at people using the term software engineer. Even though I belong in the software developer class. So you can imagine how I look at “prompt engineering”.
True engineering is dealing with unchanging and unchangeable constraints (which you find in the natural world). It’s exceedingly difficult to do without a four year degree which is usually not enough.
This other stuff you can learn in a weekend, well, I wish they would not take the word engineering.
I'm not here to defend prompt engineering I could care less what it's called but why are you people so offended by people using the term? It's inarguable that there are people that are better at getting certain results from these new models in their first shot because they know how to interact with the system better than others and understand its inner workings better.
Maybe the term engineer has been watered down from what you learned in school but in my and many other people's views its synonymous with hacker in a lot of these categories. I wouldn't call an electrical engineer an electrical hacker but if you weren't an electrical hacker you'd be a pretty garbage electrical engineer. If you've never messed around with or tested a concept or built a circuit for fun you'd probably hate your job and be worse at it. These people are doing these things with prompting and with software.
The strong feelings around the term Engineer partly date from the historical context, where inadequate engineering was more commonly and directly reflected in real human deaths. This was a huge motivator for the title of engineer to become codified and locked down to a specific set of proven and repeatable skills.
These days, most long-established engineering disciplines and similar professions are codified to the extent of having member's-only institutions and charters, and basically become "old boys clubs", including ranks and rituals.
Hence there is a divide, between these old schools of engineering and their exclusive ways, and those who use the term engineer more broadly and literally.
The old boys club mentality has many flaws, but they do have that one strong original point: that the harm one can inadvertently cause to others, under the title of engineer (or any expert), is proportional to the expertise claimed. If you claim you can design bridges, then it's incredibly important that you're right - else deaths.
Well, I’m with you, but you are not among friends on this forum. There are a lot of folks who are pretty dang happy with that “engineer” title. And they will protect it with their downvotes.
It's not engineering. Engineering is when you use a set of tools to design and create something in one attempt, i.e. without trying first many different approaches that lead nowhere, and also without being satisfied with a different outcome than you set out to obtain.
What? lol I'm not going to argue the definition of engineering with you but if you've done any real engineering you should realize it's not just some magic fluff of magically being able to do everything the first time.
Engineering takes many iterations of a concept before doing it right. You think people who engineer bridges and buildings have never thought about or attempted to design any aspect of any bridge or building before? No, they've tested a million different iterations of the same kind of thing and they've built models and they've tried things and failed.
It's why flow simulators and things like airflow chambers exist.
Yeah, some iteration may be necessary, but the point is that real engineers have a much better idea of what the outcome will be than these "prompt engineers".
You're wrong. Using LLMs and other generative AI tools well is genuinely difficult, and takes a lot of experience and skill.
Anyone can type text in a box. Understanding the full scope of what you can and cannot do with these tools - and how to best get them to do those things - turns out to be a very deep topic.
When I'm struggling to get the results I want, I've started asking ChatGPT to give me a prompt that I then feed back into it. It's not perfect, but it's made getting the right prompt way easier.
Software engineering is a type of engineering, but very few programmers actually do or even understand software engineering. Believe me, I know: I have hired and fired a lot of them.
Software engineering is. The problem is most people think they are software engineers simply because a company makes such title for a job. If that is accepted, then why not call janitors as senior principal custodial enginner.
Is it just me or are the self proclaimed "prompt engineers" on Twitter just a new iteration of crypto bro grifters? Dont get me wrong, I think LLMs and generative AI are really cool and useful and have already improved my productivity to some degree, I just get no-talent, self-promoter, vaporware vibes from people I see pushing "prompt engineering" the hardest
(Edit: I agree with the premise of article, I just wanted to rant about about AI bros)
Anything that feels "techy" but doesn't require practice, study, or math is always going to be a magnet for grifters who were doing beer bongs while others were learning to code.
(Edit: I'm not trying to imply anything about the linked author of this piece. They actually do have code examples!)
FWIW, I don't think learning to code is inherently virtuous. I do think putting in the time to be good at something is more virtuous than trying to find the easiest way to latch onto whatever the hot new industry is, which is my impression of most prompt engineering at this point.
> a magnet for grifters who were doing beer bongs while others were learning to code
My experience as a test dev is that testers, especially the ones who can't code, are into credentialism and overly-technical taxonomies that describe their field. I claim people who can't do get certified (and those who make money from the certifiables teach).
I hate to say it, but LLMs/AI are entering the "solution in search of a problem" phase. The tech is crazy powerful, no doubt, but it's going to be gradually adopted by companies and woven into products we use. It will not immediately lead to hundreds of successful billion dollar startups as "AI bros" on Twitter will have you believe.
Agreed, it's kind of really useful for a lot of things people have been using NLP to solve. In some ways the human interaction piece expand the scope but overall it doesn't mean NLP suddenly solved all problems
Surprised it took this long! In my last job in academia, people started trying to turn their ho-hum ideas into gold by attaching "deep learning" or related terms around the same time other people started doing the same thing with "blockchain." Some people had great ideas and some of them turned out to be useful. But it was painfully obvious to everybody but the person talking when they were trying to obfuscate mediocrity with fancy words.
Maybe they were there all along and people just stopped ignoring them? Maybe it's all of the hucksters whose absurd coin schemes dried up or never panned out looking for a new hook?
I agree with you, but I think you should read the article. The author makes this exact point: "Much of it is coming from individuals who are peddling around an awful lot of "Prompting" and very little "Engineering"."
Basically, the author is arguing that just fiddling around with the words you enter in a ChatGPT window is not prompt "engineering". Instead, he talks about some specific techniques for e.g. blending prompts, indicating some words deserve more attention, etc.
Yeah my comment isn't about the substance of the article (I read it and don't have any complaints) it's just a general complaint about the concepts mentioned in the title.
And yes, I know this is a classic hacker news trope, I just couldn't help myself
It's not just twitter, YouTube is also a cesspool of such stuff. Some channels are straight up reading the docs in the video with no insight provided. And the thumbnails, why are the optimising like a Mr Beast video with huge head in the thumbnail.
Siraj Rawal the infamous ai influencer is back to being an AI influenza again.
I get an almost accurate estimate of overhype by following tech influenzas now.
Yes, absolutely. There's a lot of people moving in to capitalize on the space between what people imagine they can do with LLM's, and what is currently possible. The difficulty in estimating what the LLM is likely to do with an output, the actual randomness that is involved in getting there, and the variability of the output creates a situation where you might as well put up an "INSERT GRIFT HERE" sign.
Even this post (which I think is making some pretty well informed and intentioned suggestions) exists largely because getting exactly what you want out of an LLM can be pretty difficult. Even fairly static tasks like data extraction can have aggravatingly variable outputs. I don't think that most of these are the _right_ way to get to the goal but are rather, largely clever hacks that can help a user try and nudge the LLM towards the desired latent space when adjusting the instructions fails.
I'm a dev, obsessed with ai, I'm building a startup around it now, I think of prompt engineers like spaghetti coders writing very insecure and buggy WordPress plugins and themes and calling themselves an engineer.
I've worked in PHP most of my career, but there's more engineering in an mvc than using an auto install script and installing and configuring, plugins.i guess my point is, I kinda just roll my eyes anymore.
I feel the same way with prompt engineers who don't know how to use langchain or llama index etc and aren't working on some kind of cognitive architecture to milk gpt4 for all it's worth.
tldr: I think prompt engineering is a great, legit field, but half the people in it are pretenders, although, if they can use gpt half as good as they claim, then gpt can handle most of the coding etc.. so it's probably easier to blur the line a little.
I just spent some time tweaking image generation parameters; 0.1 that way, 0.01 back again. I finally managed to eliminate the weird artifact while leaving everything else mostly the same.
Then I realized, I'm the one doing gradient descent now. I'm the one navigating a high dimensional space looking for that local optimum. I'm the prompt engineer.
Well the whole article gives me the feeling of exactly what the author detects in others calling themselves prompt engineers. "You're not prompt engineers, /I'm/ a prompt engineer" and then goes on to suggest blending prompt results and performing some basic calculations.
True "prompt engineering" is knowing how to get a desired output, reliably, from an input. How to do things like information extraction, inference, expansion, transformation, instruction, etc. But sure, we can zipper two outputs together and write tetchy gists too.
I don't understand the proposed "Prompt Weighting". The method of prompt weighting in StableDiffusion doesn't seem to have any direct analog in the context of an LLM. What exactly is being proposed here?
I don't even care about it. I just write natural language. I don't have time or the motivation to keep up with esoteric incantations and that too with a limited shelf life, that may be version or release specific.
I do think there is some value in putting the AI in the correct context role if you're using the API for some app.
89 comments
[ 4.0 ms ] story [ 150 ms ] threadContext: https://news.ycombinator.com/item?id=35640310#35642665
Buff Doge: Engineer in 1923: I can design a bridge and build it with my bare hands, fix any vehicle, and maintain an internal combustion engine.
Weak doge: Prompt engineer in 2023: halp plz i cant write prompt correctly.
Shrewd, sly, or cunning, maybe. Possibly also smart, but that's closer to the specific quality that's being displayed.
Should we call a sysadmin a "prompt engineer" too? No, because that would be stupid.
True engineering is dealing with unchanging and unchangeable constraints (which you find in the natural world). It’s exceedingly difficult to do without a four year degree which is usually not enough.
This other stuff you can learn in a weekend, well, I wish they would not take the word engineering.
Maybe the term engineer has been watered down from what you learned in school but in my and many other people's views its synonymous with hacker in a lot of these categories. I wouldn't call an electrical engineer an electrical hacker but if you weren't an electrical hacker you'd be a pretty garbage electrical engineer. If you've never messed around with or tested a concept or built a circuit for fun you'd probably hate your job and be worse at it. These people are doing these things with prompting and with software.
These days, most long-established engineering disciplines and similar professions are codified to the extent of having member's-only institutions and charters, and basically become "old boys clubs", including ranks and rituals.
Hence there is a divide, between these old schools of engineering and their exclusive ways, and those who use the term engineer more broadly and literally.
The old boys club mentality has many flaws, but they do have that one strong original point: that the harm one can inadvertently cause to others, under the title of engineer (or any expert), is proportional to the expertise claimed. If you claim you can design bridges, then it's incredibly important that you're right - else deaths.
Up I don’t hire a civil engineer for a bridge or your house that got their training from YouTube. It’s actually illegal. Engineers must be licensed.
ᐟˢ
Engineering takes many iterations of a concept before doing it right. You think people who engineer bridges and buildings have never thought about or attempted to design any aspect of any bridge or building before? No, they've tested a million different iterations of the same kind of thing and they've built models and they've tried things and failed.
It's why flow simulators and things like airflow chambers exist.
(Given how inherently unpredictable LLMs are that's actually pretty impressive.)
Anyone can type text in a box. Understanding the full scope of what you can and cannot do with these tools - and how to best get them to do those things - turns out to be a very deep topic.
I see using ChatGPT directly as prompting, not prompt engineering.
(Edit: I agree with the premise of article, I just wanted to rant about about AI bros)
(Edit: I'm not trying to imply anything about the linked author of this piece. They actually do have code examples!)
FWIW, I don't think learning to code is inherently virtuous. I do think putting in the time to be good at something is more virtuous than trying to find the easiest way to latch onto whatever the hot new industry is, which is my impression of most prompt engineering at this point.
My experience as a test dev is that testers, especially the ones who can't code, are into credentialism and overly-technical taxonomies that describe their field. I claim people who can't do get certified (and those who make money from the certifiables teach).
Maybe they were there all along and people just stopped ignoring them? Maybe it's all of the hucksters whose absurd coin schemes dried up or never panned out looking for a new hook?
"Everything is about to change. You are being left behind. Read this thread to know more"
Basically, the author is arguing that just fiddling around with the words you enter in a ChatGPT window is not prompt "engineering". Instead, he talks about some specific techniques for e.g. blending prompts, indicating some words deserve more attention, etc.
And yes, I know this is a classic hacker news trope, I just couldn't help myself
Even this post (which I think is making some pretty well informed and intentioned suggestions) exists largely because getting exactly what you want out of an LLM can be pretty difficult. Even fairly static tasks like data extraction can have aggravatingly variable outputs. I don't think that most of these are the _right_ way to get to the goal but are rather, largely clever hacks that can help a user try and nudge the LLM towards the desired latent space when adjusting the instructions fails.
I've worked in PHP most of my career, but there's more engineering in an mvc than using an auto install script and installing and configuring, plugins.i guess my point is, I kinda just roll my eyes anymore.
I feel the same way with prompt engineers who don't know how to use langchain or llama index etc and aren't working on some kind of cognitive architecture to milk gpt4 for all it's worth.
tldr: I think prompt engineering is a great, legit field, but half the people in it are pretenders, although, if they can use gpt half as good as they claim, then gpt can handle most of the coding etc.. so it's probably easier to blur the line a little.
Then I realized, I'm the one doing gradient descent now. I'm the one navigating a high dimensional space looking for that local optimum. I'm the prompt engineer.
Who else would have written it?
True "prompt engineering" is knowing how to get a desired output, reliably, from an input. How to do things like information extraction, inference, expansion, transformation, instruction, etc. But sure, we can zipper two outputs together and write tetchy gists too.
https://www.deeplearning.ai/short-courses/chatgpt-prompt-eng...
Oh… so… very mostly porn. Not the first time, not the last.
I do think there is some value in putting the AI in the correct context role if you're using the API for some app.