They are against Natural Science & the enlightenment of humanity.
Existence is wonderfully observable & learnable. It took human kind to create this particularly infernal information void, to have manufactured such a degrading & omnipresent local system of black holes.
These are woefully against all spirit & should be torn down. More positively, there's so many positive pro-human pro-Augment-Intellect things tech could be doing; we need new powerful positive symbols that establish visible legible expansion of understandability.
I agree. It's one thing to delegate labor to a machine, but I find myself very uncomfortable when the machine takes the role of creative expression. Art is a form of communication which transmits feeling, experience and emotion, but there's nothing a machine puts into this, there's nothing being transmitted, it's just a rendering, it's just pushing your buttons that make you go 'neato.'
I fear people will start taking the beautiful and profound for granted, it'll just be content.
I dont particularly mind there being some places where tech is in the lead, where it can claim the initiative.
But it just feels like humanity secedes more and more ground & understanding to humanity each month, and there's so few visible notable frontiers where humanity is staking down real wins. Rather than have advances in general purpose computing, we keep getting more and more apps, more and more far-off cloud systems.
> but there's nothing a machine puts into this, there's nothing being transmitted
That depends on your prompt. You can make it longer if you have more to say. It's a conditional language model, you can select the place it starts from and how to go. But you can't blame it for bad prompts.
Instead of seeing LLM as "nothing there, just token probabilities", I'd rather see it as a distillation of human culture. It's like a mirror house with infinite reflexions and complexities. A place to contemplate, a microscope where you can study something in detail, a simulator where you can deploy experiments.
Not too interesting to solve a "problem" that solves itself.
Just going from GPT-3.5 to GPT-4 I'd say intent inference has improved by an order of magnitude. It almost feels mind-reader-ish.
I've been waffling because I'm trying to cope with the absurdity of this product by highlighting the flaws (it can fail badly on tough problems), but we've gone from CleverBot to a genie that can instantly analyze your written intent and produce a highly logically coherent, well-worded analysis in a very short period of time.
I agree. One of the engineers on my team came to me with a script (maybe 20 lines of bash) like "hey isn't it cool that I got ChatGPT to generate this?" and after studying it for a few minutes I was like "it is cool, but now spot the bug". It was near the end of the day so I asked them to follow up with me in the morning after they'd found and fixed the bug.
I ended up having to point it out directly the next day. It was a spurious -A NUM argument that would, in a production scenario, cause it to recognize valid data as invalid because the full context of the data would be missing.
All this to say, it is a very useful tool, but we should be highly suspicious of its output, and really make sure we understand what it's giving us. The ability to read and mentally parse and execute code will become extremely important, and give people with that ability a leg up.
I won't do that...but I will try it myself. Give me a few.
EDIT: Well I'll be damned.
"However, there is a bug in the script as it only checks for the tags attribute in the first 10 lines of the resource block using the grep -A10 command. "
It takes GPT-4 perhaps 25-30 tries on the API to help me write an entire class with complex functionality, including testing, correcting and asking for new output.
Sometimes, it can one-shot the problem.
Gonna go with Carmack and say that "hand-coding" isn't going to last.
Another thing it can do is reliably debug a file, or point me in the right direction, if I explain the bug and provide the code.
It's scary as hell because I can reliably assume that it will always understand the nuance to the content I provide.
How do you interface with gpt4? Using chatgpt free mode (so gpt3.5) when i try to have it reitarate it usually changes the whole approach after a while. One example is when writing a CI file it starts with a node image but later while fixing something unrelated it switches to an alpine image.
With GPT-4, quite often, if you just ask it to review its own result for correctness, it'll spot the issue without further prompting. So you can just always ask that, although then you risk dealing with hallucinated non-bugs.
> The ability to read and mentally parse and execute code will become extremely important
Scarily, it'll probably become more scarce at the same time. It's a lot easier to parse code in your head when you're writing it every day from the ground up. Even a couple weeks away from your own code makes it harder to follow, no matter how well commented it is. Working with other people's code is always an order of magnitude harder, but there's an unspoken trust that they at least put their mind to it and tested it against the tasks it was supposed to perform.
Reading mostly-AI-generated code to spot bugs could be a specialty indeed, but it's one perfectly adapted to black hat hackers.
Trust me, if you're lucky enough to get to sharpshoot your own code and refine it every couple years, you really want as many comments as possible to get you into the headspace you were in when you reduced it the first couple times. If its logic is still readable without any comments, then it probably isn't very dense code.
I go back and look at shit I wrote ten or fifteen years ago and feel like it was written by a super intelligent alien. The again, I feel a weak version of that when I look at code I wrote last month.
my friend asked it a c++ question and it produced a smart sounding answer and syntactically valid code but utterly wrong. i think it just borrowed semantics from other languages. but it gave him the answer he was hoping for so he bought it.
Reading minds is not a hard problem to solve. The Akinator can do that reliably and it's not even a neural network. (I won't even mention the venerable millenia-old fortune telling industry.)
>>> When users cannot predict how input controls affect outputs they have to resort to trial-and-error, which is frustrating. This is a major issue when using generative AI for creating new content and it will remain an issue as long as the mapping between the input controls and outputs is unclear. But we can improve AI interfaces by enabling conversational interactions that can let users establish common ground/shared semantics with the AI, and that provide repair mechanisms when such shared semantics are missing.
And this, ladies and gentlemen, is the reasoning for working on your prompt engineering.
The prompt is the interface. Prompting is a communication skill, a management skill, and a technical skill.
I doubt many will be hired as a "prompt engineers" in coming years (some already have been, by idiotic corps) - but being a bad prompt engineer will be a noticeable lack.
Are you saying that a "good prompt" has the same properties no matter what network you're feeding it to? If so, why? It seems to me that writing a good prompt would require an understanding of the nature of the network you're using, at a minimum - for example "trending on artstation" is common in image generation prompts, and that's not going to be very useful input when generating text for a blog post.
If not, what is the specific thing people should be practicing right now?
This is really a lot less necessary with GPT-4. What required careful prompting in 3.5 often you can give it something slapdash in 4 and it can do a great job figuring out intent.
Sooner or later we'd have to be more and more precise with prompts. We'd be going from "I want a function that takes anything and returns itself" to "I : a -> a" ... and hey that looks like a programming language
> "I want a function that takes anything and returns itself"
Sounds like you're looking for a function "I : a -> I". Precision really seems important. Programming in English is like being granted wishes from a djinn. You'll get what you asked for, but beware. But currently you don't even get that.
A lot of the 'look what I made with AI' images that get shared around also don't include the creator's workflow. There's usually lots of trial-and-error, manual painting/inpainting, multiple models involved etc. and explaining all that is a lot harder than just saying 'I used stable diffusion'.
And this is why I can't get behind LLMs as a universal GUI replacement. Not in their current form.
We know that users are increasingly likely to leave a page as load time increases. Now what happens when you have an unpredictable black box that sometimes doesn't do what the user expects, or flat out refuses to cooperate? This is poor UX for many tasks.
I can see LLMs being a better interface for the command line. Where I can just say "give me the ffmpeg command to flip a video upside down"
ChatGPT gave me "ffmpeg -i input.mp4 -vf "vflip" output.mp4" and then explained what each part of the command does. I searched on google and the results were all for random tools when I specified ffmpeg, and the ones that were correct had complex walls of text, adverts everywhere, and other bloat.
My first non-ad result on Google for "ffmpeg flip video upside-down" was [0], which includes both that and several other options including rotating 180 degrees (a different way of getting upside-down).
Knowing more precisely that rotation isn't what I want, googling for "ffmpeg vertical flip video" sends me to [1] in the very first result, which is the same as what you got. Except it also specifies copying the audio without reencoding it, I'm not sure what your version would do without including that.
The first link has the entire "above the fold" content as useless bloat including ironically a huge block about SO closing the question as off topic. There is some good content in there if you dig though. The second one has a huge number of annoying popups all over the page (4 visible with adblock enabled).
The ChatGPT one just leads with the answer and explains it after. It's a non destructive command so if I later decide I also want audio, I can just continue the conversation and ask for audio to be included.
I couldn't agree more, these tools are very useful, but interaction is almost just like that with humans, you use a word, they understand it as another. So you have to correct them, to explicitly tell them what you want. They are very good tools when you they and you are talking about the same thing. Indeed just like human conversation.
The instant feedback is better than with humans. Like humans, these models can take one thing to mean another, but because we are still using them for tasks that require instant feedback, we can catch the errors early and correct. It becomes worse if the are to be used on longer lasting tasks. The entire project could be derailed by one misunderstood concept. Just like a human project.
Natural language is a fair interface. But with lots of ambiguity. When I say 'clear', do you mean understandable, easily recognisable or clear like water. I faced that dilemma when I used that word as a design prompt because it can mean things which are opposites of each other. Some languages are so rich they avoid such ambiguities.
A common language can be good. It should not be designed by a committee, it should evolve with these tools naturally.
Amazing interfaces for particular scenarios if you look at them from another angle though.
Yesterday I wanted to write a bash script to do a particular task. "
I never properly learnt bash and it takes me a very long time to do something simple, furthermore, in this case, I didn't actually know the particular tools which were needed for what I wanted to write.
After inputting, it correctly solves the exact problem for me which was perfect.
Working backwards, I then asked it to explain what each part of the script was doing -- and learned the syntax for sed etc, along with gaining a better understanding of what what each command was doing.
The point i'm trying to make here, is that it's amazing to tackle some interactions more naturally. Top down, instead of bottom up.
This can help reduce the paralysis that occurs when working with the unknown.
Someone very skilled with photoshop might understand the tool very well and work with particular tools to come up with a result. but Natural language can have you provide the desired end result and work down to fine details.
I've had the same experience (near exact same it seems, with writing bash scripts) - using it as an iterative guided tool provides a bright light to follow along an otherwise uncertain path, when you know where you want to go but not exactly how to get there, yet you've got enough experience to know when you're going off track.
Did you actually learn it though, enough so you understand it and can recall it? If you did then next time you need a bash script you'll be able to quickly figure out the basics of it, or at least Google for something and modify it with your new knlwledge. I don't think it's much of a stretch to suggest that you won't do that and you'll just use GPT again instead. And next time you might not bother asking it to explain everything...
That's not a criticism of you or of GPT. You have an awesome new tool that magically writes things for you. The most efficient way to use it is to let it do its magic and move on with more important things. All I'm suggesting is that most people, given such a tool, will use it and not learn from it.
Talking about “actually” learning something feels very “no true Scotsman”. You could replace GPT in your comment with StackOverflow, Google, or a textbook and they’d read like the same criticism, right? Referencing something until you remember it is normal learning?
Yeah, that's a fair comment. I took "learned" to mean more than maybe the OP meant it to be. I stand by my broader point though - most people will use GPT without using it to expand their knowledge.
Only if it works flawlessly. If it has subtle bugs you spend about as much time solving the task with GPT as if you didn't use GPT. And it requires deep insight into what might go wrong.
Not necessarily, akin to using some other resource like stack overflow, I now know what the command was attempting to achieve and can debug it from there. ChatGPT is actually fantastic for breaking down outputs and explaining what it was trying to do at each part, which can help significantly for getting you to the final result you want
Yeah good point, I guess I meant I now know that “sed” is the command I want to use for what I wanted to do, and now a little more experienced with knowing when and how I can use it
I feel like this is valid criticism for Dall-e, but not so much for stable diffusion.
Stable diffusion (and to a lesser extent midjourney) exposes a LOT more knobs to tune. Layer on controlnet and other extensions, and you can really iteratively refine an image to get closer and closer to what you want.
The key is to be able to 'lock in' a depth map, rough composition, seed, etc, and then tune the other parameters to refine what you're looking for.
It's definitely not a one-shot solution, but its still miles faster than learning how to do digital painting and manually rendering things. I don't feel like it matters if you have to generate 16 images to find one to start building off of when each image can be rendered in 1/1000th the time a human could.
What i find interesting are the discussions around good vs bad prompts and being adept at “prompt engineering”
We’ve had this in the engineering world a long time in the form of Google and StackOverflow searches.
The great engineers i’ve worked with know most stuff they need day-to-day, but when they run into issues it’s their ability to search for and find the right answer that gives them advantage over others who don’t know where to look (or even start).
Another skill that'll be pretty valuable in this era is one many of the best technical leaders possess: the ability to coach desirable outcomes from squishy autonomous black boxes over time.
Getting excellent, exact performance out of deterministic systems is an impressive feat, but autonomy means variability, and getting excellent performance out of variable systems (especially illegible ones) is a different game.
I agree 100%. I first had this thought last night when reading “Eight Things to Know about LLMs” (select quote from section 8 below)[0]
> Simply prompting a model to “think step by step” can lead it to perform well on entire categories of math and reasoning problems that it would otherwise fail on (Kojima et al., 2022). Similarly, even observing that an LLM consistently fails at some task is far from sufficient evidence that no other LLM can do that task (Bowman, 2022).
The eerie thing to me is that this is coaching. I have never coached anything that isn’t alive by any definition. My feelings about this realization are ambivalent.
Doesn't prompting "think step by step" simply increase the probability the model will generate its output in the style of places where it saw "step by step" before (eg: tutorial content)? It's not really "deciding" to think step by step. Maybe I am mistaken.
I used to think that LLMs for coding would mostly be a tool to make experienced developers much more productive, but after coming across this video I think it will be used by normal people with no programming experience to make small basic programs.
This is so much harder than most people think. I would reckon that as soon as a complication shows up, 75% of non coders will get helplessly stuck.
Complication meaning something slightly beyond changing variable names: you need a different if condition, the snippet doesn't contain the required import, the version has advanced slightly.
Most people will have no idea where to even paste the code. The key take away from the video I linked is that an AI can tell people not only what code to copy, but where to copy it and what to change if it doesn't work.
> where to copy it and what to change if it doesn't work
Google can do that too. (Well, it could if it wasn't permanently broken from SEO spam and monetization efforts.)
Which is the key takeaway: the "chat" part of ChatGPT is a stupid smoke and mirrors gimmick. The salient useful parts are the large and well-curated training database.
Which is technically quite possible to do without "AI" or text generators; though possibly not realistic from a business sense. There's no business case for "information search without spam and ads".
Indeed, ChatGPT really excels at being confidently incorrect. Ask it anything non-trivial and chances are that it has written code which is subtly wrong, and sometimes very subtly.
it got me today. produced correct output for good input but did not consider a very big edge case that i didn't initially think about.
if this is how we're going to write code then there's going to be even more broken edge cases than usual.
I’ve never seen code look so correct, sort of work but be so strangely crap.
This is my experience of GPT generated code too. It needs refactoring as soon as it's generated for me to be happy with it.
The real question is whether that actually matters though. If an AI tool is writing ugly code that works, and the 'developer' is only interacting with that code through prompts, then what the code looks like is irrevelant. No one will see it. It'd be like worrying about what the inside of a Photoshop file or a Word doc looks like.
I think we're some way off using AI to write and modify code like it's an opaque file, but once we're there it won't matter if that code is ugly and terrible. No one will look.
Yes, I can easily see a future where many abstractions we currently use in code will go away, because we'll just use LLMs to change the code when needed, instead of trying to future proof code.
Sounds like an assembly programmer trying C for the first time. Sure the produced assembly isn't as good as hand written. But with Moore's law, maybe it doesn't matter?
This makes me wonder: will we start to see the reverse like we have in search engines? Will people start finding ways to do SEO for language models to try to get them to output stuff about their products more often?
They'll certainly try, but I am somewhat optimistic here, as you can feed the language model with some ground truths that allows it to detect the marketing nonsense.
Another nice thing with language models is that they condense the information, it doesn't matter how many webpages there are for a product, if you ask ChatGPT about it, it will list it exactly once. It knows it is the same product each time and the list it produces will be created specifically for your query. Meanwhile Google gets cluttered with duplicate information, since your query is implicitly about webpages mentioning a product, not products themselves.
That said, there is still plenty of work to do here. BingChat is completely unusable when it comes to product search, much worse than Google, as it will just pick the first three search results, which tend to be SEO spam, and summarize their content. ChatGPT is much better here, but its lack of direct Web access also handicaps it rather heavily.
Exactly, though an addendum is not just their ability to search when the need arises, but also to know WHEN to resort to search, and with what angle. This is not just isolated to coding though, experienced the same in research as well. So much hinges on your ability to know when to search for literature and how.
But LLM black boxes are pretty great at surfacing search leads. They might hallucinate, but they are much more precise semantically than search. LLMs understand when you say something in your own words.
In fact one trick is to go to LLM to get the "closed-book" answer, and then use that answer to formulate a proper search query for Google.
Q: What is the height of Everest.
A: The height of Everest is 8800m
Search: "The height of Everest is 8800m"
Result: 8849m
Using the generated answer as search phrase, even if it is factually wrong, works because it has the right structure and good keywords.
I always felt my best asset as an engineer was my googling skills. I always also felt like this is almost cheating, but of course in a way (in most ways) it's not. I often solve problems for
colleagues by returning a google response from the first page of search results. Meaning, most likely, that I had a better google query than they had.
I wonder if this skill can become more useful in the age of "prompt engineering"? Will my googling skills be useless and I need to start over? Or will my ability to google naturally convert into a decent prompt-engineering skill given a reasonable amount of practice?
>Meaning, most likely, that I had a better google query than they had.
there's two parts, good google query, but also figuring out which of the results returned is the most likely to be useful.
I've often found results for people that were on the first page of the most obvious query in the world to start with, but they were not the first result but down a bit and maybe looked a bit weird.
I used to think this, but I'm not sure google-fu is the real superpower here. I've been surprised to encounter many developers who just don't seem to be able to read and comprehend the output of a program; logs, stack traces, etc.
That's the basic — sometimes only — input to a search query. I suspect the ability to comprehend and describe a problem is a skill that translates quite smoothly to a world of prompt-based interaction.
> Even worse, it is unclear to me how to change the prompt to move the image towards the image I want.
An image program I use has sliders for things called "High", "Low
", and "Gamma". I don't know what these mean, but I can immediately see the effects they have when I use them. I can also reverse the effects back to the original image by putting them back.
I don't know if AI could easily reverse back changes, at least over a couple of iterations. But I wonder if things like sliders or knobs could be added to AI interfaces to allow more gradual iteration of a text or image, or idea of a text or image, than having the AI generate an iteration based on a new plain text prompt, or even "conversational interface".
You could have a ton of sliders with names that the user doesn't understand. You could link sliders together so that moving one moves them all, and then unlink individual ones as desired to see what they do. This would provide more immediate and tailorable feedback than trying to get your words just right.
A while ago someone posted https://www.calligrapher.ai/ (discussion: https://news.ycombinator.com/item?id=34530011)
The sliders were interesting and generally useful but I found it frustrating that they were not repeatable. Since then I've started to think I was approaching the sliders the wrong way by assuming they directly controlled the output, rather than being weights in one direction or another.
there are sliders for things like "cfg" and "denoising" and various other things.
with some trial and error you can figure out what they do and how to tune them, the issue is that image generation still takes 10 or 30 seconds so it's not a quick process.
but yes, I'd still welcome more dials. more is a bigger pain to get started but useful after you figure them out
Maybe ways to freeze what you like and iterate the rest? Not just parts of an image, or particular phrases in a text, but stylistic elements such as shadows or shapes or colors or tenses and word usage.
The recommendation to support conversational interactions is very good. After all, if you consult with a human expert, one of the main things they'll be doing is conversing with you to figure out your requirements. I find it a bit frustrating that even ChatGPT will give you a giant answer straightaway instead of clarifying your requirements first. (Prompting can help with this somewhat, if you explicitly tell it to ask clarifying questions.)
We are finding that using context hints injected dynamically into ChatGPT prompts by way of separate classifiers is potentially the ultimate solution for many business needs. Having the very first thing be an explicit natural language feature parsing phase can make the downstream generative stuff way more predictable. We could actually go back and look at the traces around a user's original prompt to determine why it might have generated nonsense downstream.
The challenging part of this is inverting everything and figuring out what all of your classes are and how to arrange training data to fit them. Set theory is another new dragon here. Smaller models like Ada and Babbage are ideal for this, but we are looking at 90's tech too (SVMs). If we need 10k separate models because we have so many classes, then I'd prefer to be able to train them in a few seconds/minutes on a CPU.
Fine tuning gigantic models like Davinci is a dead-end at this point IMO. A fleet of microscopic models working together to feed 1 big powerful (untunable) model feels a lot more manageable.
You are right. Apologies. I corrected it. I am still spinning on some results.
To answer the hypothetical - I do think handcrafted solutions by professionals are still the best option in a lot of places. Less weird edges and things to worry about. But, a lot of hard things are quickly reduced to routine crap once you have effectively 'done it' a few times.
Prompt engineering is a band-aid fix for the real underlying problem (one we can't solve) - all human languages are inherently vague and prone to a variety of different interpretations.
i’m curious to know more about how the model would know what is blade runner outside of stills found online. is it able to watch and analyze YouTube videos or ““ watch movies and transcribe the text and also pull out stylistic elements from the cinematic work?
I am a strange guy — I like to understand things. That translates to me often using microframeworks where I can build most stuff myself. I don't like to use fully featured solutions that already do 75% of what you want but where you have to understand everything and read the whole source code and sell your first born to get the remaining 25%.
So I'd rather buy wood and nails to build the hut I like than to buy some hut and hammer it into the hut I wanted.
The first is a really fulfilling task, the last annoying. And so I rather drive my car to my destination than let it drive itself and have to control if it doesn't make a mistake.
Collaborating with humans may be time-consuming but at least we can establish common ground and fix misunderstandings through conversation. But I feel ChatGPT is a step in the right direction, where you can question or drive the conversation. I dont think we are far from common sense and symbolic reasoning to these models. I bet AI researchers will bridge this gap soon
I was about to say. The chat paradigm solves the issue. I nearly always get what I need after rephrasing or adding more context, which is exactly how it works with humans.
Every time I use Siri, it reminds me of this problem.
And it’s true that humans are not necessarily better.
But that’s not entirely true. My closest friends know me. If I say “put up some turnstile,” they know I’m probably talking about the band because I like the band.
Taking to these AIs and voice assistants is like talking to an anonymous person I’ve just met for the first time, every time, so you always need extra qualifiers.
116 comments
[ 3.1 ms ] story [ 166 ms ] threadExistence is wonderfully observable & learnable. It took human kind to create this particularly infernal information void, to have manufactured such a degrading & omnipresent local system of black holes.
These are woefully against all spirit & should be torn down. More positively, there's so many positive pro-human pro-Augment-Intellect things tech could be doing; we need new powerful positive symbols that establish visible legible expansion of understandability.
But can it do it without baby steps?
I fear people will start taking the beautiful and profound for granted, it'll just be content.
But it just feels like humanity secedes more and more ground & understanding to humanity each month, and there's so few visible notable frontiers where humanity is staking down real wins. Rather than have advances in general purpose computing, we keep getting more and more apps, more and more far-off cloud systems.
> but there's nothing a machine puts into this
What makes you so sure?
What makes you think a machine has any wants and desires?
That depends on your prompt. You can make it longer if you have more to say. It's a conditional language model, you can select the place it starts from and how to go. But you can't blame it for bad prompts.
Instead of seeing LLM as "nothing there, just token probabilities", I'd rather see it as a distillation of human culture. It's like a mirror house with infinite reflexions and complexities. A place to contemplate, a microscope where you can study something in detail, a simulator where you can deploy experiments.
Just going from GPT-3.5 to GPT-4 I'd say intent inference has improved by an order of magnitude. It almost feels mind-reader-ish.
I've been waffling because I'm trying to cope with the absurdity of this product by highlighting the flaws (it can fail badly on tough problems), but we've gone from CleverBot to a genie that can instantly analyze your written intent and produce a highly logically coherent, well-worded analysis in a very short period of time.
I have a similar experience but when I actually look at what’s been returned I find strange things in there.
Like it often works, but it’s rarely as correct as it seems. At least for a given use case.
I ended up having to point it out directly the next day. It was a spurious -A NUM argument that would, in a production scenario, cause it to recognize valid data as invalid because the full context of the data would be missing.
All this to say, it is a very useful tool, but we should be highly suspicious of its output, and really make sure we understand what it's giving us. The ability to read and mentally parse and execute code will become extremely important, and give people with that ability a leg up.
EDIT: Well I'll be damned.
"However, there is a bug in the script as it only checks for the tags attribute in the first 10 lines of the resource block using the grep -A10 command. "
Sometimes, it can one-shot the problem.
Gonna go with Carmack and say that "hand-coding" isn't going to last.
Another thing it can do is reliably debug a file, or point me in the right direction, if I explain the bug and provide the code.
It's scary as hell because I can reliably assume that it will always understand the nuance to the content I provide.
Scarily, it'll probably become more scarce at the same time. It's a lot easier to parse code in your head when you're writing it every day from the ground up. Even a couple weeks away from your own code makes it harder to follow, no matter how well commented it is. Working with other people's code is always an order of magnitude harder, but there's an unspoken trust that they at least put their mind to it and tested it against the tasks it was supposed to perform.
Reading mostly-AI-generated code to spot bugs could be a specialty indeed, but it's one perfectly adapted to black hat hackers.
: No.
//damn you, explain why I wrote this code.
: Who knows, meat dummy?
Signed, yourself from the future.
I go back and look at shit I wrote ten or fifteen years ago and feel like it was written by a super intelligent alien. The again, I feel a weak version of that when I look at code I wrote last month.
So yeah, commenting
Extracting intent from one piece of text when a number of interpretations exist that I cannot count is not.
And this, ladies and gentlemen, is the reasoning for working on your prompt engineering.
The prompt is the interface. Prompting is a communication skill, a management skill, and a technical skill.
I doubt many will be hired as a "prompt engineers" in coming years (some already have been, by idiotic corps) - but being a bad prompt engineer will be a noticeable lack.
Start practicing.
If not, what is the specific thing people should be practicing right now?
Sounds like you're looking for a function "I : a -> I". Precision really seems important. Programming in English is like being granted wishes from a djinn. You'll get what you asked for, but beware. But currently you don't even get that.
If it's useful now it's not idiotic for a company to hire for people good at solving their problems.
It's not :) - they're hype-hiring.
There’s so much “hey look at this from AI” and I look at what I get and it’s… not good/ frustrating.
We know that users are increasingly likely to leave a page as load time increases. Now what happens when you have an unpredictable black box that sometimes doesn't do what the user expects, or flat out refuses to cooperate? This is poor UX for many tasks.
ChatGPT gave me "ffmpeg -i input.mp4 -vf "vflip" output.mp4" and then explained what each part of the command does. I searched on google and the results were all for random tools when I specified ffmpeg, and the ones that were correct had complex walls of text, adverts everywhere, and other bloat.
Knowing more precisely that rotation isn't what I want, googling for "ffmpeg vertical flip video" sends me to [1] in the very first result, which is the same as what you got. Except it also specifies copying the audio without reencoding it, I'm not sure what your version would do without including that.
[0] https://stackoverflow.com/questions/3937387/rotating-videos-...
[1] https://filme.imyfone.com/video-edit-tutorials/ffmpeg-flip-v...
The ChatGPT one just leads with the answer and explains it after. It's a non destructive command so if I later decide I also want audio, I can just continue the conversation and ask for audio to be included.
The instant feedback is better than with humans. Like humans, these models can take one thing to mean another, but because we are still using them for tasks that require instant feedback, we can catch the errors early and correct. It becomes worse if the are to be used on longer lasting tasks. The entire project could be derailed by one misunderstood concept. Just like a human project.
Natural language is a fair interface. But with lots of ambiguity. When I say 'clear', do you mean understandable, easily recognisable or clear like water. I faced that dilemma when I used that word as a design prompt because it can mean things which are opposites of each other. Some languages are so rich they avoid such ambiguities.
A common language can be good. It should not be designed by a committee, it should evolve with these tools naturally.
Yesterday I wanted to write a bash script to do a particular task. "
I never properly learnt bash and it takes me a very long time to do something simple, furthermore, in this case, I didn't actually know the particular tools which were needed for what I wanted to write.
After inputting, it correctly solves the exact problem for me which was perfect.
Working backwards, I then asked it to explain what each part of the script was doing -- and learned the syntax for sed etc, along with gaining a better understanding of what what each command was doing.
The point i'm trying to make here, is that it's amazing to tackle some interactions more naturally. Top down, instead of bottom up.
This can help reduce the paralysis that occurs when working with the unknown. Someone very skilled with photoshop might understand the tool very well and work with particular tools to come up with a result. but Natural language can have you provide the desired end result and work down to fine details.
Did you actually learn it though, enough so you understand it and can recall it? If you did then next time you need a bash script you'll be able to quickly figure out the basics of it, or at least Google for something and modify it with your new knlwledge. I don't think it's much of a stretch to suggest that you won't do that and you'll just use GPT again instead. And next time you might not bother asking it to explain everything...
That's not a criticism of you or of GPT. You have an awesome new tool that magically writes things for you. The most efficient way to use it is to let it do its magic and move on with more important things. All I'm suggesting is that most people, given such a tool, will use it and not learn from it.
If their GPT generated code doesn't work they'll use GPT to refine it until it does. People will learn GPT rather than bash.
Stable diffusion (and to a lesser extent midjourney) exposes a LOT more knobs to tune. Layer on controlnet and other extensions, and you can really iteratively refine an image to get closer and closer to what you want.
The key is to be able to 'lock in' a depth map, rough composition, seed, etc, and then tune the other parameters to refine what you're looking for.
It's definitely not a one-shot solution, but its still miles faster than learning how to do digital painting and manually rendering things. I don't feel like it matters if you have to generate 16 images to find one to start building off of when each image can be rendered in 1/1000th the time a human could.
We’ve had this in the engineering world a long time in the form of Google and StackOverflow searches.
The great engineers i’ve worked with know most stuff they need day-to-day, but when they run into issues it’s their ability to search for and find the right answer that gives them advantage over others who don’t know where to look (or even start).
Getting excellent, exact performance out of deterministic systems is an impressive feat, but autonomy means variability, and getting excellent performance out of variable systems (especially illegible ones) is a different game.
> Simply prompting a model to “think step by step” can lead it to perform well on entire categories of math and reasoning problems that it would otherwise fail on (Kojima et al., 2022). Similarly, even observing that an LLM consistently fails at some task is far from sufficient evidence that no other LLM can do that task (Bowman, 2022).
The eerie thing to me is that this is coaching. I have never coached anything that isn’t alive by any definition. My feelings about this realization are ambivalent.
[0]: https://cims.nyu.edu/~sbowman/eightthings.pdf
I’ve never seen code look so correct, sort of work but be so strangely crap.
https://www.youtube.com/watch?v=IyKKhxYJ4U4
Complication meaning something slightly beyond changing variable names: you need a different if condition, the snippet doesn't contain the required import, the version has advanced slightly.
Google can do that too. (Well, it could if it wasn't permanently broken from SEO spam and monetization efforts.)
Which is the key takeaway: the "chat" part of ChatGPT is a stupid smoke and mirrors gimmick. The salient useful parts are the large and well-curated training database.
Which is technically quite possible to do without "AI" or text generators; though possibly not realistic from a business sense. There's no business case for "information search without spam and ads".
This is my experience of GPT generated code too. It needs refactoring as soon as it's generated for me to be happy with it.
The real question is whether that actually matters though. If an AI tool is writing ugly code that works, and the 'developer' is only interacting with that code through prompts, then what the code looks like is irrevelant. No one will see it. It'd be like worrying about what the inside of a Photoshop file or a Word doc looks like.
I think we're some way off using AI to write and modify code like it's an opaque file, but once we're there it won't matter if that code is ugly and terrible. No one will look.
https://arxiv.org/abs/2302.10149
Another nice thing with language models is that they condense the information, it doesn't matter how many webpages there are for a product, if you ask ChatGPT about it, it will list it exactly once. It knows it is the same product each time and the list it produces will be created specifically for your query. Meanwhile Google gets cluttered with duplicate information, since your query is implicitly about webpages mentioning a product, not products themselves.
That said, there is still plenty of work to do here. BingChat is completely unusable when it comes to product search, much worse than Google, as it will just pick the first three search results, which tend to be SEO spam, and summarize their content. ChatGPT is much better here, but its lack of direct Web access also handicaps it rather heavily.
In fact one trick is to go to LLM to get the "closed-book" answer, and then use that answer to formulate a proper search query for Google.
Q: What is the height of Everest.
A: The height of Everest is 8800m
Search: "The height of Everest is 8800m"
Result: 8849m
Using the generated answer as search phrase, even if it is factually wrong, works because it has the right structure and good keywords.
I wonder if this skill can become more useful in the age of "prompt engineering"? Will my googling skills be useless and I need to start over? Or will my ability to google naturally convert into a decent prompt-engineering skill given a reasonable amount of practice?
there's two parts, good google query, but also figuring out which of the results returned is the most likely to be useful.
I've often found results for people that were on the first page of the most obvious query in the world to start with, but they were not the first result but down a bit and maybe looked a bit weird.
That's the basic — sometimes only — input to a search query. I suspect the ability to comprehend and describe a problem is a skill that translates quite smoothly to a world of prompt-based interaction.
An image program I use has sliders for things called "High", "Low ", and "Gamma". I don't know what these mean, but I can immediately see the effects they have when I use them. I can also reverse the effects back to the original image by putting them back.
I don't know if AI could easily reverse back changes, at least over a couple of iterations. But I wonder if things like sliders or knobs could be added to AI interfaces to allow more gradual iteration of a text or image, or idea of a text or image, than having the AI generate an iteration based on a new plain text prompt, or even "conversational interface".
You could have a ton of sliders with names that the user doesn't understand. You could link sliders together so that moving one moves them all, and then unlink individual ones as desired to see what they do. This would provide more immediate and tailorable feedback than trying to get your words just right.
but yes, I'd still welcome more dials. more is a bigger pain to get started but useful after you figure them out
Also reminds me of Monty Python's Stay Here and Make Sure He Doesn't Leave https://youtube.com/watch?v=2f5MvVx8RM8
The challenging part of this is inverting everything and figuring out what all of your classes are and how to arrange training data to fit them. Set theory is another new dragon here. Smaller models like Ada and Babbage are ideal for this, but we are looking at 90's tech too (SVMs). If we need 10k separate models because we have so many classes, then I'd prefer to be able to train them in a few seconds/minutes on a CPU.
Fine tuning gigantic models like Davinci is a dead-end at this point IMO. A fleet of microscopic models working together to feed 1 big powerful (untunable) model feels a lot more manageable.
To answer the hypothetical - I do think handcrafted solutions by professionals are still the best option in a lot of places. Less weird edges and things to worry about. But, a lot of hard things are quickly reduced to routine crap once you have effectively 'done it' a few times.
Like, you know, managing humans?
So I'd rather buy wood and nails to build the hut I like than to buy some hut and hammer it into the hut I wanted.
The first is a really fulfilling task, the last annoying. And so I rather drive my car to my destination than let it drive itself and have to control if it doesn't make a mistake.
And it’s true that humans are not necessarily better.
But that’s not entirely true. My closest friends know me. If I say “put up some turnstile,” they know I’m probably talking about the band because I like the band.
Taking to these AIs and voice assistants is like talking to an anonymous person I’ve just met for the first time, every time, so you always need extra qualifiers.