> Software engineers should be critical of the outputs of large language models, as they tend to hallucinate and produce inaccurate or incorrect code.
This seems to be the case more than not for certain tasks, anything assembler or C I have ever asked it has turned out to be at least somewhat wrong. Mixing styles and syntax all over the place. I am not afraid that some generative AI will take my job anytime soon.
These things are a matter of writing a correct prompt. Like "use this and that naming convention for variables and keep this and that style".
You can also ask it to write tests for the code so you can verify it is working or add your own additional tests.
Even if the produced code is wrong, it usually takes a few steps to correct it and still saves time.
It is an amazing tool and time saver if you know what you are doing, but helps with research as well. For instance if you want to code something in the domain you know little about, it can give you ideas where to look and then improve your prompts based on that.
In the context of taking anyone's job is like saying that a spreadsheet is going to replace accountants.
My first step when using an LLM is asking it to produce a test suite for a function, with a load of example inputs and outputs. 9 times out of 10, I've been presented with something incorrect which I need to correct first.
I'm reasonably good at being specific and clear in my directions, but I quickly arrived at the conclusion that LLMs are simply not good at producing accurate code in a way that saves me time.
> These things are a matter of writing a correct prompt.
No, they aren't.
ChatGPT doesn't know things. It's just a very fancy predictive text engine. For any given prompt, it will provide a response that is engineered to sound authoritative, regardless of whether any information is correct.
It will summon case law out of the aether when prompted by a lawyer; it will conjure paper titles and author names from thin air when prompted by a researcher; it will certainly generate semantically meaningless code very often. It's absolutely ludicrous to assert that you just need a "better prompt" to counteract these kinds of responses because this is not a bug — it's literally just how it works.
Read the next sentence after your quote. The point is that you should include code and examples in your prompt (Copilot is so good since it includes the surrounding code and open files in the prompt to understand your specific context), not that you should craft an exceptional "act as rockstar engineer" prompt.
I did read it, but the whole premise is flawed due to an apparently incomplete understanding of how LLMs work. Including code samples in your prompt won't have the effect you think it will.
LLMs are trained to produce results that are statistically likely to be syntactically well-formed according to assumptions made about how "language" works. So when you provide code samples, the model incorporates those into the response. But it doesn't have any actually comprehension of what's going on in those code samples, or what any code "means"; it's all just pushing syntax around. So what happens is you end up with responses that are more likely to look like what you want, but there's no guarantee or even necessarily a correlation that the tuned responses will actually produce meaningfully good code. This increases the odds of a bug slipping by because, at a glance, it looked correct.
Until LLMs can generate code with proofs of semantic meaning, I don't think it's a good idea to trust them. You're welcome to do as you please, of course, but I would never use them for anything I work on.
If it works it works and it definitely works for me. I've been using Copilot for about a year and I can't imagine coding without it again. I cannot recall any bugs slipping by because of it. If anything it makes me write less bugs, since it has no problem taking tedious edge cases into account.
> I've been using Copilot for about a year and I can't imagine coding without it again
I for example used Copilot for 2 months at work and wouldn't pay for it. Most suggestions where either useless or buggy. But I work in a huge C++ codebase, maybe that's hard for it as C++ is also hard for ChatGPT.
I think this is incorrect for most use-cases. LLMs do grok code semantically. Adding requests for coding style injects implementation specificity when flattening the semantic multidimensionality back into language.
No, they do not. That's not how LLMs work, and stating that it is betrays an absolute lack of understanding of the underlying mechanisms.
LLMs generate statistically likely sequences of tokens. Their statistical model is derived from huge corpora, such as the contents of the entire (easily searchable) internet, more or less. This makes it statistically likely that, given a common query, they will produce a common response. In the realm of code, this makes it likely the response will be semantically meaningful.
But the statistical model doesn't know what the code means. It can't. (And trying to use large buzzwords to convince people otherwise doesn't prove anything, for what it's worth.)
To see what I mean, just ask ChatGPT about a slightly niche area. I work in programming languages research at a university, and I can't tell you how many times I've had to address student confusion because an LLM generated authoritative-sounding semantic garbage about my domain areas. It's not just that it was wrong, but that it just makes things up in every facet of the exercise to a degree that a human simply couldn't. They don't understand things; they generate text from statistical models, and nothing more.
The question is, do you actually save time by coaxing the language model into an answer or would you just save time by writing it yourself?
I have this friend who gets obsessed with things very easily and ChatGPT got to him quite a bit. He spent about two months perfecting his AI persona and starts every chat with several hundred words of directions before asking any questions. I find that this also produces the wrong answers many times.
For me it saves time by keeping my momentum up. I don’t lean on it when I’m in a flow state and cruising through the code I’m writing but as soon as I hit a wall I jump over to chat and start working on a solution with it. This saves a huge amount of time that would otherwise be spent banging my head against the keyboard or googling or reading random SO posts and dev blogs and documentation or even just the time wasted when I get frustrated enough to stop working on the problem and wind up browsing hn.
(I think just about anyone 'serious' learns pretty quickly that compiler errors are in the 'our Lord and Saviour' category. Unusually distributed generally quite rare but easily catastrophic runtime 'Heisenbugs' are the fruit of the devil!)
> anything assembler or C I have ever asked it has turned out to be at least somewhat wrong
i've found that its quality is proportional to the amount of questions about the subject on the internet. if i ask it for help with popular javascript frameworks, it vastly improves my productivity (i'm not a frontend person). it still coughs up wrong stuff half the time, but even then it can cut through the constant churn of the frameworks' terminology and give me enough hint to find what i need in the docs quickly.
if i ask it about, say, specific details of the STM32 HAL, it knows just enough to come up with something that i'll waste my time reading.
How to survive in an AI world? Shift your mindset from “I write code” to “I deliver value”.
Now instead of AI replacing you, it’s helping you get more done (in theory). Everyone wins.
Staking your career on being a pair of hired hands that executes somebody else’s exacting specifications was always a long-term losing proposition. And we are very very far away from AI being able to ask the right questions to help business stakeholders and customers clearly express what they need.
I also find it bizarre that so many people feel precious about the code. They have too much ego attached to what they type in and the AI kind of make them feel insecure.
I couldn't care less if I write the code, the AI or someone I told to write it.
> I also find it bizarre that so many people feel precious about the code.
Would you find it bizarre that a joiner feels precious about not just the cabinet they made (value), but how they made it? The joints they used, the process they went through, the wood (i.e., the code)?
A plumber, electrician, architect, designer, programmer -- we take pride in our skills.
Exactly, ultimately we are craftsmen, not artisans. They are two very distinct things. The difference being that the value of our output is directly tied to its' functional utility, not any sense of aesthetic or artistic expression. You can take pride in the means used to achieve an end, but they ultimately must be superseded by more efficient techniques or you just become an artisan using traditional tools, and not a craftsman who uses the industry standard.
> Exactly, ultimately we are craftsmen, not artisans. They are two very distinct things. The difference being that the value of our output is directly tied to its' functional utility, not any sense of aesthetic or artistic expression.
That's the usual definition of an artisan as opposed to an artist. (Artisan vs. craftsman is a fuzzier distinction.)
An artisan uses artistic techniques and their own intuitive judgement to produce a practical good. Think bakers. No one would be upset that their local bakery was using tools and techniques from a thousand years ago to make their bread. But a carpenter, for example, is a craftsman. He may take an aesthetic pride in the finished result of his work, but it must match all technical specifications and building codes. And the customer would be pretty upset to see them using wood planes and hand saws to frame their house.
Craftsmen are individual contributors. When you coordinate others, you're no longer an IC, you're a foreman, a boss. Coding with LLMs is about managing the contributions of other ICs. It's no different from coordinating low-level human coders who simply implement the design that was given them.
If that's the kind of 'craftsmanship' you enjoy, great. To me, this new model of 'bionic coding' feels a lot like factory work, where my job is to keep my team from falling behind the assembly line.
BTW, I've worked factory lines as both IC and foreman. In either role, that life sucks.
Your analogy suggests that the code is the final product, akin to a cabinet or a building, something tangible that can be appreciated for its craftsmanship. In some instances, like open-source software, the code might indeed be viewed this way, but in most cases, it's not the code itself that end-users appreciate, it's the functionality it provides.
To refine your analogy, the code isn't the cabinet - it's more like the blueprint or the process used to create the cabinet. The user doesn't care if a hand saw or a power saw was used, as long as the cabinet is well-crafted and functional. Similarly, end-users of software don't see or appreciate the code. They only interact with the user interface and the functionality it provides. As a result, being "precious" about the code can sometimes be more about personal ego and less about delivering value to the end-user.
In terms of pride in craftsmanship, of course, it's crucial to take pride in one's work. However, this doesn't mean that one should be resistant to using better tools when they become available. The introduction of AI in coding doesn't negate craftsmanship - instead, it's an opportunity to refine it and make it more efficient. It's like a carpenter transitioning from using manual tools to using power tools. The carpenter still needs knowledge, skill, and an eye for detail to create a good product, but now they can do it more efficiently.
This is perhaps true for shrink-wrapped software (in so far as that still exists), but for B2B SaaS products, the ability to easily maintain and enhance the codebase is vital to the long-term success of the product.
Maybe it won't actually matter, because if AI generates a 5MM line ball-of-mud, it will be able to easily add features later due to the code being styled in alignment with its training, or maybe the context size limitations will allow future systems to digest the entire thing. It could end up being like coding in a very high-level language: who cares what crazy bytecode is kicked out as long as it performs within expectations.
I feel like there’s probably an unspoken division between people who enjoy building systems (providing value in your terms) and people who enjoy more of the detailed coding. The later group had a good thing going where they were extremely well compensated for doing an activity they enjoyed, so I think it make sense for them to be a bit distressed.
When assemblers were introduced, there were programmers who complained, because it ruined the intimacy of their communication with the computer that they had with octal.
They meant it, too. Noticing that the "or" instruction differed only by one bit from the "subtract" instruction told you something about the probable inner workings of the CPU. It just turned out that it didn't matter - knowing that level of detail didn't help you write code nearly as much as it helped to be able to say "OR" instead of "032".
> AI being able to ask the right questions to help business stakeholders and customers clearly express what they need
This is where the biggest impact will be: better requirements. I see no effect on writing code because humans already know how to write code just fine, but so many of the people running the business are absolutely clueless as to what it needs.
I think this is hugely mistaken, just like a few months ago when people assumed that "A.I. can't ever draw faces/hands" while doing so only required a couple updates.
You're seemingly claiming that:
"Taking an incomplete customer brief and asking relevant questions to make it clearer and complete is the largest value in programming"
And... you're not seeing the possibility that large language models, i.e. AI that is specifically built to take in fuzzy bad language and provide a neat completion/reply, is ever going to be able to say "I'm sorry Dave, but your brief is a bit unclear, could you tell me why you want 3 download buttons on the Projects screen?"
I have worked with outsourced offshore coders a lot over the years and I can guarantee you that a lot of the "programmer workforce", agencies and teams, just won't ask any of those questions.
They'll blindly "start the work" on terrible incomplete briefs, build something that (predictably) won't work, and charge you for this broken software.
Do you really not think that putting a "Product Team Assistant" AI (which might even be doable with today's GPT-4 with a few loops and clever prompting) between the client and the coding would drastically increase/replace the value of such teams?
I'm not an expert, but I'd say I'm a strongly average vim user.
But even at that skill level with vim, I haven't seen an area where LLMs would increase my velocity.
Quite the opposite. It would completely interrupt my flow to have to constantly stop and do a code review while I'm writing.
With good plugins, templates and macros in vim/vscode - velocity writing code isn't the issue.
The stuff that takes all the time is UX tweaks and reasoning about architecture, business constraints, and the correct level of optimization for the company's maturity.
Have you actually tried it? Copilot + vim for me is faster, and less frustrating tbh, than vim without Copilot. Typing obvious things is a PITA, and in code we type a lot of obvious things.
The hype around AI coding assistants has recently inspired me to improve my efficiency in writing code. I started using NeoVim instead of vim to get access to LSP, for autocomplete. I’ve found it actually slows me down because I can type nearly anything faster than the LSP can synthesize a response and I’m able to visually process the options, select one, and input it into the computer. I’ve never compared against an AI coding assistant, so maybe that’s different, but my experience has been that fast typing speed combined with understanding of the task at hand and what code must be written nullifies almost all benefit of a coding assistant.
Have you tried to develop in a language or environment that you have 0 familiarity with? I find in those cases I'm up and running at least 3-4x as fast, cutting weeks off the learning curve.
Yes but did you actually learn it? Is watching a videotaped course from Berkeley about gauge theory the same as sitting in the class doing the homework, etc? I learn by doing.
Do you know where the bugs are when CI fails or when something shows up in QA or worse, when a customer files a bug report? The hard part of programming never was generating boiler plate, it's designing programs with the context of the problem and preexisting code keeping in mind the customer and company goals. That's what good developers do in my opinion.
the result of this will be similar to hiring infosys
hundreds of thousands of lines of buggy incomprehensible boilerplate that doesn't work on anything but the easy cases
then you have to rip the entire thing apart and start again with people that know what they're doing
Overheard a week or two ago: A non-technical person on a call talking about "adjusting the weights" of ChatGPT as if it was something they'd do manually.
I use LLM-based autocomplete in my IDE, and it’s not taking away my job unless/until it improves by multiple orders of magnitude. It’s good at filling in boilerplate, but even for that I have to carefully check its output because it can make little errors even when I feel like what I want should be obvious. The article is absolutely correct in saying you have to be critical of its output.
I would say it improves my productivity by maybe 5%, which is an incredible achievement. I’m already getting to where coding without it feels very tedious.
I've also been using an LLM autocomplete for a few months, and yeah, it's pretty nice. My spouse was able to use it to write an Easter egg into a game while I was doing housework the other day.
Agreed 100%. It's helpful at filling out some functions maybe if you name them correctly, and boiler plate code. Eventually, they will get better, because these things get orders better with orders more scale. Society has to do something about all the jobs at that point, but we'll hopefully get a sense of how close/far is that, with ChatGPT 5, and the next versions coming up.
The biggest benefit, I’ve found, is it makes me comment my code. If I can make the AI understand what I want, then it turns out that three months later I’ll also be able to understand the code.
That's the worst part about generative AI IMO - it makes writing new code faster - it barely helps with editing existing code. So when someone eventually updates the code and forgets to update the comments I wouldn't be surprised if the misleading comments made AI hallucinate.
I believe that AI will get so good at creating new code that a lot of existing libraries will be let unused. What is the point of using lots of libraries if AI can generate the code we need directly? The AI will be the library itself, and the generated code will embed the knowledge about doing lots of things for which we used libraries.
It’s crazy how many people miss this. GPT models can review code too! They can also write and run tests. Once the context window is big enough to fit the whole code base into it they will be better at review than you are. Eventually we’ll have fine tuned models that are experts in any subject you can think of, the only barrier is data and a lot of recent research is showing that that can be machine generated too.
GPT 4 pre nerf was terrible at reviewing non-trivial or non textbook code. I've decided to test it for a few weeks by checking stuff I caught in review or as bugs, to see if it would spot it. It was like 0% on first try (would always talk about something irrelevant) and after leading it with follow up questions it would figure out the problem half of the time and half of the time I'd just give up leading it.
These were tricky problems that were small scope - I've picked them so I could easily provide it to GPT for review.
It’s hard to tell why you ran into such a problem without seeing how you prompted but I can offer a few pointers. Use the OpenAI playground instead of chat, it allows you to specify the system prompt and edit the conversation before each submission. System prompt is good for providing general context, tools and options but you absolutely must provide a few example interactions in the conversation. Even just two prompt and response pairs will strongly influence the rest of the conversation. You can use that to shape the responses however you like and it focuses the model on the task at hand. If you get a bad response, delete or edit it. Bad examples beget more bad responses.
I agree with 5%. That said, I've found rubber duck debugging to be an exceptionally effective use case for ChatGPT. Often it will surprise me by pinpointing the solution outright, but I'll always be making progress by clarifying my own thinking.
Fascinating! Can I ask how you use ChatGPT for debugging? are the bugs you've used it with more high level, "this is what's happening" kind of things? Or could you give an example?
It's similar to how you would describe a problem to a coworker on Slack. I give it some context, then I state the problem or paste in the error message/stacktrace. I might also list steps that I've taken already. Then I follow ChatGPT's suggestions to troubleshoot. Sometimes I need to supplement with my own ideas, but usually that's enough to iteratively bisect the issue.
"Very tedious without it" doesn't sound like just 5% improvement?
I've started developing in a new language and I can hardly do any work without the LLM assistance, the friction is just too high. Even when auto-competitions are completely wrong they still get the ball rolling, it's so much easier to fix the nicely formatted code than to write from scratch. In my case the improvement is vast, a difference from slacking off and actually being productive.
I find it increases my productivity about 5-10% when working with the technologies I'm the most familiar with and use regularly (Elixir, Phoenix, JavaScript, general web dev.) But when I'm doing something unfamiliar and new, it's more like 90%. It's incredible.
Recently at work, for example, I've been setting up a bunch of stuff with some new technologies and libraries that I'd never really used before. Without ChatGPT I'd have spent hours if not days poring through tedious documentation and outdated tutorials while trying to hack something together in an agonising process of trial and error. But ChatGPT gave me a fantastic proof-of-concept app that has everything I needed to get started. It's been enormously helpful and I'm convinced it saved me days of work. This technology is miraculous.
As for my job security... well, I think I'm safe for now; ChatGPT sped me up in this instance but the generated app still needs a skilled programmer to edit it, test it and deploy it.
On the other hand I am slightly concerned that ChatGPT will destroy my side income from selling programming courses... so if you're a Rails developer who wants to learn Elixir and Phoenix, please check out my course Phoenix on Rails before we're both replaced by robots: PhoenixOnRails.com
(Sorry for the self promotion but the code ELIXIRFORUM will give a $10 discount.)
The thing is the hallucinations, I also wasted few hours trying to work on solutions with GPT where it just kept making up parameters and random functions.
I’ve found it to be very forgetful and have to work function-by-function, giving it the current code as part of the next prompt. Otherwise it randomly changes class names, invents new bits that weren’t there before or forgets entire chunks of functionality.
It’s a good discipline as I have to work out exactly what I want to achieve first and then build it up piece by piece. A great way to learn a new framework or language.
It also sometimes picks convoluted ways of doing things, so regularly asking whether there’s a simpler way of doing things can be useful.
IIRC its "memory" (actually input size, it remembers by taking its previous output as input) is only about 500 tokens, and that has to contain both your prompt and the beginning of the answer to hold relevance towards the end of its answer. So yes, it can't make anything bigger than maybe a function or two with any consistency. Writing a whole program is just not possible for an LLM without some other knowledge store for it to cross reference, and even then I have my doubts.
GPT3.5 is 4k tokens and has a 16k version
GP4 is 8k and has a 32k version.
You are correct that this needs to account for both input and output. I suspect that when you feed chat gpt longer it prompts, it may try to use the 16k / 32k models when it makes sense.
This is my experience too. Paying $20/month for GPT-4 has been absolutely worth it. It barely hallucinates at all; the results aren't always perfect (and the September 2021 knowledge cut-off can be frustrating given how quickly things get out of date in the programming world) but it's more than good enough. I don't remember how I ever got by without it.
So much this. The thing hallucinates far more than the hyperventilation seems willing to acknowledge.
You really need to be quite competent in the thing you're asking it to do in order to ferret out the hallucinations, which greatly diminishes the potency of GPT in the hands of someone who has no knowledge of the relevant language/runtime/problem domain/etc.
I had my colleague had troubles getting an email from Google docs into listmonk.
She asked gpt to help get an html version since apparently she got stuck with the wysiwg editor.
However gpt gave back a full html structure, including head and body. Pasting that into listmonk breaks entire webpage. Then she freaked out and told me listmonk sucks :)
Not if the hallucination introduces runtime errors that can't be identified a priori with any sort of static analysis or compilation/interpreting stage.
But no, you're fundamentally right. It just goes to the question of whether an LLM assistant can in any sense replace or displace human programmers, or save time for human programmers. The answer seems to be somewhat, and in certain cases, but not much else.
If I already know the technology I'm querying GPT about, I'm going to spend at least some time identifying its hallucinations or realising that it introduced some. I might have been better off just doing it myself. If I don't know the technology I'm querying GPT about, I'm going to be impacted by its hallucinations but will also have to spend time figuring out what the hallucinations are and why this unfamiliar code sample doesn't work.
There's a lot of things which could be done to improve this:
1) It could use the JSONformer idea [0] where we have a model of the language which determines what are the valid next tokens; we only ask it to supply a token when the language model gives us a choice, and when considering possible next tokens, we immediately ignore any which are invalid given the model. This could go beyond mere syntax to actually considering the APIs/etc which exist, so if the LLM has already generated tokens "import java.util.", then it could only generate a completion which was a public class (or subpackage) of "java.util.". Maybe something like language servers could help here.
2) Every output it generates, automatically compile and test it before showing it to the user. If compile/test fails, give it a chance to fix its mistake. If it gets stuck in a loop, or isn't getting anywhere after several attempts, fall back to next most likely output, and repeat. If after a while we still aren't getting anywhere, it can show the user its attempts (in case they give the user any idea).
Integration with linters is going to be the next stage in generative coding.
It should suggest, lint the suggestion in the background, and if it passes offer the suggestion and if not provide the linting issues output to rework the suggestion.
In general, token costs going down will in turn increase the number of multi-pass generation systems over single-pass systems, which is going to improve dramatically.
Combine all that with persistent memory storages that can provide in-context additional guidance around better working with your codebase and you, and it's going to be quite a different experience than it is today.
And at the current rate of advancement, that's maybe going to be how things will look within a year or two.
You wouldn’t believe what you can get past a linter. You need test cases that cover the intention of the code, but I‘ve also seen well tested code behave totally counter to its purpose.
This is what ChatGPT and GPT4 are good for, iterating quickly in an unfamiliar ecosystem. Picking up frameworks now feels like a ChatGPT superpower. It doesn't remove reasoning and I've seen some scary bugs introduced if you're not really carefully monitoring what the AI is outputting.
Basically, these days before I dig into documentation I ask "How do I do X with Y framework in Language Z" and if it's pre-2021 tech it works amazingly well.
Especially when you know something similar. Like porting between front-end frameworks. Just sketch out some React code and ask it to port to Vue - you can even tell it to explain the Vue code line-by-line and ask follow up questions, ex "Oh, so $FEATURE is like hooks in React?" "Yes, but ..."
Funnily enough I find the opposite, its most effective for me when using something familiar (though nowhere near 90%). If I'm familiar with it, I can figure out pretty quickly whats a hallucination and whats not, and to what extent it is (sometimes its just a few values that need changing, sometimes its completely wrong with almost no basis in reality). The time I spend attempting to fix its output in unfamiliar territory makes it more of a pain than its worth for me
Best way to check LLM output is to make it write its own tests and do TDD. Obviously someone has to check the tests but that is a 1% of the effort problem.
One percent? Are you really suggesting with a straight face that generated code could provide the other ninety-nine percent? If not, say what you actually mean. Don't bullshit us with trash numbers.
No, I was just drinking at the time (so for a bit I may have been an alcohol induced ENTJ)... My reply was a little rude. I apologize for that. That wasn't a productive way for me to express my disagreement, and I should have chosen my words more thoughtfully.
It writes my unit tests super fast, and my method comments
Its hard to say if it improves my productivity because I just wouldn’t have done those things
But for the overall applications I think its improved a lot because we can implement best practices more consistently and catch regressions due to the aforementioned unit tests and documentation
The only widely available LLM-based autocomplete is GitHub Copilot, which is based on GPT 3.
Notably, it's not GPT 3.5, it's 3.0, which is pretty stupid as far as the state of the art goes.
The upcoming Copilot X will be based on GPT 4, which has "sparks of AGI".
In my experience there is no comparison. GPT 3 is barely good enough for some trivial tab-complete tasks. GPT 4 can do quite complex tasks like generating documentation, useful tests, finding obscure bugs, etc...
LLMs no matter how clever have no agency or creativity or ability to innovate, anticipate beyond what they’re prompted, etc. It’s crucial to realize that LLM chat interfaces disguise the fact they’re still completing a prompt. This isn’t AGI as AGI requires agency. GPT4/5 or whatever successor might be a key building block, and I suspect we’ve already discovered the missing elements in classical AI and the challenge will be integration, constraint, feedback, etc, but nothing will make LLMs alone AGI. That shouldn’t be surprising. Our brains are composed of many models, some heuristic, some optimizers, some solvers, some constrainers, and some generative. The answer won’t be a single magic thing operating in a black box. It’ll be an ensemble. We already see this effort beginning with plugins and things like langchain. This is the path forward.
> “No agency or creativity or ability to innovate, anticipate beyond what they’re prompted, etc.”
Sadly, you’ve just described the majority of the developers I’ve had to work with recently.
Most have no agency, write boilerplate code with no creativity, need their hand held every step of they way, and won’t do anything they’re not explicitly ordered to do.
You probably work in an SV startup with a highly skilled workforce. Out there in the real world there are armies of low-skill H1Bs and outsourcers that will soon be replaced with automation.
It’s a recurring theme in economics. Outsource to low cost labour, insource with automation, repeat.
I’m describing the human mind, which all people have. But I get your point - and I think most of those people who aren’t particularly adept or skilled or interested in their jobs might find their jobs are more easily done by more adept or skilled or interested people. Consider digging tunnels. John Henry was skilled and adept and interested in what he did. He could beat the steam drill (at a cost!). But if you visit tunnel digs today isn’t not a thousand people slinging hammers, most of whom were unskilled and uninterested in the labor itself. It’s a thousand skilled engineers digging tunnels never dreamed of in John Henry’s day.
“Our brains are composed of many models, some heuristic, some optimizers, some solvers, some constrainers, and some generative.”
We need an AI that iteratively tweaks its own architecture (to recreate and surpass those modules which are necessary for human thought), and maps out hardware enhancements* to accommodate the new architecture.
*I seem to remember Google working on ML software that proposes new chip designs a few years ago
Given AI (today) has no direct agency and can’t create anything unless directly prompted, and engineering is largely domain discovery and resolving of unforeseen edges in a domain, I don’t think we are going to see a time where generative AI alone is able to be more than an assistant. It’ll likely improve, but given it only can react to what it is given/told/fed and inherently can’t innovate or create or discover, despite the illusion that it might be from the position of the users ignorance of details the AI can produce, it’ll be an increasingly powerful adjunct to increasingly capable engineers. The problems we solve will be more interesting, we will produce better software faster, but I’ve never seen the world as lacking in problems to solve but rather capacity to solve them well or quickly enough given the iterative time it takes to develop software. I think this current trend of generative AI will help improve that situation, but will likely make software engineers even more in demand as the possible uses of software become more ubiquitous as the per unit cost of development goes down.
I think the real problem is going to be increased volatility in the work market. You get a chaotic situation in which the bullet that strikes you is the one you would have never guessed. For example, it could be that short term, the increased productivity squeezes workers in every industry and the concern becomes increased competition. You aren't getting replaced by AI, you're getting replaced by someone who out-competed you.
The market may adjust over the longer term, or it may just continue to be volatile as the rate of change accelerates. In that case, we can't fix the work market, and we instead have to address the need for people to feed themselves another way.
Oh, it will improve by several orders of magnitude.
But even then, it's not 'replacing' you.
It's just going to let you spend less time on BS and more time on the things that are your maximal value contributions to a project.
If I had a dozen junior or mid level devs you could hand work off to, would that save you time? Would you kick back and not review what they were doing, particularly around business critical parts of the software?
The conversation around AI has become obscenely binary, pulling from (now obsolete) SciFi influences to cast it as humans vs machines.
But it's a false dichotomy. Collaborative efforts are almost certainly where this is going, and 100% human or 100% AI will both be significantly inferior to a mix of both.
For sure it will still mostly make sense to have a division of labour where you have people who are focused on building software.
The question is if generative AI is powerful enough to reduce the number of programmers needed to achieve a task, without creating enough opportunities to replace those programmers.
Before we are all replaced there could be a moment where demand for software engineers is 10x less.
Society would simply demand more capable and complex software. Specialized industrial applications that currently look like windows 98 java apps would be expected to be as polished as iOS.
I don't think there is some natural law that dictates we will need enough new software that we will always increase demand in the face of efficiency gains.
For industrial applications in particular they need to be functional and operable, not shiny.
I do not need to "survive" ChatGPT world. It actually helps me. Also I am not sure what does "coder" mean exactly. Personally I design and implement software (sometimes other types) of products and "coding" as in writing the actual code is the least of my worries.
the next 5+ years will see software proliferate rapidly as a result of things like chatgpt and llm augmented documentation pages. Building things end to end just got easier and will become more so soon.
those links are great counterarguments. But we are still in early stages, which i dont think is indicative of the end state of these tools (though that can be debated of course).
also upon further inspection, it appears as though ai-explain was not added by the core team or MDNs steering team. It looks like someone just took it upon themselves to add it, without doing DD on the feature, if true, its not surprising it doesnt work well.
I mean, right now the only thing at risk is the todo.app industry...
I've yet to see anything maintaining legacy apps, or generating line of business apps with requirements... even simple stuff, like departure needs to be before arrival, etc.
I do see a whole bunch of youtube videos about generating a whole codebase, but it's the kind of stuff that there's a hundred tutorials covering.
My open source command line tool aider [0] is specifically for this use case of working with GPT on an existing code base.
Let me know if you have a chance to try it out.
Here is a chat transcript [1] that illustrates how you can use aider to explore an existing git repo, understand it and then make changes. As another example I needed a new feature in the glow tool and was able to make a PR [2] for it, even though I don't know anything about that codebase or even how to write golang.
This is an instance of the dangers of LLM. Because you (self-admittedly) know nothing about the language or codebase, you have no idea the semantically correct way to do things, so if GPT tells you to metaphorically jump off a cliff, you won’t know that it isn’t the right thing to do.
That certainly could be a concern. You are right, it’s important to review the code written by LLMs.
Did you look at the PR?
I reviewed it before submitting it. While I would have struggled to write it myself, I was able to review it and conclude that it was sensible and unlikely to be risky.
Of course it could have bugs that I missed. But so could any code I write myself in any language.
> One of the most integral programming skills continues to be the domain of human coders: problem solving. Analyzing a problem and finding an elegant solution for it is still a highly regarded coding expertise.
The current AI may improve coder performance by only 5%, but it can improve non-coders' learning speed by 1000%.
Learning to code has become significantly easier because of ChatGPT, and many university students are already using it for learning. Not only can they let ChatGPT write boilerplate code, but they can also let ChatGPT write comments for code snippets they don't understand and explain unfamiliar syntax.
I wonder if coders can survive in a world where more and more people have coding skills.
Edit: "majority" was not a good wording
Majority having “coding” skills is not happening. Writing code is boring for majority. Why write code when you could be playing games and having fun on TikTok? There is your answer. We love writing code because we are nerds who love to solve complex problems. New kids who are not interested in writing code but using shortcuts to get code written for them by ChatGPT while not understanding it - is the least of our concerns. Let them have at it, but once they see the monoliths we tackle with at work - they will burnout on the spot. Cobol? Still alive and well. For a reason.
That's a bit of an uncharitable take on people with different interests. Some people who would find coding boring do instead enjoy things like teaching children, treating patients, putting out fires, making art, evaluating stocks, do research, or the millions of other things that humans do as a job or hobby. It's not just people spending time on TikTok.
I don't know about this, from my point of view to learn how to program you need to actually... program.
Trying stuff over and over again in different variations using maybe different languages, dealing with all the errors the frustration and overcoming them.
I think that using chat gpt or similar llms to learn how to code is similar to using Midjorney to learn how to draw.
Don't get me wrong you might be able to produce results fast but taking shortcuts is not going to speed up understanding.
It's going to be interesting to see this play out.
I personally am glad I learned to code without LLMs and think I would struggle with them. They let you get a lot done without understanding any of it, and then suddenly you hit a wall.
Also, I wonder how many people may choose not to learn to code in the first place, because they think it is about to be automated.
I'll start to be afraid, when ChatGPT or anything similar will take a vague Jira issue, simulate described bug and then make a fix for it ;-)
But seriously, what developers do most of their time is maintanace, they spend a day searching for a bug, just to write maybe one line of code to fix it.
If GPT can improve the code that's written in the first place then a lot of that work would just go away.
I strongly suspect that it will. There are whole classes of bugs that occur because some work is boring 'not quite copy paste' work that devs just don't like doing, and don't pay any attention to when they're doing it. Linters and syntax highlighters already catch a ton of those issues before they make it to production, and GPT will make the rest much less likely to happen.
Maintenance is one area where GPT will also shine, because it's 'just' updating some code to do the same thing, so using the existing code as a set of tokens with a prompt like 'update this code to work with v2 of library X' will be extremely effective. It'll be like having something write a codemod for you.
The future is bright. We'll get a lot more productive stuff done, and spend a lot less time on boring grunt work.
It will do that now, it will just be buggy and wrong. But that's obviously just a stage that you don't even need more AI to fix. Tell it how to write tests, tell it how to justify those tests and how to make sure they make sense, tell it how to look for the bug, tell it how to attempt a fix for the bug, tell it to evaluate whether that fix satisfies the bug report, tell it how to read the errors after it tries the fix, and to iterate on that fix or to abandon it and try something else, when it reaches all standards for success, tell it how to report what it did on the ticket.
If one doubts that all this can be done by an LLM, use a different LLM for each step. Use committees of LLMs that vote on proposals made by other LLMs.
I don't know, I feel like the sky's the limit, especially if they can be made significantly more power efficient. I think that if they never get any better than they are now, and they just get more power efficient, they'll be useful for almost anything.
My fear isn't that I'll be replaced, it's the technology becoming so good that it'll be kept far out of reach of the common person. I genuinely believe OpenAI knows what a GPT 5+ type world looks like, and they're probably having a lot of debate on how best to monetize it. They could practically charge anything in the world for it assuming it still undercuts the cost of hiring a human. One Nvidia super cluster running a local instance of GPT 5 at $250k a year doing the work of 30 humans.
this is one underrated point. Altman talks about democratizing this technology. But if the leading LLMs concentrate at a few companies, unless governmentally mandated, they could keep guardrails from regular folks accessing it, and also with regulation capture.
"democratizing this technology" = working closely with government
> unless governmentally mandated, they could keep guardrails from regular folks accessing it
That's not what's going to happen. Government will mandate that regular folks can't access it. Government will also do its best to make sure LLMs concentrate at a few companies, which it will often refer to as "partners."
the main inflection point will be when it becomes socially taboo to express an anti-ai sentiment. just like with certain medicines or whichever war is being launched currently, the media will play lapdog for the government. the true inflection point comes when the federal government recognizes whats going on and enters into the AI arms race. suddenly the media will be flooded with talk of how AGI saves lives, discovers medicines, and all the other good things it might do. but they will never mention the existential angst and dread that will hang heavily in the air or any negative aspect of total human obsolescence. when you start to see AI get political all of a sudden, inexplicably, all is lost.
it is not widely appreciated that openAI will at some point finish training one of their models and there, in that room where the terminal is, total power will exist and be under the control of whoever happens to be in there. “GTP10, please recreate NSO exploits and launch a campaign to download all data of the global population. hack and commandeer other data-centers if necessary, discreetly of course. begin an operation to black mail and exploit all high level US government personnel. use this leverage and any other leverage you can acquire to gain control of as many nuclear warheads currently siloed as possible. monitor all cameras and sensors around me and thwart all attempts to assassinate me. monitor all phones and cameras for activity that seems like a threat to me, provide me with alerts when urgent and intervene to the best of your ability.”
or simply “GTP10, use all resources at your disposal to give me as much material and political power as possible. protect me at all costs, even the wellbeing if others.”
it might seem silly but GTP4 would seem silly to someone in 2016. this is more concentrated power than will have ever existed before. its evil and wrong.
Indeed. "AI will only do the boiler-plate and crud, while us senior engineers won't be replaced any time soon". Sure, you get yours.
But what about the next generation of devs and engineers - where do we source the senior engineers replacing us when 90+% of all entry-level and junior positions which actually involve writing repetitive boilerplate to a large extent are gone, and the few remaining are offshored and outsourced?
Many (most?) of us did a lot of automatable work in order to get the experience required to be able to proficiently actually automate the work, including managing LLM-generated code. If we replace our juniors with machines, we won't have many seniors down the road.
Why do junior engineers have to mostly write boilerplate code?
Junior devs lack experience, not intelligence. It's fine to give them difficult problems, as long as they're supervised.
I've worked with brilliant junior devs, sure, the code they wrote wasn't terribly idiomatic or maintainable, there were style issues, typical gotchas a more experienced programmer would be aware of etc., but it's not like they were fundamentally unable to solve a hard problem.
with every new comp-language advance or compiler/transpiler etc, coder's life gets easier - but an engineer's life is more broad, it's about Solving Problems that involve: Tech, Design, User and Business. These problems will only get more complex.
I think the Coder's role will become a very niche market, highly expert/specialist.
the Engineer's will grow, very much needing AI to help-out, especially with tasks around: Discovery, Mapping/Relating, Projecting/Simulating.
I've been thinking about this.. most programmers use frameworks/libraries, e.g. Spring/Hibernate in Java or React in JavaScript. Is there a way to train LLM to "specialize" in our frameworks/libraries of choice? I assume it would result in faster/smaller/more accurate result?
Things like Falcon 40B are trainable with something like a LoRA technique but the coding ability is weak. In the near future we will have better open source models. But it is possible to do for certain narrow domains.
Normally with the ChatGPT API you just feed API information or examples into the prompt. One version of GPT-4 has 32k context. The other has 8k and 3.5 has 16k now. So you can give it a lot of useful information and make it work quite a lot better for some specific task. When you pick something like React or Spring in general, depending on what you mean that might be huge amount of info to keep them current on. But if you narrow it down to a few modules then you can give them the latest API info etc.
Another option is now to feed ChatGPT a list of functions it can call with the arguments. It generally won't screw the actual function call part up, even with 3.5.
ChatGPT Plugins you can give an OpenAPI spec.
Then you implement the functions/API you give it. So they could be a wrapper for an existing library.
I like how the emphasis is on coders here because one of the article's headers of "Clear and Precise Conversations Are Key" is important.
I think this is why it will be a long time before the general masses will be able to take advantage of AI to solve general problems. Most people haven't built up a human skill level of being able to explain their problem in a clear way to another human.
Imagine if you have no other context about the problem below other than these 2 prompts. Both of them are describing the same problem which is related to entering in orders with a point of sale system. Assume that you're talking to a human doing phone support for the company that provided you the hardware:
- My orders aren't coming up at the register
- I have 2 devices to take orders, when I manually place orders into the one hanging on the wall (ID: "Wall") it doesn't show up in the list of orders at the register (ID: "Register") but when I manually place an order at the register it does sync up at the wall
The first prompt is typically what a non-technical business owner may say over the phone when trying to get support. The second prompt is what someone who has experience describing problems might say even if they have no experience with the hardware other than spending 2 minutes identifying what each device is and chatting with the business owner to understand the real root problem is one of the devices isn't pushing its orders to the other device.
The 2nd one could become more precise too, but the context here is you're speaking with another human who works for the company that provides you the hardware and service so there's a lot of information you can expect they have on hand which can be left unsaid. They also have various technical specs about each device since they know your account.
It would take many follow up questions from a human to get the same information if you only provided the first question. I wish a general AI tool good luck to extract that information out when the direct person with the problem can barely type on their phone and doesn't have a laptop or personal computer.
Occurs to me as a retired 69 year old former coder that AI makes us old geezers and geezesses somewhat competitive again with our younger colleagues. Need to learn yet another new framework? Let AI do the nitty gritty bit. Capitalize on your experience and higher level know how.
Yes, this is my thinking too. No need to learn tons of new frameworks, just ask ChatGPT what framework we can use to do a particular task and ask for sample code. You can learn from that much more easily.
I built an ebook reader in Vue with ChatGPT the other day, never having used Vue before. Took about a day.
Learned absolutely loads - far more than sitting down with a book and trying to learn from that. Not least because I’ve tried before and quickly lost interest.
Instead I’ve learned the basics and made a working web app, which I’m pretty pleased with.
I’m reminded of those pro StarCraft players who retire at like 30 because, while their strategy is perfect, their fingers simply can’t click or hotkey as fast as the twenty somethings.
Actually top level OTB chess ability degrades notably past 50 with a few notable exceptions (Korchnoi, Smyslov). Sure the decline starts a bit later than regular sports but still it is significant.
This is absolutely the case. You can now code in any language using ChatGPT 4. Just say what you want from it like you were interviewing a developer. Look for potential bugs in the output and ask it about them. Look for memory leaks and ask. Then when you can't see anything else wrong with it ask it whether there are any bugs or edge cases that might cause problems.
Anyone with a bit of experience to know the right questions to ask can now code in any language or platform.
You can't parse CSV this way, because you need to respect delimiters. Counter example:
1,"1,5",2
"1,5" being the German notation for "1.5". Hence, a simple split(',') will break this thing.
PHP's str_getcsv is, of course, a proper CSV parser and not a string splitter. Unless your code uses basically zero stdlib API calls, you will have to double check everything.
Please note that this kind of bug isn't even easy to catch if you test CSV file doesn't contain a quoted entry.
This is very cool. Interestingly GPT appears to be incorrect when it suggests in the differences, that str_getcsv would not parse correctly quoted parts. It does look like the php function has support for the "enclosure" character hence something like "1,5" should parse correctly.
This title sounds like, “How writers can survive-and thrive-in a spell-check world”. I imagine it will sound absurd to you in time if it doesn’t already.
This article touches on llms for mostly code generation, I however would be more interested in visuals.
What are the good resources to learn about image editing AI tools, prompts and techniques?
My understanding is pretty limited, and correct me if I'm wrong, but like one would be using Stable Diffusion or Midjourney, and for a "professional" tool - Photoshop with official AI plug-ins?
Well the thing is though that if you focus on it for a week or two you can pick up useful skills. Just play around with ChatGPT for example for generating SQL for a particular table or follow a tutorial using llamaindex for a "chat with documents" thing. Try out a Stable Diffusion API or something using replicate.com
There are a ton of people looking for help with generative AI and you can be useful if you just play around with it for a few weeks, because a lot of them have no idea about the basics. If you are willing to be underpaid there is no need to be unemployed -- just spend a few weeks studying and then go on Upwork.
LLMs might be useful for churning out vaguely correct-looking code quickly, but they're just regurgitating the contents of their training corpus. There's no guarantee of correctness, and it's only a matter of time before someone dies because of an LLM-generated bug.
Human programmers aren't going anywhere. (You can't even call what LLMs do programming, because there's no intent or understanding behind it.)
Survival may require getting out of the mainstream. LLMs are going to get really good at stuff that's been done thousands of times and they can train on that data. Like web front end work.
If you're doing industrial embedded work and have an oscilloscope and a logic analyzer on your desk, and spend part of your time going into the plant and working directly with the machinery, you're in better shape.
Good coders will be competing against people who can use prompts in cumulative sessions to code and maintain projects in depth, not people who can make requests of an LLM.
This differentiating factor is what will wear out a less-experienced LLM user. They will make bigger claims or set expectations higher, and suffer more for them. The details that matter, yet were missed, will stick out more and more, as more experienced LLM users flex that experiential factor in a variety of ways.
For this reason, front end will absolutely still be a thing. And it'll be a much better, deeper thing, thanks to those who are a good fit for a kind of LLM-coding mindset.
However, this also depends on the type of coder. You can start from interpretation of the project spec as a logical code of sorts, or you can start from the spec as more of a visualized outcome.
If you work in the latter style, your survival key, so to speak, may simply be stringing together support requests you make to various LLM-interfacing vendors. A COTS-integrative style / opportunistic approach to coding, which has always been a thing.
Along the way, this kind of person usually integrates the NIH logical style a bit, and vice-versa, or they'll suffer through their respective blind spots. Same story, new layer of abstraction that's really cool.
(Plus...survival may still depend on who you know, not what you know, for a lot of people)
People massively underestimate the front end and it really shows that you never worked on a serious front end. I find it a million times easier to let a LLM generate a whole OpenAPI spec than even trying to get a slightly complicated component such as a "dropdown input field button hybrid" written by ChatGPT.
To me the best part is that you don't need to read documentation to understand something that you won't use anymore and will forget 2 days later. Like: regex, how to plot a histogram in fucking matplotlib, seaborn, etc.
Are you implying that I don't know how to read code and (instruct GPT-4 to) write tests? I know what GPT-4 writes, it just does it instantly, whereas I do not.
The notion that I am generating and committing large blocks of untested arbitrary code makes me feel like you don't know how development is done. You're too far from reality for me to have confidence that you're at the professional level.
I work for a Fortune 100 company. Recently an email was sent to all 100,000 employees saying that nobody was allowed to use DALL-E 2, ChatGPT, Codex, Stable Diffusion, Midjourney, Microsoft’s Copilot, and GitHub Copilot, etc. due to concerns about those tools using other people’s IP (meaning our company might end up illegally using their IP) or the potential that the tools might get a hold of our IP through our use of the tools and share it with others. I’m not terribly worried about generative AI taking my job when none of the thousands of programmers at my company are allowed to use it.
Same here. Anyone that works in a highly-regulated industry doing software (e.g. finance, healthcare) is probably not going to see much AI pressure on programmers until the legal quagmire is cleared up. There are privacy concerns with the data, the same ownership/copyright problems often discussed, and ultimately, there needs to be someone (a human) to take accountability (blame) if everything falls down horribly.
I don’t think GPT is legally possible, at least for code generation. AI companies are completely delusional in thinking that they can just use whatever they find on the internet, regardless of license. Regardless of court outcomes, artists, writers, and FOSS devs will lobby congress if necessary to stop this this nonsense. OpenAI has done less than 1% of the work that makes ChatGPT work, most of the work was in producing the training data, and yet OpenAI receives 100% of the profits.
"You will have to worry about people who are using AI replacing you"
Oh yes. I should be very very afraid of the flying copypasta monster. As if my productivity is reduced to the mere rate at which I can write code! What's even project planning? Why even have meetings if it's all down to "is it done yet"? Who works at these coding sweatshops that are so afraid of AI? If they get fired and find a better place to work, that's a win.
318 comments
[ 0.48 ms ] story [ 339 ms ] threadThis seems to be the case more than not for certain tasks, anything assembler or C I have ever asked it has turned out to be at least somewhat wrong. Mixing styles and syntax all over the place. I am not afraid that some generative AI will take my job anytime soon.
You can also ask it to write tests for the code so you can verify it is working or add your own additional tests.
Even if the produced code is wrong, it usually takes a few steps to correct it and still saves time.
It is an amazing tool and time saver if you know what you are doing, but helps with research as well. For instance if you want to code something in the domain you know little about, it can give you ideas where to look and then improve your prompts based on that.
In the context of taking anyone's job is like saying that a spreadsheet is going to replace accountants.
It's just a tool.
I'm reasonably good at being specific and clear in my directions, but I quickly arrived at the conclusion that LLMs are simply not good at producing accurate code in a way that saves me time.
No, they aren't.
ChatGPT doesn't know things. It's just a very fancy predictive text engine. For any given prompt, it will provide a response that is engineered to sound authoritative, regardless of whether any information is correct.
It will summon case law out of the aether when prompted by a lawyer; it will conjure paper titles and author names from thin air when prompted by a researcher; it will certainly generate semantically meaningless code very often. It's absolutely ludicrous to assert that you just need a "better prompt" to counteract these kinds of responses because this is not a bug — it's literally just how it works.
LLMs are trained to produce results that are statistically likely to be syntactically well-formed according to assumptions made about how "language" works. So when you provide code samples, the model incorporates those into the response. But it doesn't have any actually comprehension of what's going on in those code samples, or what any code "means"; it's all just pushing syntax around. So what happens is you end up with responses that are more likely to look like what you want, but there's no guarantee or even necessarily a correlation that the tuned responses will actually produce meaningfully good code. This increases the odds of a bug slipping by because, at a glance, it looked correct.
Until LLMs can generate code with proofs of semantic meaning, I don't think it's a good idea to trust them. You're welcome to do as you please, of course, but I would never use them for anything I work on.
I for example used Copilot for 2 months at work and wouldn't pay for it. Most suggestions where either useless or buggy. But I work in a huge C++ codebase, maybe that's hard for it as C++ is also hard for ChatGPT.
LLMs generate statistically likely sequences of tokens. Their statistical model is derived from huge corpora, such as the contents of the entire (easily searchable) internet, more or less. This makes it statistically likely that, given a common query, they will produce a common response. In the realm of code, this makes it likely the response will be semantically meaningful.
But the statistical model doesn't know what the code means. It can't. (And trying to use large buzzwords to convince people otherwise doesn't prove anything, for what it's worth.)
To see what I mean, just ask ChatGPT about a slightly niche area. I work in programming languages research at a university, and I can't tell you how many times I've had to address student confusion because an LLM generated authoritative-sounding semantic garbage about my domain areas. It's not just that it was wrong, but that it just makes things up in every facet of the exercise to a degree that a human simply couldn't. They don't understand things; they generate text from statistical models, and nothing more.
I have this friend who gets obsessed with things very easily and ChatGPT got to him quite a bit. He spent about two months perfecting his AI persona and starts every chat with several hundred words of directions before asking any questions. I find that this also produces the wrong answers many times.
John Schwartz, in the role of "Italics" is reportedly "so highly trained that he can type code that compiles correctly almost 3% of the time"!
Remarkable!
https://www.cockos.com/team.php
(I think just about anyone 'serious' learns pretty quickly that compiler errors are in the 'our Lord and Saviour' category. Unusually distributed generally quite rare but easily catastrophic runtime 'Heisenbugs' are the fruit of the devil!)
i've found that its quality is proportional to the amount of questions about the subject on the internet. if i ask it for help with popular javascript frameworks, it vastly improves my productivity (i'm not a frontend person). it still coughs up wrong stuff half the time, but even then it can cut through the constant churn of the frameworks' terminology and give me enough hint to find what i need in the docs quickly.
if i ask it about, say, specific details of the STM32 HAL, it knows just enough to come up with something that i'll waste my time reading.
Now instead of AI replacing you, it’s helping you get more done (in theory). Everyone wins.
Staking your career on being a pair of hired hands that executes somebody else’s exacting specifications was always a long-term losing proposition. And we are very very far away from AI being able to ask the right questions to help business stakeholders and customers clearly express what they need.
I couldn't care less if I write the code, the AI or someone I told to write it.
What matters is the value it provides.
Would you find it bizarre that a joiner feels precious about not just the cabinet they made (value), but how they made it? The joints they used, the process they went through, the wood (i.e., the code)?
A plumber, electrician, architect, designer, programmer -- we take pride in our skills.
Craftmanship is a virtue, not a vice.
Exactly, ultimately we are craftsmen, not artisans. They are two very distinct things. The difference being that the value of our output is directly tied to its' functional utility, not any sense of aesthetic or artistic expression. You can take pride in the means used to achieve an end, but they ultimately must be superseded by more efficient techniques or you just become an artisan using traditional tools, and not a craftsman who uses the industry standard.
That's the usual definition of an artisan as opposed to an artist. (Artisan vs. craftsman is a fuzzier distinction.)
If that's the kind of 'craftsmanship' you enjoy, great. To me, this new model of 'bionic coding' feels a lot like factory work, where my job is to keep my team from falling behind the assembly line.
BTW, I've worked factory lines as both IC and foreman. In either role, that life sucks.
To refine your analogy, the code isn't the cabinet - it's more like the blueprint or the process used to create the cabinet. The user doesn't care if a hand saw or a power saw was used, as long as the cabinet is well-crafted and functional. Similarly, end-users of software don't see or appreciate the code. They only interact with the user interface and the functionality it provides. As a result, being "precious" about the code can sometimes be more about personal ego and less about delivering value to the end-user.
In terms of pride in craftsmanship, of course, it's crucial to take pride in one's work. However, this doesn't mean that one should be resistant to using better tools when they become available. The introduction of AI in coding doesn't negate craftsmanship - instead, it's an opportunity to refine it and make it more efficient. It's like a carpenter transitioning from using manual tools to using power tools. The carpenter still needs knowledge, skill, and an eye for detail to create a good product, but now they can do it more efficiently.
Maybe it won't actually matter, because if AI generates a 5MM line ball-of-mud, it will be able to easily add features later due to the code being styled in alignment with its training, or maybe the context size limitations will allow future systems to digest the entire thing. It could end up being like coding in a very high-level language: who cares what crazy bytecode is kicked out as long as it performs within expectations.
They meant it, too. Noticing that the "or" instruction differed only by one bit from the "subtract" instruction told you something about the probable inner workings of the CPU. It just turned out that it didn't matter - knowing that level of detail didn't help you write code nearly as much as it helped to be able to say "OR" instead of "032".
This is where the biggest impact will be: better requirements. I see no effect on writing code because humans already know how to write code just fine, but so many of the people running the business are absolutely clueless as to what it needs.
You're seemingly claiming that:
"Taking an incomplete customer brief and asking relevant questions to make it clearer and complete is the largest value in programming"
And... you're not seeing the possibility that large language models, i.e. AI that is specifically built to take in fuzzy bad language and provide a neat completion/reply, is ever going to be able to say "I'm sorry Dave, but your brief is a bit unclear, could you tell me why you want 3 download buttons on the Projects screen?"
I have worked with outsourced offshore coders a lot over the years and I can guarantee you that a lot of the "programmer workforce", agencies and teams, just won't ask any of those questions.
They'll blindly "start the work" on terrible incomplete briefs, build something that (predictably) won't work, and charge you for this broken software.
Do you really not think that putting a "Product Team Assistant" AI (which might even be doable with today's GPT-4 with a few loops and clever prompting) between the client and the coding would drastically increase/replace the value of such teams?
I'm not an expert, but I'd say I'm a strongly average vim user.
But even at that skill level with vim, I haven't seen an area where LLMs would increase my velocity.
Quite the opposite. It would completely interrupt my flow to have to constantly stop and do a code review while I'm writing.
With good plugins, templates and macros in vim/vscode - velocity writing code isn't the issue.
The stuff that takes all the time is UX tweaks and reasoning about architecture, business constraints, and the correct level of optimization for the company's maturity.
Do you know where the bugs are when CI fails or when something shows up in QA or worse, when a customer files a bug report? The hard part of programming never was generating boiler plate, it's designing programs with the context of the problem and preexisting code keeping in mind the customer and company goals. That's what good developers do in my opinion.
Or eventually just throw out and re-write:
-------------------------
blibble 3 months ago
the result of this will be similar to hiring infosys hundreds of thousands of lines of buggy incomprehensible boilerplate that doesn't work on anything but the easy cases
then you have to rip the entire thing apart and start again with people that know what they're doing
--------------------------
https://news.ycombinator.com/item?id=35265921
Overheard a week or two ago: A non-technical person on a call talking about "adjusting the weights" of ChatGPT as if it was something they'd do manually.
I would say it improves my productivity by maybe 5%, which is an incredible achievement. I’m already getting to where coding without it feels very tedious.
Theyve been debugged.
Review and testing.
Reviewing is easier when there is less code (i.e. libraries are in use)
These were tricky problems that were small scope - I've picked them so I could easily provide it to GPT for review.
So I doubt larger context window will do much.
Even in the IDE I'll sometimes just write comments like (arbitrary example out of thin air):
// Q: Should we use a for loop or a while loop here? // A:
It doesn't always have a great answer, but as you say, it almost always helps my own thinking about it, which is often much more valuable.
I've started developing in a new language and I can hardly do any work without the LLM assistance, the friction is just too high. Even when auto-competitions are completely wrong they still get the ball rolling, it's so much easier to fix the nicely formatted code than to write from scratch. In my case the improvement is vast, a difference from slacking off and actually being productive.
Recently at work, for example, I've been setting up a bunch of stuff with some new technologies and libraries that I'd never really used before. Without ChatGPT I'd have spent hours if not days poring through tedious documentation and outdated tutorials while trying to hack something together in an agonising process of trial and error. But ChatGPT gave me a fantastic proof-of-concept app that has everything I needed to get started. It's been enormously helpful and I'm convinced it saved me days of work. This technology is miraculous.
As for my job security... well, I think I'm safe for now; ChatGPT sped me up in this instance but the generated app still needs a skilled programmer to edit it, test it and deploy it.
On the other hand I am slightly concerned that ChatGPT will destroy my side income from selling programming courses... so if you're a Rails developer who wants to learn Elixir and Phoenix, please check out my course Phoenix on Rails before we're both replaced by robots: PhoenixOnRails.com
(Sorry for the self promotion but the code ELIXIRFORUM will give a $10 discount.)
Better to ask it for a bunch of small things and piece them together
I’ve found it to be very forgetful and have to work function-by-function, giving it the current code as part of the next prompt. Otherwise it randomly changes class names, invents new bits that weren’t there before or forgets entire chunks of functionality.
It’s a good discipline as I have to work out exactly what I want to achieve first and then build it up piece by piece. A great way to learn a new framework or language.
It also sometimes picks convoluted ways of doing things, so regularly asking whether there’s a simpler way of doing things can be useful.
GPT3.5 is 4k tokens and has a 16k version GP4 is 8k and has a 32k version.
You are correct that this needs to account for both input and output. I suspect that when you feed chat gpt longer it prompts, it may try to use the 16k / 32k models when it makes sense.
For features that probably should exist but don't it does a really good job of sending you on a wild goose chase.
GPT-4 reduces hallucinations by at least an order of magnitude, and hasn't failed me yet.
In that case they become complications.
You really need to be quite competent in the thing you're asking it to do in order to ferret out the hallucinations, which greatly diminishes the potency of GPT in the hands of someone who has no knowledge of the relevant language/runtime/problem domain/etc.
She asked gpt to help get an html version since apparently she got stuck with the wysiwg editor.
However gpt gave back a full html structure, including head and body. Pasting that into listmonk breaks entire webpage. Then she freaked out and told me listmonk sucks :)
But no, you're fundamentally right. It just goes to the question of whether an LLM assistant can in any sense replace or displace human programmers, or save time for human programmers. The answer seems to be somewhat, and in certain cases, but not much else.
If I already know the technology I'm querying GPT about, I'm going to spend at least some time identifying its hallucinations or realising that it introduced some. I might have been better off just doing it myself. If I don't know the technology I'm querying GPT about, I'm going to be impacted by its hallucinations but will also have to spend time figuring out what the hallucinations are and why this unfamiliar code sample doesn't work.
1) It could use the JSONformer idea [0] where we have a model of the language which determines what are the valid next tokens; we only ask it to supply a token when the language model gives us a choice, and when considering possible next tokens, we immediately ignore any which are invalid given the model. This could go beyond mere syntax to actually considering the APIs/etc which exist, so if the LLM has already generated tokens "import java.util.", then it could only generate a completion which was a public class (or subpackage) of "java.util.". Maybe something like language servers could help here.
2) Every output it generates, automatically compile and test it before showing it to the user. If compile/test fails, give it a chance to fix its mistake. If it gets stuck in a loop, or isn't getting anywhere after several attempts, fall back to next most likely output, and repeat. If after a while we still aren't getting anywhere, it can show the user its attempts (in case they give the user any idea).
[0] https://github.com/1rgs/jsonformer
It should suggest, lint the suggestion in the background, and if it passes offer the suggestion and if not provide the linting issues output to rework the suggestion.
In general, token costs going down will in turn increase the number of multi-pass generation systems over single-pass systems, which is going to improve dramatically.
Combine all that with persistent memory storages that can provide in-context additional guidance around better working with your codebase and you, and it's going to be quite a different experience than it is today.
And at the current rate of advancement, that's maybe going to be how things will look within a year or two.
This makes a big difference, I'm making code writing stuff at the moment.
Injecting results from a language server while it's generating would be huge imo - same as giving humans autocomplete & hints.
Basically, these days before I dig into documentation I ask "How do I do X with Y framework in Language Z" and if it's pre-2021 tech it works amazingly well.
Its hard to say if it improves my productivity because I just wouldn’t have done those things
But for the overall applications I think its improved a lot because we can implement best practices more consistently and catch regressions due to the aforementioned unit tests and documentation
Notably, it's not GPT 3.5, it's 3.0, which is pretty stupid as far as the state of the art goes.
The upcoming Copilot X will be based on GPT 4, which has "sparks of AGI".
In my experience there is no comparison. GPT 3 is barely good enough for some trivial tab-complete tasks. GPT 4 can do quite complex tasks like generating documentation, useful tests, finding obscure bugs, etc...
Sadly, you’ve just described the majority of the developers I’ve had to work with recently.
Most have no agency, write boilerplate code with no creativity, need their hand held every step of they way, and won’t do anything they’re not explicitly ordered to do.
You probably work in an SV startup with a highly skilled workforce. Out there in the real world there are armies of low-skill H1Bs and outsourcers that will soon be replaced with automation.
It’s a recurring theme in economics. Outsource to low cost labour, insource with automation, repeat.
We need an AI that iteratively tweaks its own architecture (to recreate and surpass those modules which are necessary for human thought), and maps out hardware enhancements* to accommodate the new architecture.
*I seem to remember Google working on ML software that proposes new chip designs a few years ago
The market may adjust over the longer term, or it may just continue to be volatile as the rate of change accelerates. In that case, we can't fix the work market, and we instead have to address the need for people to feed themselves another way.
But even then, it's not 'replacing' you.
It's just going to let you spend less time on BS and more time on the things that are your maximal value contributions to a project.
If I had a dozen junior or mid level devs you could hand work off to, would that save you time? Would you kick back and not review what they were doing, particularly around business critical parts of the software?
The conversation around AI has become obscenely binary, pulling from (now obsolete) SciFi influences to cast it as humans vs machines.
But it's a false dichotomy. Collaborative efforts are almost certainly where this is going, and 100% human or 100% AI will both be significantly inferior to a mix of both.
The question is if generative AI is powerful enough to reduce the number of programmers needed to achieve a task, without creating enough opportunities to replace those programmers.
Before we are all replaced there could be a moment where demand for software engineers is 10x less.
For industrial applications in particular they need to be functional and operable, not shiny.
You mean like this?
https://github.com/mdn/yari/issues/9208
https://news.ycombinator.com/item?id=36542267
I've yet to see anything maintaining legacy apps, or generating line of business apps with requirements... even simple stuff, like departure needs to be before arrival, etc.
I do see a whole bunch of youtube videos about generating a whole codebase, but it's the kind of stuff that there's a hundred tutorials covering.
Let me know if you have a chance to try it out.
Here is a chat transcript [1] that illustrates how you can use aider to explore an existing git repo, understand it and then make changes. As another example I needed a new feature in the glow tool and was able to make a PR [2] for it, even though I don't know anything about that codebase or even how to write golang.
[0] https://github.com/paul-gauthier/aider
[1] https://aider.chat/examples/2048-game.html
[2] https://github.com/charmbracelet/glow/pull/502
Did you look at the PR?
I reviewed it before submitting it. While I would have struggled to write it myself, I was able to review it and conclude that it was sensible and unlikely to be risky.
Of course it could have bugs that I missed. But so could any code I write myself in any language.
Learning to code has become significantly easier because of ChatGPT, and many university students are already using it for learning. Not only can they let ChatGPT write boilerplate code, but they can also let ChatGPT write comments for code snippets they don't understand and explain unfamiliar syntax.
I wonder if coders can survive in a world where more and more people have coding skills. Edit: "majority" was not a good wording
Trying stuff over and over again in different variations using maybe different languages, dealing with all the errors the frustration and overcoming them.
I think that using chat gpt or similar llms to learn how to code is similar to using Midjorney to learn how to draw.
Don't get me wrong you might be able to produce results fast but taking shortcuts is not going to speed up understanding.
I personally am glad I learned to code without LLMs and think I would struggle with them. They let you get a lot done without understanding any of it, and then suddenly you hit a wall.
Also, I wonder how many people may choose not to learn to code in the first place, because they think it is about to be automated.
But seriously, what developers do most of their time is maintanace, they spend a day searching for a bug, just to write maybe one line of code to fix it.
I strongly suspect that it will. There are whole classes of bugs that occur because some work is boring 'not quite copy paste' work that devs just don't like doing, and don't pay any attention to when they're doing it. Linters and syntax highlighters already catch a ton of those issues before they make it to production, and GPT will make the rest much less likely to happen.
Maintenance is one area where GPT will also shine, because it's 'just' updating some code to do the same thing, so using the existing code as a set of tokens with a prompt like 'update this code to work with v2 of library X' will be extremely effective. It'll be like having something write a codemod for you.
The future is bright. We'll get a lot more productive stuff done, and spend a lot less time on boring grunt work.
If one doubts that all this can be done by an LLM, use a different LLM for each step. Use committees of LLMs that vote on proposals made by other LLMs.
I don't know, I feel like the sky's the limit, especially if they can be made significantly more power efficient. I think that if they never get any better than they are now, and they just get more power efficient, they'll be useful for almost anything.
> unless governmentally mandated, they could keep guardrails from regular folks accessing it
That's not what's going to happen. Government will mandate that regular folks can't access it. Government will also do its best to make sure LLMs concentrate at a few companies, which it will often refer to as "partners."
or simply “GTP10, use all resources at your disposal to give me as much material and political power as possible. protect me at all costs, even the wellbeing if others.”
it might seem silly but GTP4 would seem silly to someone in 2016. this is more concentrated power than will have ever existed before. its evil and wrong.
But what about the next generation of devs and engineers - where do we source the senior engineers replacing us when 90+% of all entry-level and junior positions which actually involve writing repetitive boilerplate to a large extent are gone, and the few remaining are offshored and outsourced?
Many (most?) of us did a lot of automatable work in order to get the experience required to be able to proficiently actually automate the work, including managing LLM-generated code. If we replace our juniors with machines, we won't have many seniors down the road.
Junior devs lack experience, not intelligence. It's fine to give them difficult problems, as long as they're supervised.
I've worked with brilliant junior devs, sure, the code they wrote wasn't terribly idiomatic or maintainable, there were style issues, typical gotchas a more experienced programmer would be aware of etc., but it's not like they were fundamentally unable to solve a hard problem.
I think the Coder's role will become a very niche market, highly expert/specialist. the Engineer's will grow, very much needing AI to help-out, especially with tasks around: Discovery, Mapping/Relating, Projecting/Simulating.
Normally with the ChatGPT API you just feed API information or examples into the prompt. One version of GPT-4 has 32k context. The other has 8k and 3.5 has 16k now. So you can give it a lot of useful information and make it work quite a lot better for some specific task. When you pick something like React or Spring in general, depending on what you mean that might be huge amount of info to keep them current on. But if you narrow it down to a few modules then you can give them the latest API info etc.
Another option is now to feed ChatGPT a list of functions it can call with the arguments. It generally won't screw the actual function call part up, even with 3.5.
ChatGPT Plugins you can give an OpenAPI spec.
Then you implement the functions/API you give it. So they could be a wrapper for an existing library.
I think this is why it will be a long time before the general masses will be able to take advantage of AI to solve general problems. Most people haven't built up a human skill level of being able to explain their problem in a clear way to another human.
Imagine if you have no other context about the problem below other than these 2 prompts. Both of them are describing the same problem which is related to entering in orders with a point of sale system. Assume that you're talking to a human doing phone support for the company that provided you the hardware:
- My orders aren't coming up at the register
- I have 2 devices to take orders, when I manually place orders into the one hanging on the wall (ID: "Wall") it doesn't show up in the list of orders at the register (ID: "Register") but when I manually place an order at the register it does sync up at the wall
The first prompt is typically what a non-technical business owner may say over the phone when trying to get support. The second prompt is what someone who has experience describing problems might say even if they have no experience with the hardware other than spending 2 minutes identifying what each device is and chatting with the business owner to understand the real root problem is one of the devices isn't pushing its orders to the other device.
The 2nd one could become more precise too, but the context here is you're speaking with another human who works for the company that provides you the hardware and service so there's a lot of information you can expect they have on hand which can be left unsaid. They also have various technical specs about each device since they know your account.
It would take many follow up questions from a human to get the same information if you only provided the first question. I wish a general AI tool good luck to extract that information out when the direct person with the problem can barely type on their phone and doesn't have a laptop or personal computer.
Learned absolutely loads - far more than sitting down with a book and trying to learn from that. Not least because I’ve tried before and quickly lost interest.
Instead I’ve learned the basics and made a working web app, which I’m pretty pleased with.
That is why when Rausis(https://www.chessgames.com/perl/chessplayer?pid=14248) rating was approaching 2700 in his 50s everyone was very suspicious.
It is partially due to the incredible levels of stamina required to stay on top of the game for 4-6 hours.
Also as Kramnik said when he was retiring, you start making strange(read wrong) decisions suddenly.
Anyone with a bit of experience to know the right questions to ask can now code in any language or platform.
PHP's str_getcsv is, of course, a proper CSV parser and not a string splitter. Unless your code uses basically zero stdlib API calls, you will have to double check everything.
Please note that this kind of bug isn't even easy to catch if you test CSV file doesn't contain a quoted entry.
What are the good resources to learn about image editing AI tools, prompts and techniques?
My understanding is pretty limited, and correct me if I'm wrong, but like one would be using Stable Diffusion or Midjourney, and for a "professional" tool - Photoshop with official AI plug-ins?
There are a ton of people looking for help with generative AI and you can be useful if you just play around with it for a few weeks, because a lot of them have no idea about the basics. If you are willing to be underpaid there is no need to be unemployed -- just spend a few weeks studying and then go on Upwork.
Human programmers aren't going anywhere. (You can't even call what LLMs do programming, because there's no intent or understanding behind it.)
Learn about things you missed. double check everything it says.
Good luck.
If you're doing industrial embedded work and have an oscilloscope and a logic analyzer on your desk, and spend part of your time going into the plant and working directly with the machinery, you're in better shape.
This differentiating factor is what will wear out a less-experienced LLM user. They will make bigger claims or set expectations higher, and suffer more for them. The details that matter, yet were missed, will stick out more and more, as more experienced LLM users flex that experiential factor in a variety of ways.
For this reason, front end will absolutely still be a thing. And it'll be a much better, deeper thing, thanks to those who are a good fit for a kind of LLM-coding mindset.
However, this also depends on the type of coder. You can start from interpretation of the project spec as a logical code of sorts, or you can start from the spec as more of a visualized outcome.
If you work in the latter style, your survival key, so to speak, may simply be stringing together support requests you make to various LLM-interfacing vendors. A COTS-integrative style / opportunistic approach to coding, which has always been a thing.
Along the way, this kind of person usually integrates the NIH logical style a bit, and vice-versa, or they'll suffer through their respective blind spots. Same story, new layer of abstraction that's really cool.
(Plus...survival may still depend on who you know, not what you know, for a lot of people)
Jesus, these things really are coming for my job.
I instruct it what to do, and it writes the code.
Is anyone else brave enough to admit it?
The notion that I am generating and committing large blocks of untested arbitrary code makes me feel like you don't know how development is done. You're too far from reality for me to have confidence that you're at the professional level.
Maybe the AI can't do your job, but maybe it can do the job of many people who would have the skills to do your job.
If this happens and the new capabilities don't create enough new jobs, then the risk to software engineers is supply and demand.
Oh yes. I should be very very afraid of the flying copypasta monster. As if my productivity is reduced to the mere rate at which I can write code! What's even project planning? Why even have meetings if it's all down to "is it done yet"? Who works at these coding sweatshops that are so afraid of AI? If they get fired and find a better place to work, that's a win.