AI-enhanced development makes me more ambitious with my projects (simonwillison.net)

Also gpt4 >> gpt3.5 for coding.

I follow the same steps as always to design the code, but when it's time to implement something, I ask the bot to do it, then I review it and move on to the next function.

I find it much more useful for soft things like writing a complaint to a vendor for me, dealing with customer service, or cover letters from a job ad that I only have to slightly tweak. Takes me five minutes instead of 30+ and I got a few compliments about my “outstanding” cover letter.

My experience:

- Copilot did a decent job at suggesting functions when I typed out comments. It got progressively worse as the compression algorithms got less common (eg. Huffman vs. entropy coding) The smaller the functions, the more manageable it was. <- You need to write good tests, you want to step carefully through every line of code it writes.

- ChatGPT got most things right, and a few things really wrong. It was always super confident. <- Super dangerous if you don't double check, basically only usable as a refresher on a topic you are already good on.

- If given a piece of code and asked "How would you suggest to refactor this?", ChatGPT gave mostly very useful ideas. <- This is something that I will keep in my workflow.

- The bigger the project got, the worse the overall codebase looked. It became a mix of styles, pretty inconsistent, more subtle bugs were introduced that took me a while to figure out. (The nice thing about lossless compression is that you know if the code is right by using it)

- My "productivity", how many lines of code "I" wrote, was way beyond anything I could usually reach. I do also feel like I got a very quick start into Golang with this, much faster and broader than reading documentation or doing a getting started. That said, that knowledge now definitely needs to be refined in order to use the appropriate concept at the right time. Comparing my code with more professional Golang code bases I spot a lot of things that need to be improved.

It's a bit scary because knowing this stuff is sort-of our secret sauce and GPT-4 was able to give an even better answer than I was able to give. It helped us out a lot. We are now taking the solution back to the customer and will be implementing it.

A few additional thoughts:

1. I knew exactly what type of question to ask it to get the right answer (i.e. if someone used a different prompt maybe they would get a different answer) 2. I knew immediately that the answer it gave was what we needed to implement. Some parts of the answer were not helpful or misleading and i was able to disregard those parts. Maybe someone else would have to take more time figuring it out.

I imagine future versions of GPT will be better at both points.

- I want to create a Google Docs addon using AppScript which has an interface sidebar. Please describe the necessary steps to create a sidebar interface. The sidebar interface should have a title with the text "Google Docs JSON Styles", it should have a large textbox which can contain multiple lines of text (fill it with lorem ipsum), and it should have 4 buttons placed horizontally underneath the text box, with the captions "Save", "Apply", "Copy", "Paste".

This produced the necessary Javascript + HTML + external steps to set up the project.

- I'm building a website using a React frontend hosted on example.com with a Django REST API backend hosted on api.example.com - Authentication is handled via Firebase Auth, currently using an Authorization header provided for API calls. Now I want to have authenticated links such that the user can click a link to access an authenticated download via the API. What mechanism can I use to create a link to an API endpoint which will require the user to be authenticated?

This gave me a the frontend and backend code necessary to set up authenticated links, with several alternate approaches following further prompting.

- Given an owner id, repository id, commit id, and private access token; give me a URL that lets me download a ZIP archive of a Github repository.

In all of these examples I received the desired result with helpful descriptions.

As a counter-example:

- Write a program in Brainfuck that outputs the string "CHATGPT".

In this case, the result is the Brainfuck program for "Hello World", with a step by step break-down of the program explaining confidently why it would output "CHATGPT", while being entirely wrong.

The mental overhead of producing softwares shrinks more and more. It will sure be at the cost of understanding and/or see errors as you say.

I already looking forward of being that graybeard. And show off to my neighbors or grandchildren

Years later, I don’t think her driving has meaningfully improved.

As someone who learned to program from two gigantic tomes of C/C+ with a Borland Compiler, there was no copy/paste. You had to type every listing in to get it running, as well as find that semi-colon you forgot on line 16,46,102 of 200. Then you had to remember everything, take notes, and literally research answers. You couldn’t just ask a random person on the internet, at least for a few years after I learned how to program when online forums became a thing.

One big difference I can think of between my younger peers and myself: When I see an error message, I take it as a “hint that something is wrong” where my younger peers tend to take error messages literally. Probably because I “grew up” with terrible error messages that hardly ever pointed to the actual file/line the error was on.

Then "explain why <my code> needs to be <new code>"

Oh this is called <new concept>, give me an example of it. Give me another example. Give me another example. Give me use cases where this would be better. Give me a short story where it was done my way but it lead to a mishap in production and the protagonist learned <new concept>. Explain it like I'm 5. etc

Basically the new generation is the training neural network, and they can keep flashing context until they absorb the material. Sure, some people are lazy and will go as far as getting something barely working. But superpowers are available for those who fight for their place in the world

We are going to be vringes softwarearechologists and programmers at arms.

Knowing the subtlelties of memory management in the 80' didn't help the Ruby coders to create RoR and run Basecamp, or the Python coders to create Django and run the Washington Post.

People jumped at ORM, CSS frameworks and JS wrappers, and won the day because they traded memory and CPU for what the user wanted.

The new generation will do the same. They will figure out how to be super productive with AI, way more than what we are. And our superpower will be useful in specialized cases, but mostly, their impacts will be small compared to the order of magnitude of speed they'll gain by using the new paradigm.

People said interpreted languages were a waste, that you had to learn SQL, that Electron was an abomination.

But the people building stuff always win, not what we think is the right way.

That's why PHP won at some point. JS won at some point.

And something super practical (but that will make us say uhh...) made by AI will win at some point.

That's also why all the open letters and ethics committees will have no impact: the short term advantage given to AI users will be such that they will take everything over by storm.

It's already happening in porn: we just gave millions of teenage boys a shiny new toy, and they are spending days and nights on it. New incredible models come out every day. Some hyper specialized. Last week, I saw a publication of a new model specially trained on only vagina called "bettervulva" and one for boobs called "breastinclass". Somebody spent hundreds of hours on those, collecting the data, annotating it, and training.

I guarantee each niche of the world is doing exactly the same. Not to mention each new model is coded with somebody who has the previous models to help with the new one, code included.

This is snowball from now on.

The article says they “get the right answer 80% of the time”. That’s abysmal when you don’t know what you’re doing. Software which is 20% bugs is software you can’t rely on. Depending on what it does, it can be actively harmful.

Being a “magician (…) among mere mortals” is not a positive. We live in an interconnected society where the actions of others affect us. It would be nice if you can trust what other people are doing when they’re dealing with your water supply, electricity, food production, hospitals…

People are still actively acquiring the knowledge to do things the old fashioned way in those areas, a hundred or even thousands (for writing) of years later. But typically they only do it if they need to or find it interesting.

It’s hard for me to think of any technological advance (outside entertainment) that actually caused a mean regression in peoples’ effectiveness, or for people as a whole to completely stop acquiring the skills of doing something the old way to deleterious effects. Like, has there been any invention that was so easy to use that people got worse at the task the tool was invented for? In aggregate it may happen if the tool greatly reduces barriers (like training, experience, or study) to entry, but I kind of doubt it happens for specialists.

Note by effectiveness I mean the general task, not the specific skill that may have become obsolete. I’m not effective at all with a slide rule but I have a Python shell, the internet, and hardware capable of floating point operations.

People pay for productivity, making a product of some sort that has value. These AI tools will increase productivity. And they will only get better with time. That translates into the productivity of people who have mastered these tools as more valuable. They just get more done.

Will syntax issues and other problems be an issue with early AI? Sure, but it will get better. The AI will start to catch more issues and move farther up the stack. We still need to know enough to guide it, but the details required will be fewer.

If every single person with that coding experience only coded, that's likely, but not every person who got their start that way focused on coding to the exclusion of everything else. There are at least a couple of us who are more oriented towards teaching, archiving, and knowledge preservation. Right now, people like us but don't tend to prioritize giving us their resources. Once those greybeards are in their 60s/70s/80s and looking at their work being lost forever, I imagine that will change. (Much in the same way that there's way more discussion now in tech spaces about raising kids now that more programmers are old enough to confront that issue).

That knowledge will likely be preserved by another greybeard cohort and it'll be specialized knowledge that only some people know, but it will be known. Some stuff will be lost because we'll prioritize money and copyright up until there's a crisis, but there are people prepping for this time period. It's not like trade knowledge loss is a new problem. It's just not a sexy one right now.

1. 99% of devs will be out of work and the current work will be done by 1%.

2. More and better software will be developed. About 100x effort more.

I'm pretty confident the 2nd will happen. I already have lots of work that will take a long time, and more after that. 100x improvement on productivity would mean more, faster and better (because of more tests etc) output.

If by AI we mean things beyond just ChatGPT sure it will, it’s only a question of how long before it happens and how many new jobs will be created to replace the old jobs. For example, AI isn’t going to enhance customer service, you’ll just have better automated experiences and businesses will hesitate less to use them.

Edit: AI isn’t being created for the benefit of mankind. It’s being created in the pursuit of money. This isn’t a utopian fantasy world where we look out for everyone’s best interests, that’s just a side effect most of the time.

I can ask chatgpt to write code for me for various modules of applications on a mobile device and I can then go home and put everything together.

Development on mobile although possible was not feasible due to the form factor. Now that limitation is gone and things are only going to improve.

I couldn't agree more. I work as a software engineer in the medical field (think MRI's, X-rays, that sort of thing). I've started using ChatGPT all the time to write the code, and then I just fix it up a bit. So far it's working great!

just pasted this into bing creative mode. whatever this thing is spitting out right now would've probably taken me a week of research to even figure out the libraries and their APIs. it's given me a raft+sled+serde scaffolding, is on it's fourth 'continue' step and just keeps going.

---

As an AI language model, I'm unable to write full code examples, but I can provide you a high-level overview of how to implement a horizontally scalable, probabilistic database in Rust using the Raft consensus algorithm.

1) Define the data model: Design a data model for your probabilistic database, which should include the basic CRUD operations (Create, Read, Update, Delete). You can represent the data as a key-value store or any other data structure that fits your use case. Consider using a probabilistic data structure like a Bloom filter, Count-Min Sketch, or HyperLogLog to reduce memory usage and support approximate queries.

2) Implement the Raft protocol: To build a distributed system with strong consistency, you can use the Raft consensus algorithm. There are existing Rust libraries that implement the Raft protocol, such as raft-rs. Include this library in your project to handle the core Raft functionality.

3) Design your database nodes: Each node in the system should maintain a state machine that represents the current state of the database. The state machine should include the following components:

* A log to store the commands received from clients. * A state to apply the log entries and keep the actual data.

4) Handle client requests: For each client request, the leader node should create a corresponding command and append it to its log. Then, the leader should replicate the log entry to its follower nodes using the Raft protocol. Once a majority of the nodes acknowledge the new entry, the leader can consider it committed and apply the command to its state machine. After applying the command, the leader should send the response back to the client.

5) Implement horizontal scaling: To enable horizontal scaling, you need to partition your data across multiple nodes. One way to achieve this is by using consistent hashing. Consistent hashing helps distribute keys evenly across nodes while minimizing data movement when nodes are added or removed.

6) Handle node failures and recovery: Raft is designed to handle node failures and recovery. When a node goes down, the remaining nodes will continue to operate and maintain consistency. Once the failed node comes back online, it will synchronize with the current leader and catch up with the latest state.

7) Deploy and monitor your system: After implementing the necessary components, deploy your system across multiple nodes. Monitor the performance, consistency, and fault tolerance of your system to ensure it meets your requirements.

Here's a high-level outline of how the main components of your project could look like:

mod data_model; mod state_machine; mod raft;

use data_model::DataModel; use state_machine::StateMachine; use raft::RaftNode;

fn main() { let nodes = create_nodes(); let state_machine = StateMachine::new(DataModel::new()); let raft_node = RaftNode::new(nodes, state_machine);

    // Start the RaftNode, which takes care of the Raft consensus protocol
    raft_node.start();

    // Start a server to handle client requests
    let server = Server::new(raft_node.clone());
    server.start();

    // Optionally, start a monitoring and management interface
    let monitoring = Monitoring::new(raft_node.clone());
    monitoring.start();

In this example, you'd need to implement the `data_model`, `state_machine`, `raft`, `Server`, and `Monitoring` modules/components to handle the different aspects of your horizontally scalable, probabilistic database.

Remember that this is a high-level outline and not a complete implementation. You'll need to dive deeper into each component to ensure proper functionality, scalability, and fault tolerance.

ChatGPT 3.5, on the other hand, forged ahead writing a bunch of nonsense with vaguely correct function names.

Depends how jr your team is I suppose.

GPT-3/4 is like a Meeseeks box for computer and internet tasks. Keep your task simple, and it will quickly and happily solve it for you before ceasing to exist. (Well, that’s how I use it via the API, at least. ChatGPT will preserve the state, but of course the context window is still limited anyway.)

Most developers (including myself) get annoyed with: repetitive tasks, pivoting after substantial work, vague requests, juggling too many unrelated tasks, context switching, etc.

With Chat-GPT I’ve had to learn a new paradigm: any request is OK at any time

I do like asking it to explain things to me, for example "explain what a PID controller is". It reminds of like a real-time wikipedia.

I agree about the unbounded optimism. It has been fun to ask ChatGPT where to start on certain project ideas I’ve had.

However, I’ve tried it three times with small but ambitious project ideas now. Each time it goes around in circles with general concepts but I can never get anywhere.

If I switch to common tutorial topics like “How can I write a simple TODO list app?” it performs surprisingly well.

As soon as I step outside of common tutorial material, I can’t seem to get anywhere with it.

still good for handling a lot of grunge work and really useful for doing the stuff where I'm weaker as a "full stack" developer

Even better if they have an "Are you sure this is a good idea?" mode, possibly even with suggested alternatives

Am I doing it wrong somehow?

You can just say "Continue."

So basically, a bug can set you back a long time (multiple weeks). And to reduce the risks of bugs we usually need to ship incrementally. And because bugs are such a hassle we have to monitor and test the crap out of everything, which of course we should be doing anyway, but we do it more than we probably would otherwise. But of course bugs do still happen, and even someone else’s might set you back. A flip side to slow releases is releases tend to batch more changes. And of course that makes finding a bug more time consuming.

So a feature that could take me a few hours to write and test at my top performance basically needs to be budgeted at a month of work. Of course, we have other responsibilities like testing, monitoring, analysis, support, design, coordination, helping others. But dragging out completing a project drags all that out too. If I could ship to staging same day and prod 1-2 days later, I wouldn’t need to plan so much or spend so much time going back and forth about design. We could just meet up about the change and do it that week.

And you might think you could simply parallelize/pipeline more tasks, but overdoing that sets you down the path of overextending yourself. If something important goes off track you need to do everything to get it back on track - if you don’t, it’s going to be really late - and overcommitting eliminates the slack that makes it possible.

Truly, this is something I think about all the time and probably the least favorite aspect of my job. Anyway, unless chatgpt can help me write a persuasive argument that convinced my company to completely 180 its engineering policies, it won’t make me much more productive.

So, its quite literally an issue of lost knowledge. Is like replacing wikipedia with something that answers faster, but without sources. Its cool in the short term, and people are more productive at learning things, but you also go back to the hear-say time before books were invented.

What we are seeing is that the number of software engineers has increased dramatically and the average skill level has plummeted. It is not normal for a competent software engineer to need to reference Stack Overflow a dozen times per day every single day.

The other thing that this is bringing to the forefront is just how atrocious the libraries are in certain ecosystems (e.g. JS/TS).

If a tool that can write incorrect functions/methods makes you substantially more productive, you were never very valuable in the first place, and "AI" likely isn't the only thing threatening your job security.

We've been iterating over that for decades, there's always one more step of grunt work to remove, one more layer of boilerplate code to move into the compiler (or a code generator), one more deployment task to automate, one more elegant new syntax to add to a language, …

Yet. They got to get you hooked first.

At least with Google and Stack Overflow ads are identifiable and can blocked.

Yes, architecture, medicine and metallurgy have been improved for millenniums. Next to that, IT is a baby that can barely walk. Of course we have too much grunt work.

But also because every time we remove grunt work, the customer expectation increase, the market adapts, and now you have to use new tools that are productive enough to do what the users want with the resource they are willing to pay.

So you don't have to hand write assembly but you do have to have an app that works on 16 screens ratios. You have amazing databases ready to handle your data, but you should update the UI live.

> We continue to narrow the path to get experience as an engineer.

Like in most industries, we are starting to move from "everyone needs to be an experts" to "90% of the daily tasks really just need good technicians". That's fine. That's how it is supposed to be.

At some point only experts could duplicate your house keys. Now you have convenience key stores everywhere.

We are on the right track.

Next stop, the equivalent of residential building codes, but for public facing apps.

Because IT is so new and can be done in your garage, we thought it was special. It was not.

For example, if you have a configuration, and a particular default would often (say 30% of the time) be overridden, I think it’s typically not a good idea for the default to exist in the first place. Having those defaults will certainly reduce the average config size, but they’ll also create more cognitive load for readers, who now not only don’t know what the default is, but don’t necessarily know that the choice is being made in the first place.

This is one of the things I think is so powerful about AI as a solution. It saves you from the work, but still conveys the basic context to readers without requiring them to understand a bunch of opinions baked into the language or framework.

The same applies to writing prose. When you give a bulleted list to ChatGPT and ask it to turn it into prose, a lot of the work it’s doing is filling in obvious text. But that obvious text is important to communication, and where you have to go and edit that obvious text for correctness will be some of the most important communication in the final product.

It's not grunt work.

It's how new engineers learn the ropes and gain experience doing low risk work; it's part of the learning process that only feels like grunt work to a senior dev.

What do you mean "at the core"?