We’re living in interesting times, especially from AI, ML and coding point of view. The possibilities seem endless and the reach of such tools is going to lead to an “explosion” of creativity.
I would suggest that the past is likely a good guide in this case. Personal Computers, easier languages, HTML, World Wide Web did not really make non-creative people into creative people. I can really speak for myself. I can copy some things, but I rarely can glue and remix them together in a way that make them desirable. Personally, this is the same way I see the AI. There is potential here sure, but I do not think 2 million angry bird clones is an actual threat to a creative. Now, it will hurt low effort crap, but there is so much low effort crap out there already..
I agree with you with one small reservation. This tech is an excellent way to get quick prototypes out. Most people will never go beyond that so we will see a lot more crap. But for the people that do, being able to prototype something quickly could result in some noticeably new ideas.
It is interesting to watch. Especially as an arch in "my life with technology", this time around I'm a bystander watching what people can do with it and it is impressive. My tinkering as not been as fruitful. I feel like my grandparents that never caught on to Google-fu but, just like my grandparents, I'm ok with it.
Think it could be an interesting UX pattern. Having interactive loading (spinner) games that at least give is feedback that our actions (even in between things) have impact.
It is an interesting approach to loading screens, and personally I would have expected way more games to use such a feature. Not AAAs, of course, but indie games.
Willy Beamish for the Sega CD had something similar, too. It was a godsend because that game's scenes took forever to load...I've only been able to revisit it thanks to flash carts that mostly eliminate the loading speed.
This is more interesting than the deluge of posts that say "I created an iOS app in 30 minutes using ChatGPT!" Which doesn't mean much because it could've done nothing more than create a simple hello world.
This one at least shows the finished product, which is indeed pretty impressive.
Some details I'd need to know are (a) how long did it take, (b) how many prompts, (c) how many course-corrections were required, and (d) how competent this individual was with the technologies in question.
I've personally found ChatGPT extremely empowering in lots of scenarios, but code generation was not among them.
I’ve been playing with ChatGPT code generation to make entire sites with flask, python, html+js+css, backed with SQLite db and it’s amazing. I’ve had it write like 5k lines that are all live in prod and working (not much traffic lol but still).
A huge huge factor is knowing the limitations and getting better at prompting. And identifying likely hallucinations and asking for risks etc.
I’ve found it best with tech I don’t know well (I’m an android dev using it to make websites, something I haven’t done myself in like 15 years).
Most of the coolest stuff for me is help with sysadmin and running the server. The ability to debug gunicorn errors is great.
I do have to modify the code it outputs as the project grows and it loses context, but honestly the context limits are the biggest hurdle for bigger projects and those will be lifted soon.
Edit: Most recent site I made with like 95% code from ChatGPT is https://cosmictrip.space/ which generates prompts with GPT-4 that are then used to generate space images with DALL-E.
It's a simple site but there is a secret adventure game I'm working on (GPT+Dall-E) that is open-ended image+text AI-driven game. I'm hoping to launch before Nov 6 with DALL-E 3 API (hopefully...!). The adventure game is also written like 95%+ by ChatGPT.
I've had such great success with it coding that I'm using the GPT-4 API with an agent I'm making (everyone is huh). I have function calling hooked up to generate structured subtasks that the agent can then write the code for, and support for files to include context, chat with your code, etc. It's not ready to show but the GPT-4 code generation abilities are really incredible - but you have to be experienced at prompting. Your first prompts aren't likely to be great, which is why I'm hoping my agent can have success. The idea of the agent I'm writing is a Jira/kanban style board where you have AI coders assigned to tasks that you can approve and modify etc. The tickets should automatically move across the columns as the AI checks the work etc.
+1 for its suitability in helping with systems administration.
One responsibility at my current job is administering a Windows server and trying to get it to do things that are easy on a Unix -- that should be easy anywhere -- but, on Windows, seem to inevitably degrade into nightmares. ChadGPT has given me huge amounts of blessed ammo to shoot at the nightmares, and there's no way I could do that portion of the job in a feasible time frame without it.
I feel that trapping your AI agents in a kanban board isn't going to do your survival chances a lot of good when the robot apocalypse inevitably comes for us meatbags.
That is a great point and I will definitely share my prompting experience and some real prompts in a blog post this week. I'll come back here and link to it when ready.
adventure game I'm working on (GPT+Dall-E) that is open-ended image+text AI-driven game. I'm hoping to launch before Nov 6 with DALL-E 3 API.
Some people have hooked AI dungeon / koboldAI up to stable diffusion to generate these kinds of procedural Ender's game style interactive graphical text adventures with varying degrees of success.
If your game is going to be similar, you'd better get in the habit of aggressively caching the generated imagery for it on S3 because no way the DALL-E 3 API is going to be cheap.
You are right about the context window limitation.
I exclusively use Azure OpenAI
GPT-4 32k version and it's been a game changer when coding on complex projects.
> Although the game is just 600 lines of which I haven't written ANY, [coding the game] was the most challenging part
Not quite hello world, but not too much more difficult than a shopping list. The really impressive thing to me is you can make angry birds with just 600 loc (and a couple libraries)
My guess is that the main parts of the game are physics (collisions etc) and the scoring system, so that part wasn't too surprising to me.
I was pleasantly surprised at the visual quality, I knew Midjourney could produce quality graphics assets, but I guess I didn't realize how easy it was to pull into a game.
Not only are there tons of Angry Birds clones (Angry Birds itself is kind of a clone of earlier games), there are also tons of step-by-step tutorials for making them, which were no doubt included in the training data.
GPT4 is great at this stuff, but iterative refinement doesn’t work in my experience.
As the conversation increases, the previous context is lost and the generated code deviates from its previous behaviour.
For example, “fix this bug…” can easily result in a solution that breaks some other thing. You can also see code generated in thread (1) that does exist in the final result (2), suggesting that (since this is the very top of the code), they were getting chatGPT to iteratively generate 600+ line segments.
I severely doubt this.
Creating a new Slingshot on line 20 after it is defined on line 500? That is extraordinarily unlikely unless you specifically prompted it to do that.
“loadImage('stone2.png');”, it just happened to pick the right file names? The right sprite sizes? You provided all that in a prompt and it wrote the code? Come onnnn… show us the actual prompt you used.
It seems much more likely they generated a set of class objects relatively independently, then manually assembled them into a larger file, copied the entire thing as input and then crafted a “code prompt” like “write a function that does such and such”.
It’s not impossible they used prompts like they claim (3), but I feel they are (for likes and cred) vastly overstating the “it did all the coding” part of this project.
I feel they probably hand wrote some of the code (or assembled it) and used it as input + a “now do this also” style prompt, so the output was “100% generated”, but not in the way people are assuming.
This approach tends to make GPT4 rewrite the existing code, but unless you specifically ask for (or add) comments describing the intent through out the code (missing in most of the generated code), it will drift from the previous functionality. With no test suite to verify, you won’t notice this subtle drift and things just break. There’s no mention of either of these things being done by the author.
Further more, this user has a vested interest (4) in selling training materials for AI, so it’s in their interest to appear to be an expert at this, and has provided (even when asked on X) no additional details, no “step by step” git repo with history, no actual prompts they’ve used.
Given the lack of details and the frankly unbelievable results, I think there’s fair call to be sceptical in this case.
You could generate this kind of thing from models such as codellama 34B, or GPT 3.5; but not using the method as described.
I’m… not convinced you could do it with gpt4. The prompts seem too stupid to be real (5)… but I happy to be proved wrong with more details. GPT4 is good.
Context is 8k and it's quadratic. It "sees" everything in that window. If you want to have a long conversation try Claude or some of the 32k models. Claude uses a strange kind of attention that isn't always as precise but it's very good at finding key information in huge documents.
You are welcome to be skeptical and I will happily continue to use these tools for drafting code and writing every day. I usually provide documentation for the systems I want to use on input. Then I include source code of anything I'm hacking on. If none exists I build it up using "let's do this step by step" where the first step is outlining or architecture and subsequent steps fill out components. The same pattern works for long form prose. Having 100k tokens is almost enough. It means the model can attend to a ton of relevant information.
Also, this works well when the system gives you patches. I wouldn't say "rewrite this entire thing to make this one change" because if it's anything non trivial it will shift. Again I see the same patterns for code and text. Ask for small changes, contained updates. Guide the model to work in small increments, using abstraction to deal with high level broad topics.
You wouldn’t use it refactor because the implementation will drift, but you’d use it to generate a patch to apply to the code?
Hm.
Well, I don’t think that style is relevant or practical for large scale code generation as shown even in this ~600 line example.
…but, I’d love to see the details of how it could done, end to end, by someone who has done something like that successfully and is willing to prove it by sharing the prompts and outputs.
It's very hard to ask one of these systems to copy an entire piece of text and then change one or two things in it. They will often make small errors. So it's easier to ask for patches and apply them manually to edit the code or text that's being worked on. Happy to talk more how do I find you?
> It's apparently pretty good at annoying people on HN who like to downplay its every achievement
It's not really "its" achievement. There are many open source repositories of Flappy Bird that it was trained on and there are many brilliant engineers at Google and OpenAI who have worked on LLM technology. It is their achievement if anything. It is a product. A product of others' achievement. You don't say Excel had an achievement when it calculates a good formula for you? Do you say a neural network that produces great speech to text had an achievement? Or do say the people who developed it did.
>Do you say a neural network that produces great speech to text had an achievement? Or do say the people who developed it did.
Sure Why not? The engineers at google and open ai or wherever else trained the models but they didn't teach it how to do anything it does because they don't know how to teach it to do anything it does. So yeah the achievement is on the neural network.
Many people gave the likes of alpha go achievement on super human Go play.
In terms of the code, I don't think it would be hard to create an Angry Bird clone of this quality using a GPT-4 scale model that had somehow had all knowledge of Angry Birds excluded from it.
Some of them make no mention of Angry Birds - "Wooden box. Large skeleton bone. Item assets sprites. White background. In-game sprites" - but others do: "Item assets sprites. Wooden planks. White background. In-game sprites. Similar to Angry Birds style"
My hunch is that a skilled image prompter could still get to results that were right for this particular demo even with a model that had not seen Angry Birds assets before.
We have exactly the same argument on a regular basis about whether or not we should say Musk is responsible for the success of Tesla; likewise with attributing the achievements of the USA[0] (or even just the government of the USA[0]) to the specific president of the time rather than to the nation as a whole.
And before Musk was born, similar debates about describing Armstrong as the first man on the moon, given how even saying that inherently brushes aside the contributions of all those who worked on the mission without going to space.
[0] In the UK it seems to be half on the Prime Minister and half the party of the age; I don't pay enough attention to the local opinion of politics elsewhere, though I will say the British press seems to blame everything Germany does on the German Chancellor personally…
Show all prompts step-by-step and let people reproduce.
Someone claims they did X using a couple of AI instruments, without sharing anything to prove the claim, and everyone is excited. I understand that the industry needs a new hype to keep going, but this is just pathetic.
>Graphics don't matter, let sprites be colored rectangles.
Why? Are the sprites and backgrounds too easy to recreate with MJ and Dall-e after you realized that the author actually provided the prompts for them?
Come on... I'm fairly cynical when it comes to AI but someone building a pretty complete game from scratch entirely using AI to generate the code and assets is a little beyond it 'making a cheap copy of something else'.
If you fed a competent human massive quantities of human-created code and artwork, they could fairly quickly mash it up and produce a clone of a simple game, too.
Have we created real AI, or merely 'AP' - Auto-Plagiarism?
Mobile games are a mix of half-arsed Steam game ports, polished ad-budget-driven exploitation machines and trash. I think the trash producers will be excited, not scared. The other two kinds won't care.
Everyone will think this impressive, but they’re not game developers.
You could literally make all this in probably 24 hours and not spend any time mucking around with prompts.
Like what are we even seeing here? Basically a tech demo of a physics engine, a little UI interaction for throwing a collidable entity into other entities, and some code for setting up a level?
Show me the maintenance, adding new features, bug fixing, cross platform compatibility, shaders, networking code, sound, etc.
You could argue then that this reduces the barrier to entry for people to become game developers.
Isn’t everything you just listed (physics engine, UI, collide entities) how you make a simple game? It doesn’t matter that it doesn’t have networking code, shaders, or sound. Those things can be added later.
I don’t think it’s productive to gatekeep who is and isn’t a game developer. They developed a game, that makes them a game developer—it doesn’t matter how they got there.
I'm not sure how this reduces the barrier to game developement. There are already lots of free assets and game engines designed for making arcade games that are a lot easier then say Unity or Unreal. Like https://arcade.makecode.com/ or https://microstudio.dev/ or https://scratch.mit.edu/.
And if you don't want to make arcade games there are other tools like RPG Maker, RenPy, GDevelop, and many more each of which are much easier to use then this AI pipeline (not to say it isn't impressive you can do this with AI though) and will lead to better outcomes of actually understanding game development.
All of those things require me to write code, in an annoying, slow, hard to debug click-drag format. I'd rather the AI write the code for me based on my natural language description of what I want.
My point is there isn’t a “correct” way to become a game developer— by your own response you showed that there are frameworks you can use, WYSIWYG editors, you can code a framework from scratch in a high- or low-level language, or you can work with an AI to make something that works.
This person successfully developed a game. Just because it isn’t how others would have done it doesn’t minimize the result.
I think some gatekeeping is warranted. If I stitch a wound it doesn’t suddenly mean I’m a doctor.
Similarly, using AI to develop a game makes you as much of a game developer as someone who hires someone to make a game for them. And some development studios are basically that, people hiring developers to make games. But AI can’t compete well with that.
It’s like coming up with a new fusion power plant, but it takes more energy to run than what it produces. Inspiring, but useless.
Angry Pumpkins did not exist. Now it does. In time for Halloween. You can't refute that.
Using a computer to write a letter makes you as much of a writer as someone how hites someone to write a letter for you. And some writers are basically that, hiring scribes to write for them. Computer can't compete well with that.
Yeah, being a doctor is a certifiable profession— there are tests, governing bodies, standardized curriculum— are you saying the same exists for “game developer”, a role which could mean a hobbyist, indie developer, or someone working at a huge studio?
This really sounds like “real programmers write in assembly” to me. An AI is a tool, not dissimilar from a higher level language. This person used a tool to create something that didn’t exist before. The result is right there.
Yeah the killer feature here is being able to generate premium-looking game assets without any art team or funding. That's always been the tough part of creating even simple games for solo programmers. The rest is whatever.
>ou could literally make all this in probably 24 hours and not spend any time mucking around with prompts.
I couldn’t make that art in 24 days.
The big thing about the state of current AI isn’t that it does a bunch of things, it’s that it drastically increases the number of people who can do things.
That is, I could not make this game in a day before AI. With it, maybe I could. I am far from unique.
> it’s that it drastically increases the number of people who can do things.
Giving the whole world blogspot.com 20 years ago didn't measurably improve the internet, it just added a few more winners and a gigantic increase in low-quality spam pollution.
Bloggers have broken news well in advance of large news outlets, partially because of the ability to publish online rapidly, without knowing HTML or administering a web server.
But it's mostly been used to pollute the internet with useless spam. With generative AI, the same thing will happen. The very example shown here is an Angry Birds clone, patient zero of the pay-to-win hellscape of mobile games. I don't think it's an accident that they had AI in their hands and decided to build that.
I doubt the asset store has a pumpkin in the style of Angry Birds that looks like Red, it doesn't seem a good alternative if he wanted to create this game.
Which imo summarizes what GPT4 is great at - bootstrapping devs who have a general idea of what they want, but are not familiar with the exact specifics of the technology, or need to freshen up their knowledge.
I've learned Kubernetes and Powershell this exact way.
Does 24 hours include learning how to use a game engine that you've never used before? When you've never used ANY game enginer before?
Maintenance and new features are more of the same already done. Networking code is a library. Sound is a library. Cross-platform is a library. Shaders are a library. AI fixed bugs.
> Everyone will think this impressive, but they’re not game developers.
I'm a game developer. It's impressive.
I'm extremely aware of how short a game jam demo is compared to an actual product. There's an ocean between the two.
That doesn't diminish how much easier and more approachable it suddenly is for someone with little experience to make a game jam demo, and with kinda-passing art.
But it wasn't used to generate the music in this "game" that isn't a game. This was my point - that it is unclear exactly what AI did and did not do in this case. Like, I'm 100% sure my mom could not have done this, so, not "all" AI like the title says.
I did that by building an open source VS Code extension to interact with GPT3.5/4 directly from your editor: https://marketplace.visualstudio.com/items?itemName=Biggles..... It cuts out copy/pasting between ChatGPT and your editor, or writing any boiler plate code since you can just ask it to insert the code you want, or change the code you have highlighted. You can also talk to it directly using the Whisper API!
I did a similar exercise recently when I needed to make a fairly basic rest API and CRUD frontend using 2 frameworks I wasn't particularly familiar with. I used GPT4 to generate ALL the code for it. I'll write a blog post about it soon, but a quick overview was:
I suspect it was slower than just writing the code/referencing the docs, and would be much slower than someone could do if they were experienced with the two frameworks. I had to be very specific and write a few long and detailed prompts for the more complex parts of a the application. It took around 5 hours to make the application, with a lot of that time spent sitting waiting for the (sometimes painfully slow) ChatGPT output. In a framework I'm more familiar with I think I could have easily got it done in under 2 hours
It was definitely useful for making sure I was doing it the correct way, kind of like have an expert on call for any questions. It was also very useful for generating perfectly formatted boilerplate code (some frameworks have CRUD generation built in, but this one did not).
It was a fun experiment, and I found it useful as a learning/guiding/generation tool, but I won't be using it for general day to day development any more than I currently do. For most instances it's quicker to just learn the framework well and write the code yourself.
> It was definitely useful for making sure I was doing it the correct way, kind of like have an expert on call for any questions.
I've found it to be shockingly good at this. I end up asking a lot of questions like, "what is the best directory structure for a project on {foo} platform?" or "What is the idiomatic way to do {x} in {language y}?" It has the advantage of having seen lots of projects in every language, and for some questions that automatically leads to a good answer.
I always get the classic
“It really depends on your use case and neither pattern is exactly better than the other” when asking gpt about programming patterns
it has such a completionist fetish, I have found it works much better when you basically tell it to pick a side and not go out of its way to be balanced.
Be careful and don't trust all it says. Sometimes it invents API functions which are not there, or doesn't see existing. And always very confident till you point it.
And then it’s like, “thank you for telling me about this, I’ll remember that for next time,” which is how a human ought to respond but not how ChatGPT actually learns.
It also resets inside a single conversation if you go beyond a certain number of tokens, and as far as I know there is no warning whatsoever to the user when it happens.
The obvious next thing to try would be to continuously fine tune the model on these conversations, so it actually fulfills the promises most of these models make about "learning continuously". I haven't yet seen any actual implementations of this, though. I'm sure someone's tried it, I wonder how it went.
I'm finding that code is the area where hallucinations matter the least... because if it hallucinates an API function that doesn't exist, the mistake becomes apparent the moment you actually try to run it.
It's like having an automated fact checker! I wish I had the same thing for the other kinds of output it produces.
It will, however from time to time insert lines and variables that do nothing, but could result in bugs or confusion if not removed. I’ve encountered these hallucinations a few times. Overall, I agree with your sentiment, but I think it’s important to note that running isn’t always the indicator or correctness we think it to be.
You're still supposed to read it, like you hopefully wouldn't blindly paste a big code block from SO. A useless/unused line or variable doesn't seem that hard to spot?
Of course, but ease of detection can vary relative to the complexity of the code being returned. GPT-4, correctly prompted, can produce some pretty complicated stuff. But it also hallucinates in ways that are more subtle than one might think. The example I’m thinking of, it created an unused variable in a set of fairly complex ML training set up scripts that I mostly caught because I was familiar with all the proper inputs. But the unused variable was quite plausible if you were not familiar, new to the domain etc.
Compilers automatically detect unused variables. Unused variables are the last of your problems. You should be far, far more worried about all the misused variables.
In this case “unused” might mean declaring or initializing a variable and then assigning or reassigning a value to it later. In this case, it is technically used, so most linters won’t pick that up. But it actually does nothing so it’s wasted cycles.
I've noticed some other funny things, maybe harmless but undesirable.
One example was really really hard to spot. Once I queried GPT 3.5 for a function to do X, and it did pretty well, when I looked closer though, it had wrapped 90% of the code in an unnecessary if statement. I looked at the code an thought something was off until I realized.
My point here is, if that was easy to spot, who knows what else people are missing because even in a simple case, unless you're actually trying to spot issues, you likely won't see them.
If it imagines a function it's fine. You can fix it in one prompt. But when it doesn't see it may move in the wrong way and produce limited solution. And you wouldn't know.
You would know in a similar way to a colleague of yours having done this code the previous day, or you joining a company with legacy code. It either has tests or you can't really trust it anyway regardless. You can ask the AI to also write tests for you, inspecting tests is usually much faster than understanding all the nuances of the code.
> because if it hallucinates an API function that doesn't exist
Yes, absolutely agree.
And also I’m no fanboy but when it does this and I only notice because the code doesn’t run, half the time I’m thinking to myself that the API function really should exist because it’s so logical!
I see it all the time. Even more annoying is when the API exists, but in a different class or with different parameters and outputs than you need, and the model would claim it does exactly what you need right until the time you try to use it and discover it can't work in your case.
Often I find that the hallucinations is how the API or lib should have been if it was more sane. Maybe someone could turn this into a virtual API critique.
Conversely, doing so has helped me flesh out my thoughts on many occasions. As I ran into obstacles with errors or imprecise prompting, I realized my design had issues or edge cases I hadn’t take into account. Perhaps it would be better if I wrote out several paragraphs describing my intentions before taking up most coding tasks, but I hardly think my boss would be in support of this!
My role unfortunately doesn’t allow for this. I do more “in the trenches” data science stuff. The value is quite obvious to me for a role where there is more space for this or in product planning.
That's a great fit for scenarios where you do know programming but don't know the particular language and framework on which you suddenly have to do some maintenance or improvement.
For many things the available documentation is poor, and asking a bot is much more helpful.
Yes exactly. I had to be very specific and tackle the project in the same way I would if I was fully writing the code. First data schema, then models, then controllers with CRUD and views, then routes, then authentication, then test cases, then TS transpiling, etc...
It's definitely not something someone with zero coding experience could easily do, and I feel even a junior developer would struggle with anything even as complex as this. You have to have the concept and structure in your head, it's just writing the code for it.
> You have to have the concept and structure in your head, it's just writing the code for it.
I wonder how far you'd get with the technique of asking ChatGPT to lay out its plans first, kind of like the improvements in math questions you see when you ask it to write down its reasoning before committing to any answer.
"This is what I'm looking for: XXX. What are the different pieces of this that I'm going to need to create?"
"Ok, the first thing on your list you gave me was a database to store the data. What would the structure of that database look like?"
"Can you give me the code to create that database?"
Etc etc. i.e putting the actual code as the last step each time.
This is the best form of prompting for generating code. You tell it to first generate a technical spec for solving the stated problem problem, consider multiple options, return for your review. You then use a trigger command like “build” to then implement, once you’ve specified any changes.
I suspect it would work surprisingly well! But still, I think you would have to have a fairly good concept of coding to do even that. For example, my mum who has zero concept of what a database is (beyond the dictionary definition) would not be able to piece together an application like this (sorry mum!). But a junior developer would probably be able to just about do it depending on complexity.
For me, using GPT felt very slow, but for a junior it might actually be faster than trial/error and Googling. Also, ChatGPT is only going to get better and better, so we can expect it to become quicker and easier to do such things.
What I think you're overlooking is that most people can only do a few hours of hardcore coding at peak productivity a day (3hours for me, maybe)
So you could spend 3hours babysitting GPT4 to write some code for you, but then you'd still have 3 hours of peak code productivity that day that you haven't "used up" yet
I’m the opposite, personally. I can code for 5 or 6 hours just fine if I’m “in the zone”, but I can’t deal with LLMs for more than an hour or two max, usually less. I find their sweet spot is when I need to ask one or two questions with one or two follow-ups, 5-10 minutes ideally. They can sometimes be a big win in small doses if you can keep it to that kind of interaction. But they are just draining to interact with for longer periods. For me they’re a lot like being a TA stuck in multi-hour office hours with undergrads once you get past a few questions. Just a really shitty slog.
It’s thinking either way. I would even wager that the trivial code that GPT writes may be easier to read for me, than some convoluted, human language description of the same thing, done with numerous corrections at every point.
The relative uniformity of code is a positive for human understanding as well, e.g. a^2+b^2=c^2 is easier to parse/transmit the idea over any “spoken” version of the same thing.
True. Although I did find the writing of prompts to be quite exhausting due to tedium and waiting for the output to be frustrating, so that would dig into my energy for peak coding. I would say it uses less concentration, but about the same amount or even more effort. But also it was a very narrow test, maybe for certain things (especially repetitive code or boilerplate code) it could be very beneficial.
I realize this comment isn’t directly related to the article, but it got me thinking… I wonder what effect AI will have on the next generation of gaming consoles.
Will they get beefier GPU capabilities to leverage local models?
Will they use cloud capacity to host models and games require always-online capability to play as intended?
Some mixture of both?
Also, I hope that AI won’t turn into the next way to just ruin everything that was already good. A bunch of rehashed unoriginal ideas, like “it’s Pac-Man, but with AI!” Please, no.
Despite the expected contrarianism in the comments, and I promise I'm being positive here, I'm pretty sure GPT-4 did really well on this task because a quick Google search shows a bunch of existing projects spanning blogs, GitHub and YouTube that it almost certainly trained on:
This is not a Bad Thing (tm) -- it's actually really sick because you can quickly get out of the weeds and get productive, especially if your skillset is not as deep as you'd have needed to accomplish even half of this a decade ago.
Nobody ever said paint by numbers was capital-P "Painting", but sometimes it's a blast to do one. I remember being 12 and making custom WADs for Doom / Hexen; my 6th grade son builds endlessly-creative and complicated modded Minecraft worlds with detailed machinery and electrical circuits and all this crazy seemingly-adherent-to-real-world-physics shit. Angry Pumpkins is arguably an order of magnitude better than simply re-skinning a Cyberdemon, because lowering the "time-to-screen" with any project (and in this case, providing a blueprint) is fun, creative, and most importantly ENCOURAGING for the next generation.
Thanks for writing this. This really mirrors my own perspective. Yeah, these tools aren't exactly "learning to fish," but if this gets someone excited about "fishing', then that's a good thing. A lot of my early coding experiences were in a similar vein to what you described, altering gamemaker projects and things like that. And even now I have a lot of fun playing with this AI stuff. It can help me go from 0 to 20% on something I previously knew nothing about, and sometimes that initial boost is all I need to get over the friction and actually do something cool. Or sometimes I realize that it's not worth the amount of effort it would take to go to the next 80%. That's OK too.
I totally understand the cynicism around this stuff, but for me it's like... 99% exciting and cool.
That's just marketing. McDonalds will sell you what it calls a Hamburger with a capital-H, but we could argue endlessly about the veracity of that.
In the end it doesn't matter -- some people like it, some people hate it, it's good, it's terrible, it's whatever you at this moment in your life decide it is for you, but regardless of opinions, it's still edible.
I just love what this guy made mostly because he followed through and made something, anything, and put it out there (presumably for the pure joy of creation).. and it got us all talking about it lol
> That's just marketing. McDonalds will sell you what it calls a Hamburger with a capital-H, but we could argue endlessly about the veracity of that.
Well they can't sell you a quarter pound of horse meat as third of a pound of beef. There are laws about this and I believe they do deserve criticism. But that criticism is completely orthogonal to how good the "hamburger" tastes and the nutritional value of it.
But personally I could never do marketing. I feel like their skills lay in getting as close to a lie as possible without technically being one. But I also think this is why people are not happy with a lot of products, because they aren't getting what they were expected. But at the same time I think companies are in an arms race that has simply been a race to the bottom where we're at or near and no one can get an economic edge simply by telling the whole truth and nothing but the unquestionable truth. The (near) liars have the distinct advantage in a world where it is impossible to have objective validators. It's why we have reviewers but why reviewers also got metric hacked. Idk what the solution here is but I definitely understand why people are upset and I wouldn't call this a fruitful endeavour where we'll argue endlessly. That's more about people having difficulties in expressing why they feel frustrated.
And some people think Synthesia is actually teaching them piano.
It might get them into the door. It might give them all they are seeking and that’s all awesome. They might find that they hit a wall and need to go learn how to read sheet music. Or they don’t.
The thing about “it’s not real programming” is that I don’t think it matters. They might just end up a bit surprised that they hit a wall and have to go back and learn more of the fundamentals.
I learned just enough about “chord progressions” in order to ask ChatGPT to make both the lyrics and chord progressions for me
I just input them and humans like it
I wouldn’t have initially known the terms to use. Nowadays I can prompt for that too though “what concepts do I need to understand in this field” “tell me more about number 3” and I hope it didn’t just make something up but it’s good enough to converse with humans about - who also make things up
At best, I expect it will maybe reach parity with (more likely fall short of) the kind of accessibility that other low-code application development tools and WYSIWYG website builders have reached. Specifically, I can see where this kind of utility will be too limited for professionals and skilled amateurs to bother with, beyond maybe doing some kind of “boilerplate+”, and yet it will remain out of reach for the vast majority of laypeople who will quickly realize that programming requires more than just a grasp of syntax and an IDE. Square Space and Wix haven’t meaningfully impacted the professional web design market as near as I can tell, and Airtable hasn’t cost any SQL engineers their jobs.
Just my gut instinct, though, and I’ve certainly been wrong before.
Sure, and that deserves more critique than this. But the critique there is that you're being marketed a product under false pretenses. I'll give you an example to help clarify. In another thread[0] the OP is showing off their ML programming assistant. The project is unquestionably cool and has every right to be on the front page of HN. But there are still major questions I have. The implicit critique here comes form the fact that the author says "beats GPT 4 at coding" but then their basis for this claim is simply via HumanEval performance. In reality the evidence does not support the claim (it doesn't say the contrary either, it simply is indeterminate -- not to be confused with orthogonal/irrelevant). This is marketing. I'm not sure if it is dishonest marketing, but I would not call it honest marketing either. It's in the gray. The thread originally had an intro from OP who was specifically requesting comments too and so the context is appropriate, other than the nature of simply being on HN.
I like it too. I also wrote a negative comment, but it was about how the title was not completely honest, which is a different issue than whether or not this was an impressive feat of AI.
I just like that I don’t need to hire designers anymore
I didn’t mind designers or their cost, I minded the time and the gamble involved with output close to what you imagined, and revisions
I also found negotiating rights to be full of hubris and pride that caused alot of more friction. Ironclad contracts being too heavy handed with that community
I’ll take the potential forfeit of copyright in exchange for instant output, i can put up a payment portal for royalty free works, i can still sell data
I wholeheartedly agree. I am often one to critique ML systems, especially LLMs and GPT. But this is my training as a researcher. I think it is important that we recognize critiques and discussions of limitations are not equivalent to calling a tool useless or worthless. It does not even devalue the tool. Rather the discussions are beneficial for two contexts: we learn the limitations to build better tools (since no tool is perfect there will always be critiques available) and we learn how to use the tool effectively (you're still not going to use LLMs to do you differential equations homework despite this certainly being within the training set). There's also the important context that we are on a tech form in a community that is well known to be inhabited by the exact type of people that build, test, and deploy these types of systems (myself included). These are not the same discussions we'd be having in the context of sitting around with my parents who are absolutely tech illiterate.
My thoughts as a highly critical person (check my history, or even a very recent highly relevant comment in another thread) are "this is pretty fucking cool" (same to that very same thread btw). I hope to see more stuff like this. It is what makes me passionate about ML and it is what keeps me coming to HN.
Your work doesn't need to be novel to be useful. Nor does it need to be useful to be good. I'm just happy to be seeing people do shit and having fun.
Very good perspective from the creator's vantage point.
From the consumer vantage point, I do worry. Even Apple's "curated" App Store is already flooded with quickly made knockoffs... as close as they can get in the hopes that they'll confuse just a few folks into a purchase. It works well enough and it's profitable enough that an entire industry does factory-style it for every popular title. (Like a digital Shein.)
The internet is already full of the quick-launch low-value-add content that AI excels at creating. We're about to see a whole lot more! The good news is AI can also be very effective at prescreening and filtering out all this content vying for our eyeballs. Right now it feels like me vs the AI content cannon, but soon it will be AI filters battling AI content while the machines sort out what makes it to my digital devices.
Honestly there's enough junk content out there that I don't see generative AI making much of a difference.
What we need is better habits around peer-to-peer recommendation. I need people I know with good taste to spend more time talking about the games that they like!
This statistical plagiarism laundering is pretty neat.
IMHO, stopping the laundering gold rush is a more urgent priority for law, than creating market moats for the current big pickaxe vendors and pretending it's about preventing HAL.
I honestly don't think the coding is that impressive. What sells it is the assets. It used to be that all quick demos / gamejam like games like these (which it honestly is) looked like crap. Now it suddenly can look a bit polished. Not just boxes and lines, but actually somewhat nice graphics (which is probably too close to what it mimics and would end with a angry letter from a bird attorney if was used in an actual game, though).
Are there any plagiarism detection tools for software? Because I'm very curious how closely this code matches one of the many JS Angry Bird clone tutorials out there.
Someone showed me a similar game type thing they had made with GPT-4 and I could find the code that it had used just by searching Github. It had taken an open source project and made some modifications and spliced a few extra bits together - which is still impressive but the bulk of it was already out there. This matched my limited experience too, unsurprisingly it does a lot better for problems where there is already code and discussion out there.
I wasn't able to find exact code for the posted game though so I think something cleverer may have occurred!
This inspired me to try and get a sprite sheet with the top down animations for a paladin for a potential RPG game. See me fail here: https://imgur.com/a/2uJyUT3
Actual order was top down variants first, and then the last one was the side view as I was curious what it'd show.
I've used midjourney to create ttrpg tokens in the past, and the best way I was able to do so was to say "aerial view" -- top down was very flakey in if it would work. I wonder if Dall-E 3 needs the same guidance.
The real challenge is generating a large code base (think more than a JavaScript loaded page) - the front end backend and everything in between and then automating the testing and deployment …
Twitter is a bad place to share original content these days, because anyone without a Twitter account not only won't be able to see anything more than the first tweet but (crucially) won't even see a visual indicator that there IS more content to see.
I have an account and half the time when I follow a link from HN to Twitter / X it just throws an error that something went wrong / can't load the tweet. And video content works even less often than that. The site seems like it barely works at this point and is rapidly becoming a ghost town. What a waste.
I have seen plenty of complaints on here about Twitter in general, but I don't think the specific point that the new registration wall makes it a bad place to post original content has been discussed much here at all.
Happy to be proved wrong.
Edit: proving myself wrong here: it looks like links to nitter show up in comments here every few days, so the fact that the Twitter registration wall needs to be bypassed is actually pretty well understood. https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
As someone that has never had a Twitter account, I used to be able to see more of the "thread" or whatever it's called. Something has recently changed to where that's no longer the case and I can now only see the first post with absolutely zero way to access the rest.
Yes that sucks, the trouble is commentary on how and why it sucks tends to (inadvertently) take over the thread which is what the GP comment did and which is why they're better avoided/get moderated.
No, it won't. There's a big random element to this, especially when you are prompting GPT-4 directly through the ChatGPT interface.
Image generation models can sometimes produce the exact same image if you fix the seed they are using - there are different procedures for doing that for different image models.
LLMs like GPT-4 can have their "temperature" dialled down, but even at 0 they aren't guaranteed to return exactly the same response to a given prompt. I believe this is because they run floating point operations in parallel across multiple GPUs and floating point arithmetic isn't actually commutative - you can get back a slightly different result if the multiplications run in a different order.
This article is about replacing programmers with AI.
I, personally, will celebrate, since it'll allow me to explore opportunities I would not have time for on my own. And I'm not worried for my career, since 99% of people will still be unable to think and write coherently enough to use the AI tools.
And I think the reality here is not so much being unwilling to pay artists as the budget being extremely low. If I've got an art budget of $10 for a quick experiment like this, I'm not going to hire an artist, I'm just going to use the circle tool in GIMP.
Did you read the actual post? They use generative AI to replace both programmers AND artists. And if anything programmers have been at risk of replacement FAR longer than traditional artists. Low code solutions in the form of Squarespace for web programming, AirTable for CRUD, Unity for making games, the list goes ON AND ON.
I find it interesting that over the past decade so much investment has gone into making no code tools, and now ChatGPT is so good at writing code that it’s probably faster, more flexible and approaching the same level of usability for technically minded but non coding type folks.
I recently had to create a demo app to consume and publish a REST service using Mendix and it took a couple of days to figure out all the details, but doing the same thing in any language (bash for example) using ChatGPT would have taken minutes.
Deployment and version control can be solved without much technical prowess using PaaS/IaaS, especially if you’re comparing your costs with enterprise no code platforms.
It may be my personal bias talking (I’ve always disliked no code platforms because they feel more cumbersome when you have to do anything serious, I dislike ActiveRecord ORMs for similar reasons) but it kind of seems like No Code will be obsolete pretty soon.
Who wants to drag and drop when you can just ask, copy and paste?
I think this solution is about the perfect thing I can come up with, conceptually. Nocode is easy but rigid. Coding is flexible but tedious and error prone.
Being able to talk out what you want to quickly get the code, as long as it's clean, gives you the flexibility to then tune as needed. And in some cases, like apparently this one, that wasn't even necessary. Exciting times ahead...
I guess the thing that's still lacking from an AI code solution is security guard rails. My demo below for example is open to injection attacks but I think that could be solved with fine tuning or custom instructions.
You can't just have amateurs copying and pasting stuff into production environments, but even the task of writing tests can be given to an AI. Like you get one AI to write some code, then you get another AI to write code to test it. The chances of perfectl complementary errors are pretty low, but even if that happens you then get an AI to write a frontend script to run automated integration tests, and then have some human quality control at the end.
Really, I think a code-based AI pipeline has much more long term potential than no code does. The interfaces are just so laborious.
I wouldn't say specialist, because GPT often ignores full context of the information. Sometimes, if a system is composed of A B C D E, and you only mention the first 4, it will answer with no problem as if E was irrelevant. As soon as you introduce E, then, it will quickly say sorry and reconsider. I'm a true hardcore generalist, and gpt can help a lot, but I wouldn't say I can become a specialist with it because it doesn't investigate at all - if you don't mention it, a lot of times it's as if it doesn't even exist.
As a technical generalist I certainly find myself pretty empowered by these LLMs. It's never been easier to dive into a new problem, figure out the boundaries and solution paths and get cracking.
328 comments
[ 1.4 ms ] story [ 297 ms ] threadhttps://spinner.franzai.com/
Think it could be an interesting UX pattern. Having interactive loading (spinner) games that at least give is feedback that our actions (even in between things) have impact.
I clearly recalled having read the news about the patent of this having expired a while ago, and from a quick search, a while ago, has been 8 years ago https://www.eff.org/deeplinks/2015/12/loading-screen-game-pa...
This one at least shows the finished product, which is indeed pretty impressive.
Some details I'd need to know are (a) how long did it take, (b) how many prompts, (c) how many course-corrections were required, and (d) how competent this individual was with the technologies in question.
I've personally found ChatGPT extremely empowering in lots of scenarios, but code generation was not among them.
A huge huge factor is knowing the limitations and getting better at prompting. And identifying likely hallucinations and asking for risks etc.
I’ve found it best with tech I don’t know well (I’m an android dev using it to make websites, something I haven’t done myself in like 15 years).
Most of the coolest stuff for me is help with sysadmin and running the server. The ability to debug gunicorn errors is great.
I do have to modify the code it outputs as the project grows and it loses context, but honestly the context limits are the biggest hurdle for bigger projects and those will be lifted soon.
Edit: Most recent site I made with like 95% code from ChatGPT is https://cosmictrip.space/ which generates prompts with GPT-4 that are then used to generate space images with DALL-E.
It's a simple site but there is a secret adventure game I'm working on (GPT+Dall-E) that is open-ended image+text AI-driven game. I'm hoping to launch before Nov 6 with DALL-E 3 API (hopefully...!). The adventure game is also written like 95%+ by ChatGPT.
I've had such great success with it coding that I'm using the GPT-4 API with an agent I'm making (everyone is huh). I have function calling hooked up to generate structured subtasks that the agent can then write the code for, and support for files to include context, chat with your code, etc. It's not ready to show but the GPT-4 code generation abilities are really incredible - but you have to be experienced at prompting. Your first prompts aren't likely to be great, which is why I'm hoping my agent can have success. The idea of the agent I'm writing is a Jira/kanban style board where you have AI coders assigned to tasks that you can approve and modify etc. The tickets should automatically move across the columns as the AI checks the work etc.
One responsibility at my current job is administering a Windows server and trying to get it to do things that are easy on a Unix -- that should be easy anywhere -- but, on Windows, seem to inevitably degrade into nightmares. ChadGPT has given me huge amounts of blessed ammo to shoot at the nightmares, and there's no way I could do that portion of the job in a feasible time frame without it.
Reading good prompting is probably one of the better ways of learning how to do it.
Some people have hooked AI dungeon / koboldAI up to stable diffusion to generate these kinds of procedural Ender's game style interactive graphical text adventures with varying degrees of success.
If your game is going to be similar, you'd better get in the habit of aggressively caching the generated imagery for it on S3 because no way the DALL-E 3 API is going to be cheap.
> Although the game is just 600 lines of which I haven't written ANY, [coding the game] was the most challenging part
Not quite hello world, but not too much more difficult than a shopping list. The really impressive thing to me is you can make angry birds with just 600 loc (and a couple libraries)
I was pleasantly surprised at the visual quality, I knew Midjourney could produce quality graphics assets, but I guess I didn't realize how easy it was to pull into a game.
Programming a new game without dozens of existing templates would be a better litmus test.
GPT4 is great at this stuff, but iterative refinement doesn’t work in my experience.
As the conversation increases, the previous context is lost and the generated code deviates from its previous behaviour.
For example, “fix this bug…” can easily result in a solution that breaks some other thing. You can also see code generated in thread (1) that does exist in the final result (2), suggesting that (since this is the very top of the code), they were getting chatGPT to iteratively generate 600+ line segments.
I severely doubt this.
Creating a new Slingshot on line 20 after it is defined on line 500? That is extraordinarily unlikely unless you specifically prompted it to do that.
“loadImage('stone2.png');”, it just happened to pick the right file names? The right sprite sizes? You provided all that in a prompt and it wrote the code? Come onnnn… show us the actual prompt you used.
It seems much more likely they generated a set of class objects relatively independently, then manually assembled them into a larger file, copied the entire thing as input and then crafted a “code prompt” like “write a function that does such and such”.
It’s not impossible they used prompts like they claim (3), but I feel they are (for likes and cred) vastly overstating the “it did all the coding” part of this project.
I feel they probably hand wrote some of the code (or assembled it) and used it as input + a “now do this also” style prompt, so the output was “100% generated”, but not in the way people are assuming.
This approach tends to make GPT4 rewrite the existing code, but unless you specifically ask for (or add) comments describing the intent through out the code (missing in most of the generated code), it will drift from the previous functionality. With no test suite to verify, you won’t notice this subtle drift and things just break. There’s no mention of either of these things being done by the author.
Further more, this user has a vested interest (4) in selling training materials for AI, so it’s in their interest to appear to be an expert at this, and has provided (even when asked on X) no additional details, no “step by step” git repo with history, no actual prompts they’ve used.
Given the lack of details and the frankly unbelievable results, I think there’s fair call to be sceptical in this case.
You could generate this kind of thing from models such as codellama 34B, or GPT 3.5; but not using the method as described.
I’m… not convinced you could do it with gpt4. The prompts seem too stupid to be real (5)… but I happy to be proved wrong with more details. GPT4 is good.
[1] - https://nitter.net/pic/orig/media%2FF9xoI8mXgAAn7v9.jpg [2] - https://bestaiprompts.art/angry-pumpkins/sketch.js [3] - https://nitter.net/javilopen/status/1719363669685916095#m [4] - https://javilopen.substack.com/ [5] - “Now, make the monsters circular, and be very careful: apply the same technique that already exists for the rectangular ones regarding scaling and collision area, and don't mess it up like before. ”
You have to generate specific api calls with specific semantics not “high level summary” and “key information” .
You have to generate token sequences that are functionally equivalent to an exact segment of the input prompt.
A lower precision is less useful for code generation.
How do you maintain existing code functionality when you give code + “refactor for x” as an input?
I’m skeptical you’ve tried this and know what you’re talking about.
Also, this works well when the system gives you patches. I wouldn't say "rewrite this entire thing to make this one change" because if it's anything non trivial it will shift. Again I see the same patterns for code and text. Ask for small changes, contained updates. Guide the model to work in small increments, using abstraction to deal with high level broad topics.
You wouldn’t use it refactor because the implementation will drift, but you’d use it to generate a patch to apply to the code?
Hm.
Well, I don’t think that style is relevant or practical for large scale code generation as shown even in this ~600 line example.
…but, I’d love to see the details of how it could done, end to end, by someone who has done something like that successfully and is willing to prove it by sharing the prompts and outputs.
It's not really "its" achievement. There are many open source repositories of Flappy Bird that it was trained on and there are many brilliant engineers at Google and OpenAI who have worked on LLM technology. It is their achievement if anything. It is a product. A product of others' achievement. You don't say Excel had an achievement when it calculates a good formula for you? Do you say a neural network that produces great speech to text had an achievement? Or do say the people who developed it did.
Sure Why not? The engineers at google and open ai or wherever else trained the models but they didn't teach it how to do anything it does because they don't know how to teach it to do anything it does. So yeah the achievement is on the neural network.
Many people gave the likes of alpha go achievement on super human Go play.
It is? OK, tell them to completely clear the model of any angry bird original and clone code, data and assets that it scarfed from the Internet.
It can retain scarfed definitions of angry and bird as well as scarfed information on making a 2D video game.
Then tell the model to create the game and see what if anything it comes up with.
Most of the Angry Birds mechanics in this demo come from the underlying Matter 2D library. Relevant demo here: https://brm.io/matter-js/demo/#slingshot
Honestly, that's most of the code part accounted for.
The bigger question is the graphical assets. The developer shared their prompts for those here: https://twitter.com/javilopen/status/1719363587351740711
Some of them make no mention of Angry Birds - "Wooden box. Large skeleton bone. Item assets sprites. White background. In-game sprites" - but others do: "Item assets sprites. Wooden planks. White background. In-game sprites. Similar to Angry Birds style"
My hunch is that a skilled image prompter could still get to results that were right for this particular demo even with a model that had not seen Angry Birds assets before.
And before Musk was born, similar debates about describing Armstrong as the first man on the moon, given how even saying that inherently brushes aside the contributions of all those who worked on the mission without going to space.
[0] In the UK it seems to be half on the Prime Minister and half the party of the age; I don't pay enough attention to the local opinion of politics elsewhere, though I will say the British press seems to blame everything Germany does on the German Chancellor personally…
Show all prompts step-by-step and let people reproduce.
Someone claims they did X using a couple of AI instruments, without sharing anything to prove the claim, and everyone is excited. I understand that the industry needs a new hype to keep going, but this is just pathetic.
Read beyond the first tweet.
If you also see six tweets, please reproduce the result following instructions in the tweets and share the recording of you doing so.
Graphics don't matter, let sprites be colored rectangles.
Why? Are the sprites and backgrounds too easy to recreate with MJ and Dall-e after you realized that the author actually provided the prompts for them?
Just reproduce the code with LLM of your choosing. If you manage to actually do so - please feel free to add graphics with any other model.
Have we created real AI, or merely 'AP' - Auto-Plagiarism?
https://nitter.net/javilopen/status/1719363262179938401
https://fosstodon.org/@javilopen@bird.makeup/111332485943091...
Also: I think you just scared the daylights out of a lot of mobile game developers and inspired many more.
You could literally make all this in probably 24 hours and not spend any time mucking around with prompts.
Like what are we even seeing here? Basically a tech demo of a physics engine, a little UI interaction for throwing a collidable entity into other entities, and some code for setting up a level?
Show me the maintenance, adding new features, bug fixing, cross platform compatibility, shaders, networking code, sound, etc.
Isn’t everything you just listed (physics engine, UI, collide entities) how you make a simple game? It doesn’t matter that it doesn’t have networking code, shaders, or sound. Those things can be added later.
I don’t think it’s productive to gatekeep who is and isn’t a game developer. They developed a game, that makes them a game developer—it doesn’t matter how they got there.
This person successfully developed a game. Just because it isn’t how others would have done it doesn’t minimize the result.
Similarly, using AI to develop a game makes you as much of a game developer as someone who hires someone to make a game for them. And some development studios are basically that, people hiring developers to make games. But AI can’t compete well with that.
It’s like coming up with a new fusion power plant, but it takes more energy to run than what it produces. Inspiring, but useless.
Using a computer to write a letter makes you as much of a writer as someone how hites someone to write a letter for you. And some writers are basically that, hiring scribes to write for them. Computer can't compete well with that.
Except it does.
This really sounds like “real programmers write in assembly” to me. An AI is a tool, not dissimilar from a higher level language. This person used a tool to create something that didn’t exist before. The result is right there.
I couldn’t make that art in 24 days.
The big thing about the state of current AI isn’t that it does a bunch of things, it’s that it drastically increases the number of people who can do things.
That is, I could not make this game in a day before AI. With it, maybe I could. I am far from unique.
Giving the whole world blogspot.com 20 years ago didn't measurably improve the internet, it just added a few more winners and a gigantic increase in low-quality spam pollution.
But it's mostly been used to pollute the internet with useless spam. With generative AI, the same thing will happen. The very example shown here is an Angry Birds clone, patient zero of the pay-to-win hellscape of mobile games. I don't think it's an accident that they had AI in their hands and decided to build that.
> With generative AI, the same thing will happen
I’m not sure what your point is. Of course it will. When powerful tools are available and perverse incentives exist, humans will follow them.
Regardless, I can pull down some off the shelf assets from an asset store and make a game with them.
Where are the ones that look different from the ones 50 other game makers used?
I don't know, and now I don't need to care.
I've learned Kubernetes and Powershell this exact way.
Maintenance and new features are more of the same already done. Networking code is a library. Sound is a library. Cross-platform is a library. Shaders are a library. AI fixed bugs.
I'm a game developer. It's impressive.
I'm extremely aware of how short a game jam demo is compared to an actual product. There's an ocean between the two.
That doesn't diminish how much easier and more approachable it suddenly is for someone with little experience to make a game jam demo, and with kinda-passing art.
Why is it not a game? I even played it a bit -- was I hallucinating?
The Twitter thread shows exactly what the AI made. It was used for the JavaScript code and the image assets.
If you don't have a Twitter account you can see the full content on Nitter https://nitter.net/javilopen/status/1719363262179938401 or in my Gist copy: https://gist.github.com/simonw/f7ed52daaa66f849858d17e0d6c1c...
I did that by building an open source VS Code extension to interact with GPT3.5/4 directly from your editor: https://marketplace.visualstudio.com/items?itemName=Biggles..... It cuts out copy/pasting between ChatGPT and your editor, or writing any boiler plate code since you can just ask it to insert the code you want, or change the code you have highlighted. You can also talk to it directly using the Whisper API!
I suspect it was slower than just writing the code/referencing the docs, and would be much slower than someone could do if they were experienced with the two frameworks. I had to be very specific and write a few long and detailed prompts for the more complex parts of a the application. It took around 5 hours to make the application, with a lot of that time spent sitting waiting for the (sometimes painfully slow) ChatGPT output. In a framework I'm more familiar with I think I could have easily got it done in under 2 hours
It was definitely useful for making sure I was doing it the correct way, kind of like have an expert on call for any questions. It was also very useful for generating perfectly formatted boilerplate code (some frameworks have CRUD generation built in, but this one did not).
It was a fun experiment, and I found it useful as a learning/guiding/generation tool, but I won't be using it for general day to day development any more than I currently do. For most instances it's quicker to just learn the framework well and write the code yourself.
I've found it to be shockingly good at this. I end up asking a lot of questions like, "what is the best directory structure for a project on {foo} platform?" or "What is the idiomatic way to do {x} in {language y}?" It has the advantage of having seen lots of projects in every language, and for some questions that automatically leads to a good answer.
It's like having an automated fact checker! I wish I had the same thing for the other kinds of output it produces.
One example was really really hard to spot. Once I queried GPT 3.5 for a function to do X, and it did pretty well, when I looked closer though, it had wrapped 90% of the code in an unnecessary if statement. I looked at the code an thought something was off until I realized.
My point here is, if that was easy to spot, who knows what else people are missing because even in a simple case, unless you're actually trying to spot issues, you likely won't see them.
Yes, absolutely agree.
And also I’m no fanboy but when it does this and I only notice because the code doesn’t run, half the time I’m thinking to myself that the API function really should exist because it’s so logical!
This is my experience. You still have to understand programming: you're just typing it out in Natural English.
I often do things like this, and if the scale of the document was somewhat larger than you describe, it would be a design document.
For many things the available documentation is poor, and asking a bot is much more helpful.
It's definitely not something someone with zero coding experience could easily do, and I feel even a junior developer would struggle with anything even as complex as this. You have to have the concept and structure in your head, it's just writing the code for it.
I wonder how far you'd get with the technique of asking ChatGPT to lay out its plans first, kind of like the improvements in math questions you see when you ask it to write down its reasoning before committing to any answer.
"This is what I'm looking for: XXX. What are the different pieces of this that I'm going to need to create?"
"Ok, the first thing on your list you gave me was a database to store the data. What would the structure of that database look like?"
"Can you give me the code to create that database?"
Etc etc. i.e putting the actual code as the last step each time.
For me, using GPT felt very slow, but for a junior it might actually be faster than trial/error and Googling. Also, ChatGPT is only going to get better and better, so we can expect it to become quicker and easier to do such things.
So you could spend 3hours babysitting GPT4 to write some code for you, but then you'd still have 3 hours of peak code productivity that day that you haven't "used up" yet
The relative uniformity of code is a positive for human understanding as well, e.g. a^2+b^2=c^2 is easier to parse/transmit the idea over any “spoken” version of the same thing.
Do you use the ChatGPT Plus version or the API? If the API, what do you usually use to access it?
Will they get beefier GPU capabilities to leverage local models?
Will they use cloud capacity to host models and games require always-online capability to play as intended?
Some mixture of both?
Also, I hope that AI won’t turn into the next way to just ruin everything that was already good. A bunch of rehashed unoriginal ideas, like “it’s Pac-Man, but with AI!” Please, no.
https://www.google.com/search?q=matter.js+angry+birds+clone
This is not a Bad Thing (tm) -- it's actually really sick because you can quickly get out of the weeds and get productive, especially if your skillset is not as deep as you'd have needed to accomplish even half of this a decade ago.
Nobody ever said paint by numbers was capital-P "Painting", but sometimes it's a blast to do one. I remember being 12 and making custom WADs for Doom / Hexen; my 6th grade son builds endlessly-creative and complicated modded Minecraft worlds with detailed machinery and electrical circuits and all this crazy seemingly-adherent-to-real-world-physics shit. Angry Pumpkins is arguably an order of magnitude better than simply re-skinning a Cyberdemon, because lowering the "time-to-screen" with any project (and in this case, providing a blueprint) is fun, creative, and most importantly ENCOURAGING for the next generation.
Anyways, I like it!
I totally understand the cynicism around this stuff, but for me it's like... 99% exciting and cool.
In the end it doesn't matter -- some people like it, some people hate it, it's good, it's terrible, it's whatever you at this moment in your life decide it is for you, but regardless of opinions, it's still edible.
I just love what this guy made mostly because he followed through and made something, anything, and put it out there (presumably for the pure joy of creation).. and it got us all talking about it lol
Well they can't sell you a quarter pound of horse meat as third of a pound of beef. There are laws about this and I believe they do deserve criticism. But that criticism is completely orthogonal to how good the "hamburger" tastes and the nutritional value of it.
But personally I could never do marketing. I feel like their skills lay in getting as close to a lie as possible without technically being one. But I also think this is why people are not happy with a lot of products, because they aren't getting what they were expected. But at the same time I think companies are in an arms race that has simply been a race to the bottom where we're at or near and no one can get an economic edge simply by telling the whole truth and nothing but the unquestionable truth. The (near) liars have the distinct advantage in a world where it is impossible to have objective validators. It's why we have reviewers but why reviewers also got metric hacked. Idk what the solution here is but I definitely understand why people are upset and I wouldn't call this a fruitful endeavour where we'll argue endlessly. That's more about people having difficulties in expressing why they feel frustrated.
It might get them into the door. It might give them all they are seeking and that’s all awesome. They might find that they hit a wall and need to go learn how to read sheet music. Or they don’t.
The thing about “it’s not real programming” is that I don’t think it matters. They might just end up a bit surprised that they hit a wall and have to go back and learn more of the fundamentals.
I learned just enough about “chord progressions” in order to ask ChatGPT to make both the lyrics and chord progressions for me
I just input them and humans like it
I wouldn’t have initially known the terms to use. Nowadays I can prompt for that too though “what concepts do I need to understand in this field” “tell me more about number 3” and I hope it didn’t just make something up but it’s good enough to converse with humans about - who also make things up
Just my gut instinct, though, and I’ve certainly been wrong before.
[0] https://news.ycombinator.com/item?id=38091869
Anecdotally a lot of my coworkers seem to say they did something ‘with chatgpt’.
I didn’t mind designers or their cost, I minded the time and the gamble involved with output close to what you imagined, and revisions
I also found negotiating rights to be full of hubris and pride that caused alot of more friction. Ironclad contracts being too heavy handed with that community
I’ll take the potential forfeit of copyright in exchange for instant output, i can put up a payment portal for royalty free works, i can still sell data
I wholeheartedly agree. I am often one to critique ML systems, especially LLMs and GPT. But this is my training as a researcher. I think it is important that we recognize critiques and discussions of limitations are not equivalent to calling a tool useless or worthless. It does not even devalue the tool. Rather the discussions are beneficial for two contexts: we learn the limitations to build better tools (since no tool is perfect there will always be critiques available) and we learn how to use the tool effectively (you're still not going to use LLMs to do you differential equations homework despite this certainly being within the training set). There's also the important context that we are on a tech form in a community that is well known to be inhabited by the exact type of people that build, test, and deploy these types of systems (myself included). These are not the same discussions we'd be having in the context of sitting around with my parents who are absolutely tech illiterate.
My thoughts as a highly critical person (check my history, or even a very recent highly relevant comment in another thread) are "this is pretty fucking cool" (same to that very same thread btw). I hope to see more stuff like this. It is what makes me passionate about ML and it is what keeps me coming to HN.
Your work doesn't need to be novel to be useful. Nor does it need to be useful to be good. I'm just happy to be seeing people do shit and having fun.
From the consumer vantage point, I do worry. Even Apple's "curated" App Store is already flooded with quickly made knockoffs... as close as they can get in the hopes that they'll confuse just a few folks into a purchase. It works well enough and it's profitable enough that an entire industry does factory-style it for every popular title. (Like a digital Shein.)
The internet is already full of the quick-launch low-value-add content that AI excels at creating. We're about to see a whole lot more! The good news is AI can also be very effective at prescreening and filtering out all this content vying for our eyeballs. Right now it feels like me vs the AI content cannon, but soon it will be AI filters battling AI content while the machines sort out what makes it to my digital devices.
What we need is better habits around peer-to-peer recommendation. I need people I know with good taste to spend more time talking about the games that they like!
I'm having great recommendations from invite-only Discords these days.
IMHO, stopping the laundering gold rush is a more urgent priority for law, than creating market moats for the current big pickaxe vendors and pretending it's about preventing HAL.
I wasn't able to find exact code for the posted game though so I think something cleverer may have occurred!
Actual order was top down variants first, and then the last one was the side view as I was curious what it'd show.
Here's a screenshot to illustrate: https://gist.github.com/simonw/f7ed52daaa66f849858d17e0d6c1c...
For people without a Twitter account, I've pasted the content of the thread into a Gist here: https://gist.github.com/simonw/f7ed52daaa66f849858d17e0d6c1c...
The most important missing link is the live demo, https://bestaiprompts.art/angry-pumpkins/index.html
Nitter link: https://nitter.net/javilopen/status/1719363262179938401
It might be but it's an interminable debate that's mostly HN-offtopic since its been long rendered uninteresting through sheer repetition.
I have seen plenty of complaints on here about Twitter in general, but I don't think the specific point that the new registration wall makes it a bad place to post original content has been discussed much here at all.
Happy to be proved wrong.
Edit: proving myself wrong here: it looks like links to nitter show up in comments here every few days, so the fact that the Twitter registration wall needs to be bypassed is actually pretty well understood. https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
The registration thing is just like the paywall thing - if there are workarounds, it's fine.
Image generation models can sometimes produce the exact same image if you fix the seed they are using - there are different procedures for doing that for different image models.
LLMs like GPT-4 can have their "temperature" dialled down, but even at 0 they aren't guaranteed to return exactly the same response to a given prompt. I believe this is because they run floating point operations in parallel across multiple GPUs and floating point arithmetic isn't actually commutative - you can get back a slightly different result if the multiplications run in a different order.
I'm guessing the mood will be less celebratory when we can stop paying most programmers.
I, personally, will celebrate, since it'll allow me to explore opportunities I would not have time for on my own. And I'm not worried for my career, since 99% of people will still be unable to think and write coherently enough to use the AI tools.
And I think the reality here is not so much being unwilling to pay artists as the budget being extremely low. If I've got an art budget of $10 for a quick experiment like this, I'm not going to hire an artist, I'm just going to use the circle tool in GIMP.
By the way, I have a nice bridge for sale.
I recently had to create a demo app to consume and publish a REST service using Mendix and it took a couple of days to figure out all the details, but doing the same thing in any language (bash for example) using ChatGPT would have taken minutes.
Deployment and version control can be solved without much technical prowess using PaaS/IaaS, especially if you’re comparing your costs with enterprise no code platforms.
It may be my personal bias talking (I’ve always disliked no code platforms because they feel more cumbersome when you have to do anything serious, I dislike ActiveRecord ORMs for similar reasons) but it kind of seems like No Code will be obsolete pretty soon.
Who wants to drag and drop when you can just ask, copy and paste?
https://chat.openai.com/share/f98d04b9-6d93-46b7-9fa9-7c9ec1...
Being able to talk out what you want to quickly get the code, as long as it's clean, gives you the flexibility to then tune as needed. And in some cases, like apparently this one, that wasn't even necessary. Exciting times ahead...
You can't just have amateurs copying and pasting stuff into production environments, but even the task of writing tests can be given to an AI. Like you get one AI to write some code, then you get another AI to write code to test it. The chances of perfectl complementary errors are pretty low, but even if that happens you then get an AI to write a frontend script to run automated integration tests, and then have some human quality control at the end.
Really, I think a code-based AI pipeline has much more long term potential than no code does. The interfaces are just so laborious.