Show HN: Angeldust – a fast and efficient video game
Website: http://angeldu.st
My single server hosts 250+K active players in one giant, dynamic fantasy world. Angeldust's game server is pervasively multithreaded, implementing John Carmack's dream of processing an immutable world state each tick.
Both world handling and network traffic routing uses any number of threads. For networking, a typical game session only uses ~3Kbps of bandwidth, enabling decent play even over 2G/EDGE networks.
The client is programmed in C++ with bits of C, Objective-C, Java and PHP. It runs really well on very low-end and obsolete hardware, all the way up to modern systems with fast GPUs.
Angeldust works perfectly on Windows XP+, even without hardware OpenGL; Mac OS X 10.6+, even on the first 2006 32-bit Intel GMA945 MacBook1,1; Linux 64-bit glibc 2.17+; Android v2.2.3+, supporting Bluetooth game controllers; and iOS 6+, though iTunes App Store forces iOS 8+ (any way around this?).
I live stream weekly on YouTube and Twitch, even right now! Come ask me your development questions! I often see developers playing together with their kids.
YouTube channel: https://www.youtube.com/AngeldustLive
One more thing: the Angeldust website ( https://angeldu.st ) uses plain HTML and CSS to offer interactive, game-related actions. It's fully functional even without JavaScript enabled.
203 comments
[ 0.26 ms ] story [ 223 ms ] threadIn the past I have very-often stopped exploring a new thing at this point but this time I am continuing! Looks great and I also eagerly await more details of it's implementation.
This is a pretty straightforward set of expectations which should solve most issues.
For one password manager, my default length was 16 and at the time I ran into a sites few sites that restricted it to 12 or 14.
Is this less common these days?
Thanks for the game.
Anther oddity is that the Android version is free while the Steam version costs money. Is there some other kind of monetisation strategies for that version?
Also: my preconception was that Android users are more likely to pirate a product. So to prevent them from downloading a malware-laden pirate APK and getting a poor experience, I'd rather have them play the game for free without hassle and tarnishing my reputation.
To compensate, the Itch.io Android APK is actually pretty much identical to the Google Play Store version, but it just includes all 3D assets too.
I'm developing a game myself, and figuring out a good method of multithreading it took a lot of work. While I am content with the results, I wouldn't say I arrived at a good solution.
It sounds like you went with a purely functional approach? Did you rebuild the entire gamestate each tick, or keep a list of changes you needed to make to the relevant data?
https://news.ycombinator.com/item?id=15036591 was the previous HN thread about it.
The Angeldust game world is tracked in an "active chunk set". A chunk is a 2D part of the world, 32x32x64 in-game blocks (meters). The active chunk set tracks only the chunks that contain, or are near, players. This chunk set is processed in parallel on any number of threads/cores configured in the server settings file.
Angeldust tracks two states for the data contained in each chunk, kind of like an array of size 2. Each tick, one state in the array is the immutable "read"-state and the other is the mutable "write"-state. Every tick these states ping-pong between the roles.
During processing the server processes the read state and copies over unchanged entity/world data by simply pushing pointers to that data into std::vector<>s in the write state. For things that did change I push a new pointer into the write state and clear up the old pointer in the next tick. Pushing pointers saves memory bandwidth since you don't have to copy all entity data for every tick. And still you can guarantee that the "read"-state will be unmodified so that other CPUs/cores/threads can freely access all of the data and pointers.
On the livestream I discussed a few cases where I do use (largely uncontested) mutexes for data synchronization: for friend chat and private chat I do a lookup of destination hero pointers in the "hero map" and then lock/unlock the hero's mutex for pushing chat messages into a vector.
Does this make sense? Let me know if I should elaborate. I will probably take a nap now, but I'll answer you (and others) tomorrow.
I'll also do another livestream tomorrow, because I feel everyone on HN has very useful questions, feedback and ideas.
For the write state, you're absolutely right that entities will need to transition between chunks. The server looks up the destination in the "active chunk set" I mentioned and then uses per-chunk std::vector<>s guarded by a mutex to push "migrating" entities. This requires a mutex since multiple CPUs/cores/threads/chunks can be pushing entities to the same destination chunk. These mutexes largely have zero to little contention.
Does that answer your question in an acceptable way?
> The server looks up the destination in the "active chunk set" I mentioned
but what if the entity teleports to an inactive chunk?
> and then uses per-chunk std::vector<>s guarded by a mutex to push "migrating" entities.
Will these migrating entities participate in later calculation within the same tick, or are they excluded? If excluded, what happens when there are two mechanisms that mutates that entity, but the first one put it into the migrating list of another chunk? If included, the thread of which chunk will continue this calculation?
And when some kind of mechanism mutates two entities in different chunks, which thread is chosen to run this mutation? And how does it mutate the state for an entity in the other chunk?
Meanwhile since you mentioned ticks, (just to be sure) are you using a barrier per tick to wait for all threads to finish the same tick? When are the migrating entities merged into the main list, and are you using another barrier for that?
As for the processing per tick: the entire active chunk set gets semi-randomly processed by threads. You'd be absolutely right that the order of operations isn't always deterministic. An entity not being processed for a tick doesn't happen since the old chunk will still process entities that are "on hold", but only perform a subset of operations. Double-processing does occur and this is sometimes visible while playing with a chat message being duplicated. This happens pretty rarely and I took care to make sure occasional double-processing doesn't interfere with gameplay mechanics.
As for synchronization: yes, each tick all processing threads join together and the server performs minor central processing and scheduling. Then the next tick begins and all threads are started again.
I'm still unsure about the following question from your answers: > when some kind of mechanism mutates two entities in different chunks, which thread is chosen to run this mutation? And how does it mutate the state for the entity in the other chunk? My initial thought was this would require adding locking on every entity, since it might be accessed from another chunk/thread. But that's clearly not ideal. Maybe have a separate list for outside-chunk changes, and merge them with internal changes later?
And regarding having two states: How/when are you sending world state to clients? My thought was to clone the current state periodically and send it to clients (can sending takes less than one tick? I don't have an idea). Sending state to clients seems to be needed if the game needs client side predication, hence this question.
All external effects that can happen to an entity are written to an atomic integer that tracks healing points, damage points and effects like poison and stunning. Angeldust currently runs on CPUs supporting atomic integers everywhere, so this doesn't have to fall back to mutexes. On platforms/compilers without atomic integers I have a software-fallback that uses mutexes for this, but currently it's not used anywhere.
Most world state is compiled into flat byte arrays each tick to save processing time, since many clients can be in the same area at once. The server bundles these world state byte arrays for chunks near a client together and sends them over the network. The client interpolates from these updates. What you see on screen lags behind subtly to make sure animations are smooth. I think this happens in most online games since you don't really want to extrapolate.
Angeldust's game pace and mechanics are designed so that the interpolation doesn't affect your aiming accuracy or game experience too much.
I am very curious how and why this happened
The client uses Objective-C to integrate certain platform features into the C++ codebase on iOS and macOS. Java is used for some functionality not provided by the Android NDK.
Not sure how this didn’t come up before? At any point in development? O.o
Edit: whoa, downvotes, really? Care to say why? This is a legitimate concern?
https://en.m.wikipedia.org/wiki/Frank_Lucas
Also i may be wrong about the importance of these results - is it easy to push those down?
Only for those who would confuse a video game for PCP. Thankfully, those seem few.
And honestly? I certainly didn't associate it with the narcotic when I read the headline. I'm not sure I've even heard that particular slang for quite a number of years.
.. but i do not get the extremely heavy downvoting and very angry comments to the people concerned about the name.
Maybe it's no problem but as i said - at least on my Google - PCP the drug comes up as the first result with a complete explanation in quick-view.
And to this someone replied "grow up" to me, like what?
Isn't the whole idea behind showing HN something to get criticism?
If i made an app that had bad connotations i would certainly know. If someone disagrees, fine, a discussion about it without mean comments like "get a life" or "grow up" would be preferable.
Let me categorically say: Hell no.
Some may want critique, but unabated criticism? That's not good for anybody (even if it does feel good to write). They're showing off the work of years of their lives; having that work shit upon is not what it's been shared for.
> If someone disagrees, fine, a discussion about it without mean comments like "get a life" or "grow up" would be preferable.
Just remember, the original post was not a useful critique, it was a disbelieving criticism for the author not doing their research. The comment also conveniently ignores that the title of this post explicitly contains "a fast and efficient video game".
Even taken charitably the original post provides little of real benefit to a discussion about the game being presented. Uncharitably, it comes across as a playground attack based around who knows the street slang for PCP.
>Let me categorically say: Hell no.
It seemed obvious to me that the GP was referring to constructive criticism (which can be very valuable). I would not say it is "the whole idea" behind "Show HN" posts, but it is definitely one of the primary motivators for most of the posts I've seen.
That being said, I'm sure most don't put up a "Show HN" post to effectively hear "this sucks" with nothing helpful in the response (and that does happen).
Merriam-Webster: https://www.merriam-webster.com/dictionary/narcotic
Collin's Dictionary: https://www.collinsdictionary.com/dictionary/english/narcoti...
Edit, regarding "no one calls marijuana a narcotic":
Not sure why I was downvoted, considering that we are discussing angel dust rather than marijuana.
The United States Government calls marijuana a narcotic:
[1]: https://www.federalregister.gov/documents/2019/08/29/2019-18...
[2]: https://www.incb.org/incb/en/narcotic-drugs/index.html
As do The UN and the World Health Organization (in process of amendment):
[1]: https://www.healthpolicy-watch.org/who-recommends-cannabis-s...
[2]: https://www.who.int/medicines/access/controlled-substances/U...
And the Australian Government: https://www.legislation.gov.au/Details/F2018L00106
As well as the Swiss Government: https://www.ch.ch/en/cannabis/
And the Government of Singapore: https://www.cnb.gov.sg/
And the European Union: http://www.emcdda.europa.eu/publications/topic-overviews/cla...
It's important to note the apparent legal definition of "narcotic" when discussing drugs, as it drastically differs from the traditional use, and includes a much longer list of substances.
In addition I have complex, dynamic geometry to render. And I wrote a custom, fixed-precision integer math library to prevent rounding errors for the almost infinite game world. Other products have weird rounding errors near the start, the middle or end of their worlds because of inconsistent floating-point accuracy. Moving to doubles only mitigates the problem instead of solving it.
And then it's just efficiency too. I have no idea how or why Unity or Unreal will treat my assets. As demonstrated on the stream: the game instantly switches between visual styles and texture quality. It lazily loads all resources so the main thread is only stalled when doing bulk uploads to the GPU. This makes the game start up in less than a second, something I couldn't pull off with a COTS engine.
Let me know if this answer didn't address your question fully.
For Angeldust I've done dozens of updates, each and every single one building upon the existing base. Looking back it's amazing to see that the product evolved in an almost linear fashion from v0.0 all the way to the current version without ever having to drop or redo features. But because the product evolved as a whole I can not really distill a time estimate for only the "client side engine".
I mean: would you consider testing the client side engine part of client side engine development? For testing you need a working server. For which you "need" a working network simulator. It's dependencies all the way down.
And how about translation work for in-engine strings, updating shared libraries and frameworks, issue tracking, writing the asset conversion toolchain (which needs assets, which need integration). It's very hard to come up with any solid number. Maybe if you can severely limit your question's scope I can come up with an educated guess.
For the 3D world, start with a checkerboard pattern or so. In fact, this is almost exactly what I did to start Angeldust. I once showed off some early game prototypes on my stream, take a look at this one for example: https://www.youtube.com/watch?v=qMs8YpkSrg0&t=3289
If you keep watching that stream you'll see other in-progress versions so you can see how my product evolved.
Angeldust's physics are relatively straightforward since it's based on a voxel grid. All game world intersections are approximated using axis-aligned bounding boxes (AABBs). I wrote the physics engine myself from scratch to reduce the number of game world data lookups, because that's pretty much the hot path in the code.
In the YouTube video from my earlier comment you can see that initially I started out with just game world boundary clipping and simple plane physics on the checkerboard. As I expanded my game world model, the physics engine grew along. So my advice would be (again) to start simple and grow from there. You'll do fine!
I've noticed a first-pass test for video game quality is to watch the speed of menu transitions -- a slow transition time is indicative of other problems (menus are the least interesting, and very common, interface a game provides you; no one wants to spend any more time in the menu than they have to, except the designers).
Speed correlates highly to quality in game implementations.
Additionally, being a multiplayer game, fun also comes almost for free -- as long as you give decent tools for supporting the social interaction, the players will bring the fun in themselves. (For some reason, a lot of games fear their player base, and strangle the interaction opportunities as much as possible).
Fast+efficient+"some fun" > Slow+inefficient+"some fun"
I don't see that connection as an issue with the game or your post. It certainly isn't doing anyone any harm, despite what others suggested in earlier comments.
That said... I'm 21 years old, and I still associate "angel dust" with PCP. The name is still used by popular rappers including Mac Miller [0] and Run the Jewels [1]. It is still defined as PCP in 6 of the top 7 results, vs 5 of the top 7 for "sherm" and 0.3/7 for "wet" on Urban Dictionary [2,8,9]. Angel dust is the most common synonym for PCP on Urban Dictionary [10]. The name "angel dust" was used to refer to PCP in a 2011 episode of Its Always Sunny in Philadelphia [3]. The name is still in common parlance on Reddit [4,5,6]. However, the drug seems to have experienced a decline in popularity, accounting for less than 0.1% of all documented hallucinogenic drug use in the United States in 2017 [7] - suggesting that the term "angel dust" is not well known simply because the drug isn't commonly used, and not because the term has been abandoned.
I just don't want anyone to get into trouble thinking that they can use the phrase "Angel Dust" without it being seen as a reference to hard drugs by the general public.
I am also surprised that this association is being rejected, considering that the creator of the game is going by "Frank Lucas" [11], which is the name of a famous drug trafficker [12].
[0]: https://www.youtube.com/watch?v=rIQqzTNRmoc
[1]: https://www.youtube.com/watch?v=R_esHn4X03U
[2]: https://www.urbandictionary.com/define.php?term=Angel%20Dust
[3]: https://itsalwayssunny.fandom.com/wiki/The_Gang_Goes_to_the_...
[4]: https://www.reddit.com/r/PCP/comments/ba1754/two_dimes_of_pc...
[5]: https://www.reddit.com/r/Drugs/comments/5tru38/what_is_angel...
[6]: https://www.reddit.com/r/serialpodcast/comments/2qove1/how_l...
[7]: https://www.drugabuse.gov/drugs-abuse/hallucinogens
[8]: https://www.urbandictionary.com/define.php?term=sherm
[9]: https://www.urbandictionary.com/define.php?term=Wet
[10]: https://www.urbandictionary.com/define.php?term=PCP
[11]: https://angeldu.st/en/game#play-now-button-3
[12]: lostgame ↗ I love your username. Do you follow any particular current of Western Ceremonial Magick? LeftHandPath ↗ I do not! (Sorry to disappoint - I’m a run of the mill atheist).
My name was inspired by a Gilfoyle line in Silicon Valley, which references Magick:
https://getyarn.io/yarn-clip/9481dd82-17de-4791-b110-0e58434...
(Since you’re on HN there is a good chance you’re familiar with the show - but if you’re not, the Gilfoyle character is a self-proclaimed LaVeyan Satanist)
And if any of you can get me in touch with John Carmack that'd be very awesome ;)!
Kudos for this extremely well polished project. This is the dream many of us have but can't or won't embark upon. This is very inspirational.
I lived with my parents during this time and could save up money as a buffer. Thanks to that support I could financially bridge these past years of development without worrying too much. Also: keeping expenses as low as possible. My product is old school and so are my computers.
Great choices, fantastic execution.
You're going to go far. Keep it up!
I've been really lucky to be able to successfully fulfill my childhood dream by making this game. But like Freddie Mercury sung: it's [not always] been a bed of roses or a pleasure cruise. I've sacrificed progress in other areas of my life to see this through to the end. I hope to catch up there starting as early as next year. Nonetheless I definitely wouldn't be the same without having taken on this adventure. I've grown so much as a developer, an entertainer and even as a person by creating Angeldust.
>> I've grown so much as [..] and even as a person by creating Angeldust
These are words of wisdom that prove your point, and I think it's a great thing you've posted them here!
Steam: https://store.steampowered.com/app/488440/Angeldust/
Android: https://play.google.com/store/apps/details?id=nl.metagaming....
iOS: https://apps.apple.com/us/app/angeldust/id687448635
PS. The author mentioned it was free on Android fyi, because it only has the "Cartoon"-engine currently. If i'm not mistaken.
I started Angeldust on my trusty, white MacBook1,1 mentioned in the starting post. Having only an Intel GMA945 for powering a 1280x800 display and trying to get a decent frame rate forces some design decisions. I quickly ran into optimizing for GPU overdraw and resources instead of being bottlenecked by the dual-core Intel CPU.
Then I ported the engine to iPhone 3GS where you get the exact opposite situation: the GPU can push lots of pixels to a 480x320 screen, but the ARM CPU is struggling to keep up. Just this 2006/2009 era combination of hardware alone forced me to be efficient with both CPU and GPU. Eventually I even got the engine running smoothly on a first-gen Raspberry Pi on a 1280x1024 display which is no small feat.
I think if you start from "today's tools" on "today's hardware" it's entirely too easy to consider any older platform inaccessible or obsolete, while if you work the other way around you appreciate the sheer power older devices have when used properly. Then moving onto newer hardware is just icing on the cake and just shows you how ridiculously powerful a modern 6-core Ryzen with an appropriate GPU is.
https://www.twitch.tv/AngeldustLive
I'm not a social media user and I currently don't keep a public blog. There are many interesting aspects of Angeldust to talk about and I have many war stories from this and previous projects that would make for an interesting read or listen.
Just this evening I talked on my livestream about my rough introduction to SIGPIPE randomly killing my server processes. I can understand that a YouTube or Twitch livestream isn't the most convenient format for hearing these stories, but it's currently my most warm and entertaining way to share them.
Streaming is a highlight in my life, while writing out stories feels like work. Very faintly I'm crystallizing a plan to share knowledge in a written way, but for now I think the livestream is going to be my preferred way for requesting and telling stories.
EDIT: I'm not really interested in separating or spinning off the engine, but you will have few problems turning the game into something entirely different.
A separate test server would be a very good companion idea.
I remember that original voxel-based engine and web server list that launched the client using their custom URI.
It was awesome how small the game was and how fun it was the simple concept (Minecraft with guns and no floating blocks).
But I always advise enthusiastic players to keep an eye out on real life. Mind your health and your relationships. Real life, family, friends—they all come first. Of course you can be friends in-game and even build meaningful relationships there, but that's no substitute for living a well-balanced "real" life.
I've been an avid gamer in the past, and right now I'm a very avid developer. I try to balance that in real life by playing sports and exercising three or four times per week. Going out on walks, to the movies, meeting up with friends. "Balance in all things" as a character from Baldur's Gate would say.
https://news.ycombinator.com/item?id=21872869
However, now I just opened App Store, searched for "angeldust", and bought it for less than 5€... so clearly it is available.
So it has something to do with the path a user takes from the link on your website thru the Apple dialog that asks where to open the link [App Store], etc.
If you're still interested in trying out the product, can you try manually searching for "Angeldust"? I'm using the Dutch App Store(s) and the product shows up on multiple Macs and with/without being signed in.
It might also help that I'm Dutch and we're supposedly pretty "direct" by default. I've also learned from dealing with Russian players who are even more versed at that craft.
250K players is impressive. Are those real players or bots?
Interesting to know more about performance too, what is your rate of update per second? How long does it take to process a tick?
Do you have a tech blog? Would be really interesting to learn more.
Currently I only have my livestreams as a public presence. I find it a very comfortable and pleasant medium for conveying who I am. It lets me share stories and ideas including non-verbal cues and that's a major plus for me. So if you want to learn more, definitely subscribe or follow on YouTube and/or Twitch:
https://www.youtube.com/AngeldustLive
https://www.twitch.tv/AngeldustLive
I honestly appreciate talking about technical things a lot, even though I might be playing what appears to be a silly video game. As for streaming: I just did three consecutive days, but will take a short break. I'll be back this Thursday.
When I was a five or six years old kid I used to program on our home computer, a Philips P2000. Monochrome green on black CRT, Zilog Z80 CPU, different ROMs for running BASIC or Pascal or whatever you hacked onto it. My very first program drew Christmas trees on screen. Very appropriate to share this story right now. I immediately showed my mom, who wasn't really impressed, because she didn't understand what I had done. I had told my computer what to do—wow!
Then one day I was walking around the schoolyard imagining the world around me being digitized and available as large, 3D blocks inside my computer. "My" P2000 wasn't advanced enough to pull it off and similarly I wasn't advanced enough to program something like that. But the idea of having a coarse, 3D replica of my world and being able to have fun in there with my school buddies always stuck with me.
Many years later, at an intellectual dead-end in my job I took a month-long vacation to regain focus on my life and myself. And I suddenly remembered my childhood dream, or fantasy, of that 3D world. And I realized that computers were now advanced enough and I was advanced enough. So I decided to take on the journey, channelling my inner five-year old.
Some players have joked about this with me. That they're actually investing their time in the figments of a five year old's imagination. And that's not far from the truth. Even though Angeldust is better, more beautiful and more professional that I had ever imagined, it is the actual implementation of my childhood dream.
So the answer to your question is that I started development somewhere between being five years old and now. Hopefully this answer still gives you some joy and insight!