Game development - looking across the aisle from the Linux world - is such an insanely backwards place. Everyone reinvents everything, most developer shops refuse to even use the standard library implementation of whatever compiler they are using, and the entire process is a grindhouse for millions of lines of code, thousands of 3D models, textures, shaders, music, and dialog and all of it often never even sees the light of day once, or at best is used in one game before discarding and redoing it all over again.
The industry has gotten better on that front in recent years at least. Unreal 4 is public source, you can make modifications or contribute back (you are just bound by the royalty fee and fact you cannot redistribute source yourself) and the asset stores of Unity / Unreal / etc, while 99% garbage, at least are starting the industry on a path towards recycling rather than NIH waste at an alarming rate.
You want to talk about culture, think of what kinds of creative experiences we would have today if every game developer open sourced their engines and assets 5 years after publishing. They can keep the maps for their game, which would still give it unique retail value, but id alone created a huge gamut of free software games through the publication of their engine sources in the late 90s and early 2000s. Think of the possibilities if those resources were not inevitably lost in some discarded outdated mainframe a decade after the games release like most are, but instead were made useful long after they made their money off them for hobbyists or indies to appreciate the hours of work designers or programmers poured into these products.
Those were my impressions moving from the open source / web world into games too. It surprises me still how little is shared in terms of reusable libraries, and often how little attention is given to reusability beyond a single project's scope. Automated testing too.
Those habits feel like a super power now, because at this point I naturally think in those terms and build accordingly.
We have quite a decent toolkit for VR at this point, but now I have the challenge of how to share any of it, because we do use a couple commercial libraries from the Unity asset store, and I don't know that there's any way to mark those as dependencies.
Often libraries are far too specialized to share or as you point out, have dependencies/licensing that prevent open-sourcing/sharing.
For example I've written various entity/component allocators, pools, and free-list implementations that run ultra-fast and are reusable across almost any game that needs an entity system, but almost each one is optimized for the performance characteristics of the target game. When it comes to game dev, you generally can get much better performance by writing things for the purpose. Usually reusability and performance are at odds in these cases. That might sound counter-intuitive to some in the web world, but it's a fact.
Even in website, you are better off doing things for example like pooling in garbage collected languages, avoiding extra allocations, and using more primitive data structures. This of course makes the code harder to read, test, maintain, etc. so developers in these cases tend not to do it or even be ignorant of these techniques. In games, you do things all the time for your specific use-cases and this really impacts testing and reusability.
It sounds counter-intuitive, but over the years I've learned in most cases reusing things in games is a fantasy only inexperienced developers get hung-up on. That doesn't mean nothing gets reused, rather what tends to happen is you learn from your mistakes/drawbacks of previous code and rewrite it better. In effect, you are reusing the code but it's not a literal drag and drop/cut and paste into a new project. Over time, if something is really solid and doesn't have too many performance/usability drawbacks, it may finally make its way into some sort of cross-project more drop-in library. An example I've directly used EA STL. Even when you reuse something though, people tend to do the equivalent of fork it per project, again limiting reusability.
I suppose in the age of better hardware with tons more ram, some of these concerns may be going away if your target platform is PC, but with mobile, more resource starved vs pc (ex: Nintendo 3ds) and embedded-like platforms gaining popularity, inevitably things circle back to the direction of optimizing per game and per hardware which hurts reusability. Is it really that valuable for example to dig up some library you used one 1 game in the lifecycle of the ps3 when you've already moved to the ps4 for example? Probably not for most developers unless you have a large window between your game start time and the platform lifecycle. Even on PC this is a problem as everything from graphics APIs to average hardware can obsolete a lot of what you've done or make it relatively inefficient/primitive in a new project.
I think there should be a small distinction between AAA and indie game development. Not all indie games need the level of specialization and optimization that AAA's do, and so they can get away with more reusable abstractions that let you move faster.
But I've seen the rationale that games require too much optimization by a few indie devs as to why higher-level abstractions aren't used (not all, of course, and in some cases it's true), and that's where I think some indies that think in terms of reusable abstractions can gain an edge. It's a culturally held belief that I think sometimes gets adopted by new devs who then won't end up spending the time learning the abstractions.
I'm no design pattern purist by any stretch, but you can often find ways of cutting corners while still keeping things abstract enough to be reusable. Documenting goes a long way towards that too, as does a culture of automated testing, and a better package manager with dependencies.
I've worked in game development in various capacities since the 80s. While I agree with your sentiment, a lot of your points are incorrect and I feel deserve a response. Sorry about the length.
Firstly, the reason why people didn't/don't use the STL is for a long time it was garbage for games. The data structures and allocators were huge offenders. If you've done any real game development, especially in the more resource constrained days, used std::vector was terribly inefficient and made no sense. This is why studios all wrote a lot of the same lower-level code. As to why they didn't share this code, it's just the nature of the business, however a lot of the performance of certain libraries over the years was highly tuned to the individual game.
Generally, games don't generate much reusability. Formats for assets change over the years, compression improves, resolution goes up, music chips improved, and so on. Even today things might have slowed-down, but programs used to produce assets make the level of effort of rehashing old assets not so worthwhile. Game studios that have tried to reuse things were often criticized for recycling when detected. The irony here is that this is actually justification for what you are saying - the assets aren't worth much long-term, so you might as well give them away along with the editors and level construction.
An important thing you're forgetting though is a lot of editors and tools used in game development are very one-off and shoddy. As a studio, you simply don't have time to sit there and write a perfect UI or even unity-like UI for your tools. Sure, it happens sometimes, especially when a tool is upgraded game-by-game, but generally tools are fire and forget and often given to the lower-skilled team members. I've rarely worked with game dev tools that were not full of bugs and woefully incomplete, but this is just the nature of the beast when you have limited time and budget - you don't spend it all on making perfect tools. This alone I suspect prevents many studios from releasing things to the public because the tools are very fragile, buggy, imperfect, incomplete, and so-on that it's almost embarrassing for some to see the light of day and too much maintenance for the benefit for others.
Regarding engines, I'm not sure what the gaming public's obsession with engines is. For well-tooled and documented engines, you have various degrees of offerings these days ranging from Unreal to CryEngine to Unity to other smaller, but still good engines. Those games with "custom" engines are probably no where near as useful. Sure, releasing the source would allow customizing and modding an individual game further, but again there's a lot of factors that go into this ranging from incomplete/broken pieces to game-specific hacks to lack of testing and so on. Unless a studio is in the business of releasing game engines and tools, it's hard to justify the risks. Often the copyrights and other things are not even owned anyway by the people who want to release them, so there's nothing you can do for example when your parent company or publisher views each and every line of code as some secret intellectual property.
What really bugs me though is why people think that game engines are some magical thing. What is a game engine even? In my experience, in many game codebases it is hard to draw a line between "engine" and "game." Sure you can set out to do this from the start in your architecture, but most of the time it won't happen that way when you are under the gun. The best you can do maybe is circle back and architect/extract things better post-release, which no one generally gets paid to do (unless selling the engine). Even then, the engine is not some magical thing that makes the entire game work. An engine can be anything from a way of handling assets + some graphics libraries and input handling to only graphics libraries to a stack of glued toge...
You've missed a few major reasons as to why the industry is like this. Let me explain:
1. We don't re-invent everything, but the hardware keeps getting better and we want to take advantage of it. To do so requires rewriting large portions of code. It is only because the basic hardware capabilities everyone has these days are pretty good, that using an engine not designed for your game has become so much more viable. Previously you'd build the engine to fit the game in order to get the performance you expect.
Shader's for example went through a massive upheaval when we changed from fixed function to custom, and now compute shaders have changed things again. Additionally there are always new algorithms presented each year which produce better results, which we must have!
2. Code that works well in an event based system does not necessarily work well when you are running it 60 times a second. This is why companies sometimes go in and replace standard functionality. We don't want to, but to get the speed everyone expects, we have to.
3. Models, textures, animations, dialog and so on are data, which the computer doesn't really understand. Where we see a person it sees a triangle mesh, or where we see signs of a bloody battle it just sees red pixels. This semantic mismatch is the biggest problem in the industry today, and makes re-usability incredibly hard. Manpower is usually the only way to overcome this, with people spending months on a single character model. Additionally better tools are always welcome but the industry has stalemated a bit in this regard.
Generally animation is still best imported from real life, EA for example have an entire building devoted to just motion capture [1] with over 100 cameras.
4. Many old games are still available on GOG and other websites, so these companies still make money from them which likely influences their decision to not release source code. Console games also are restricted by proprietary console-only APIs.
> still make money from them which likely influences their decision to not release source code.
This isn't actually a thing at all. All the id games still sell like hotcakes moreso now, because of the open source engines that keep those games modern and have dramatically improved over the originals (check out Brutal Doom for an example).
Even if your engines and your art assets are liberated, you still do not have a complete game without maps. And while the user community built up around your product would certainly make their own, the fact they are all created on the basis of your game would, almost always, drive dramatically more long-tail sales to just play your original maps that inspired an entire ecosystem than locking up the complete package to rot in video game obscurity like the thousands of Playstation 2 games are doing now.
He says that he loves Seattle NOW that he LIVES in it.
When he had to live elsewhere in Washington due to work, he only had few chances to visit it (those "3 times in 6 years"), so he didn't get the chance to appreciate it back then.
What's confusing about it?
(Besides the fact that even if somebody already loved some place very much, there's nothing contradictory about them still only managing to visit it only a few times over 6 years, or even not once -- if they had too much other stuff to do. There are people who haven't took a holiday in 10 years, much less 6 -- and even when they did, they might go somewhere else, e.g. to somewhere they have family, or to some beach ladden place).
The implication for me was that it was odd since he worked in Bellevue at Unity's offices. If it's true that he worked in their Bellevue offices, Seattle is literally a 10-minute drive away.
Your points are definitely fair considerations, but even when I was working long hours at Microsoft's Redmond campus, I was very aware of what Seattle had to offer that I was missing out on.
It seems quite a shame that it took 6 years to discover the place next door. I'm glad he's discovered it now, though.
He alludes to his goals at Unity being done while as the same time finding less enjoyment in what he did day-to-day. There’s a lot missing but the last two paragraphs shed a bit more light.
> To really improve this industry we need to upgrade our culture, not our hardware, comp-sci curriculum, or office arrangements.
Shopping in Bellevue is better than Seattle. As for culture the popular parts of Seattle are as homogenous as Portland, and Bellevue at least has a sizable immigrant population from Eastern Europe, Asia, and Africa.
But yes, Bellevue is not the most happening place in the world.
More convenient for basics, yes, but more varied and fancy?
>As for culture the popular parts of Seattle are as homogenous as Portland, and Bellevue at least has a sizable immigrant population from Eastern Europe, Asia, and Africa.
But yes, Bellevue is not the most happening place in the world.
Yeah that. By culturally I meant places to visit and things to do (concerts, book readings, stage plays, parties, gatherings, etc), not just having neighbors of different origins one will usually seldom interact with.
No, you are missing nothing. But I can relate to the sentiment and point I think he is trying to make which is it's not worth it burning yourself out working on code all day, be it games or otherwise. Life seems much better when you start living it rather than just letting it pass you by while you are at work.
Regardless, I find the writing and style to be dreadful. I was hoping for something about the drawbacks of Unity the company or Unity itself as an engine. All I can find here is what amounts to a drunken urinal conversation.
Or it's just a message to his friends and colleagues that's not intended for wider distribution. Just a little public blip of "hey, I'm not at this company anymore, but I still exist and I'm doing something new".
Hacker News does this all the time: take something that's on a person's personal website/blog that might be intended for a limited or self-selecting audience, or meant as a joke or off-the-cuff remark and blow it way out of proportion.
26 comments
[ 0.23 ms ] story [ 62.7 ms ] threadThe industry has gotten better on that front in recent years at least. Unreal 4 is public source, you can make modifications or contribute back (you are just bound by the royalty fee and fact you cannot redistribute source yourself) and the asset stores of Unity / Unreal / etc, while 99% garbage, at least are starting the industry on a path towards recycling rather than NIH waste at an alarming rate.
You want to talk about culture, think of what kinds of creative experiences we would have today if every game developer open sourced their engines and assets 5 years after publishing. They can keep the maps for their game, which would still give it unique retail value, but id alone created a huge gamut of free software games through the publication of their engine sources in the late 90s and early 2000s. Think of the possibilities if those resources were not inevitably lost in some discarded outdated mainframe a decade after the games release like most are, but instead were made useful long after they made their money off them for hobbyists or indies to appreciate the hours of work designers or programmers poured into these products.
Those habits feel like a super power now, because at this point I naturally think in those terms and build accordingly.
We have quite a decent toolkit for VR at this point, but now I have the challenge of how to share any of it, because we do use a couple commercial libraries from the Unity asset store, and I don't know that there's any way to mark those as dependencies.
For example I've written various entity/component allocators, pools, and free-list implementations that run ultra-fast and are reusable across almost any game that needs an entity system, but almost each one is optimized for the performance characteristics of the target game. When it comes to game dev, you generally can get much better performance by writing things for the purpose. Usually reusability and performance are at odds in these cases. That might sound counter-intuitive to some in the web world, but it's a fact.
Even in website, you are better off doing things for example like pooling in garbage collected languages, avoiding extra allocations, and using more primitive data structures. This of course makes the code harder to read, test, maintain, etc. so developers in these cases tend not to do it or even be ignorant of these techniques. In games, you do things all the time for your specific use-cases and this really impacts testing and reusability.
It sounds counter-intuitive, but over the years I've learned in most cases reusing things in games is a fantasy only inexperienced developers get hung-up on. That doesn't mean nothing gets reused, rather what tends to happen is you learn from your mistakes/drawbacks of previous code and rewrite it better. In effect, you are reusing the code but it's not a literal drag and drop/cut and paste into a new project. Over time, if something is really solid and doesn't have too many performance/usability drawbacks, it may finally make its way into some sort of cross-project more drop-in library. An example I've directly used EA STL. Even when you reuse something though, people tend to do the equivalent of fork it per project, again limiting reusability.
I suppose in the age of better hardware with tons more ram, some of these concerns may be going away if your target platform is PC, but with mobile, more resource starved vs pc (ex: Nintendo 3ds) and embedded-like platforms gaining popularity, inevitably things circle back to the direction of optimizing per game and per hardware which hurts reusability. Is it really that valuable for example to dig up some library you used one 1 game in the lifecycle of the ps3 when you've already moved to the ps4 for example? Probably not for most developers unless you have a large window between your game start time and the platform lifecycle. Even on PC this is a problem as everything from graphics APIs to average hardware can obsolete a lot of what you've done or make it relatively inefficient/primitive in a new project.
But I've seen the rationale that games require too much optimization by a few indie devs as to why higher-level abstractions aren't used (not all, of course, and in some cases it's true), and that's where I think some indies that think in terms of reusable abstractions can gain an edge. It's a culturally held belief that I think sometimes gets adopted by new devs who then won't end up spending the time learning the abstractions.
I'm no design pattern purist by any stretch, but you can often find ways of cutting corners while still keeping things abstract enough to be reusable. Documenting goes a long way towards that too, as does a culture of automated testing, and a better package manager with dependencies.
Firstly, the reason why people didn't/don't use the STL is for a long time it was garbage for games. The data structures and allocators were huge offenders. If you've done any real game development, especially in the more resource constrained days, used std::vector was terribly inefficient and made no sense. This is why studios all wrote a lot of the same lower-level code. As to why they didn't share this code, it's just the nature of the business, however a lot of the performance of certain libraries over the years was highly tuned to the individual game.
Generally, games don't generate much reusability. Formats for assets change over the years, compression improves, resolution goes up, music chips improved, and so on. Even today things might have slowed-down, but programs used to produce assets make the level of effort of rehashing old assets not so worthwhile. Game studios that have tried to reuse things were often criticized for recycling when detected. The irony here is that this is actually justification for what you are saying - the assets aren't worth much long-term, so you might as well give them away along with the editors and level construction.
An important thing you're forgetting though is a lot of editors and tools used in game development are very one-off and shoddy. As a studio, you simply don't have time to sit there and write a perfect UI or even unity-like UI for your tools. Sure, it happens sometimes, especially when a tool is upgraded game-by-game, but generally tools are fire and forget and often given to the lower-skilled team members. I've rarely worked with game dev tools that were not full of bugs and woefully incomplete, but this is just the nature of the beast when you have limited time and budget - you don't spend it all on making perfect tools. This alone I suspect prevents many studios from releasing things to the public because the tools are very fragile, buggy, imperfect, incomplete, and so-on that it's almost embarrassing for some to see the light of day and too much maintenance for the benefit for others.
Regarding engines, I'm not sure what the gaming public's obsession with engines is. For well-tooled and documented engines, you have various degrees of offerings these days ranging from Unreal to CryEngine to Unity to other smaller, but still good engines. Those games with "custom" engines are probably no where near as useful. Sure, releasing the source would allow customizing and modding an individual game further, but again there's a lot of factors that go into this ranging from incomplete/broken pieces to game-specific hacks to lack of testing and so on. Unless a studio is in the business of releasing game engines and tools, it's hard to justify the risks. Often the copyrights and other things are not even owned anyway by the people who want to release them, so there's nothing you can do for example when your parent company or publisher views each and every line of code as some secret intellectual property.
What really bugs me though is why people think that game engines are some magical thing. What is a game engine even? In my experience, in many game codebases it is hard to draw a line between "engine" and "game." Sure you can set out to do this from the start in your architecture, but most of the time it won't happen that way when you are under the gun. The best you can do maybe is circle back and architect/extract things better post-release, which no one generally gets paid to do (unless selling the engine). Even then, the engine is not some magical thing that makes the entire game work. An engine can be anything from a way of handling assets + some graphics libraries and input handling to only graphics libraries to a stack of glued toge...
Is the "success" of id Software an obvious result of open-source acculteration or is it an anomaly? I'd say the latter...
1. We don't re-invent everything, but the hardware keeps getting better and we want to take advantage of it. To do so requires rewriting large portions of code. It is only because the basic hardware capabilities everyone has these days are pretty good, that using an engine not designed for your game has become so much more viable. Previously you'd build the engine to fit the game in order to get the performance you expect.
Shader's for example went through a massive upheaval when we changed from fixed function to custom, and now compute shaders have changed things again. Additionally there are always new algorithms presented each year which produce better results, which we must have!
2. Code that works well in an event based system does not necessarily work well when you are running it 60 times a second. This is why companies sometimes go in and replace standard functionality. We don't want to, but to get the speed everyone expects, we have to.
3. Models, textures, animations, dialog and so on are data, which the computer doesn't really understand. Where we see a person it sees a triangle mesh, or where we see signs of a bloody battle it just sees red pixels. This semantic mismatch is the biggest problem in the industry today, and makes re-usability incredibly hard. Manpower is usually the only way to overcome this, with people spending months on a single character model. Additionally better tools are always welcome but the industry has stalemated a bit in this regard.
Generally animation is still best imported from real life, EA for example have an entire building devoted to just motion capture [1] with over 100 cameras.
4. Many old games are still available on GOG and other websites, so these companies still make money from them which likely influences their decision to not release source code. Console games also are restricted by proprietary console-only APIs.
[1] http://www.capturelab.com/about-us
This isn't actually a thing at all. All the id games still sell like hotcakes moreso now, because of the open source engines that keep those games modern and have dramatically improved over the originals (check out Brutal Doom for an example).
Even if your engines and your art assets are liberated, you still do not have a complete game without maps. And while the user community built up around your product would certainly make their own, the fact they are all created on the basis of your game would, almost always, drive dramatically more long-tail sales to just play your original maps that inspired an entire ecosystem than locking up the complete package to rot in video game obscurity like the thousands of Playstation 2 games are doing now.
The rest of the article is even more confusing. There is not much substance.
When he had to live elsewhere in Washington due to work, he only had few chances to visit it (those "3 times in 6 years"), so he didn't get the chance to appreciate it back then.
What's confusing about it?
(Besides the fact that even if somebody already loved some place very much, there's nothing contradictory about them still only managing to visit it only a few times over 6 years, or even not once -- if they had too much other stuff to do. There are people who haven't took a holiday in 10 years, much less 6 -- and even when they did, they might go somewhere else, e.g. to somewhere they have family, or to some beach ladden place).
Your points are definitely fair considerations, but even when I was working long hours at Microsoft's Redmond campus, I was very aware of what Seattle had to offer that I was missing out on.
It seems quite a shame that it took 6 years to discover the place next door. I'm glad he's discovered it now, though.
I feel like I'm missing something.
> To really improve this industry we need to upgrade our culture, not our hardware, comp-sci curriculum, or office arrangements.
But yes, Bellevue is not the most happening place in the world.
More convenient for basics, yes, but more varied and fancy?
>As for culture the popular parts of Seattle are as homogenous as Portland, and Bellevue at least has a sizable immigrant population from Eastern Europe, Asia, and Africa. But yes, Bellevue is not the most happening place in the world.
Yeah that. By culturally I meant places to visit and things to do (concerts, book readings, stage plays, parties, gatherings, etc), not just having neighbors of different origins one will usually seldom interact with.
Regardless, I find the writing and style to be dreadful. I was hoping for something about the drawbacks of Unity the company or Unity itself as an engine. All I can find here is what amounts to a drunken urinal conversation.
Hacker News does this all the time: take something that's on a person's personal website/blog that might be intended for a limited or self-selecting audience, or meant as a joke or off-the-cuff remark and blow it way out of proportion.