Somewhat surprising there is no mention of basic design principles, or understanding the quirks of human perception. My brother was a production artist for some well-known computer games in the '90s-'00s, and continually complained about programmers and managers with zero visual sense, or curiosity about understanding the artists' side.
Graphics aren't my specialty, but as a musician, sound designer and producer, by far the most effective/influential audio DSP coders I'm aware of understand the basics of music, the physics/acoustics of sounds, and the gotchas at the interface between discrete digital processes and how we perceive and interpret stimuli.
I would say being a long-time user of Photoshop and Blender helps a lot. It's not a main tool, but supplemental. Maybe AI will take over some of this though.
trigonometry->Coordinate Geometry -> Linear Algebra applied to graphics
Once you have that intuition, the rest is all figuring out the stages of the graphics pipeline and the frameworks like opengl and their constituent data structures.
Today, I would not recommend anybody to go into graphics programming:
I started in 2001, when NVidias first Geforce 1 ("the Gigatexl shadercard") was first announced:
The field developed since then with so much speed and innovations, it blows my mind of. Compared to what we could do 25years ago, the tech today is just fu*ing impressive.
Though, with this impressiveness comes a big "but": The space is developing at a speed which is really really scary. Nvidia came up with AI-based effects to influence scene & assets on their own - back then, we wouldnt have even thought about that this will be possible some day in realtime.
I do not know if its possible at all to be a "decent pro" in this field now - let me use other words: "Where is todays Jon Carmack?" - he was famous for squeezing everything out of the hardware, using ideas very hidden in the community etc. - today, there is not any competitive moat for people like him (he actually lives on his legacy), and that is because the field is so vast and evolving so fast that there is no chance to become the next one
The most useful resources I've found for graphics are scratchapixel, UC Davis' graphics lectures, songho's articles, and Essential Math for Games and Interactive Applications. I highly recommend you read this last resource front to back. Seriously, its the best freaking math reference for graphics out there.
But knowing theory is not sufficient. You also need to get your hands dirty by writing code: learn how to build a software rasterizer (check out Tiny renderer) and a ray tracer (Ray Tracing in One Weekend series). Preferably in a language like C++. Then move onto APIs. I recommend you learn OpenGL, but if you're okay with being confused as all hell try Vulkan. Or WebGPU if you're a hipster (/s).
Finally, try to build some stuff. A simple engine. A non-trivial technique. A game. Whatever.
Unfortunately, you're unlikely to get hired working as a rendering engineer without having serious connections, or by having adjacent experience in the industry. Doubly so now that everyone is convinced junior engineers are unnecessary.
Feels like we try to turn anything we do into a career or job, especially with the odd ML angle. How about you "do graphics programming" instead of "being a graphics programmer"? Like start doing simple stuff until it clicks and you see it for being logistics to the GPU, then you can layer on top all the crazy concepts. Its like a small mountain you climb and suddenly everything clicks and you think like "oh my"... the possibilities and things to experiment with...
I created and still maintain A-Frame (aframe.io). It’s been a gentle gateway to learn 3D graphics for a decade. Cool community if I can say so ha ha. Web is a great way to share stuff as you learn, collect feedback and get visibility. Many cases in the community of people that ended up doing 3d graphics professionally.
My only additions to the article would be to study your probability/statistics (can't do efficient path tracing without it) and get comfortable with integrals, especially integrals on a sphere (physically based rendering will be a lot easier to understand).
I had some difficulties making sense of _color_ in context of graphics programming, especially transfer functions (sometimes misleadingly called tone mapping).
Do you want to make games, or do 3D engine programming?
If you want to make games, use an existing engine. Unreal Engine, Unity, Godot, and Bevy are good choices. You'll learn the higher level issues of graphics, not how to push pixels around. The real problem is making it fun.
If you want to do 3D engine programming, be aware that there are too many bad game engines. In Rust land, where I am, there are three failed renderers, one unfinished one, and the one inside the Bevy engine. Those are the major projects. There are many other "I'm going to build a game engine" projects. Building a game engine takes about two years to get to the My First Renderer point. Getting to big, highly detailed, dynamic scenes is a much bigger job. Be aware of the scale difference between the first demo and a useful engine.
If you want a job, be aware that the game industry sucks. Pay is lousy, hours are lousy, jobs end when the project is completed, and, like Hollywood, there's an army of wannabees wanting in. Also, right now, because of the collapse of the Metaverse thing, there's a glut of experienced people.
Then there's mobile. Everything is a cram job. Not enough screen, not enough compute, not enough GPU, not enough battery.
This is why most indy games now are 2D. That's do-able. Often in HTML/JavaScript.
> Do you want to make games, or do 3D engine programming?
Yup. If you start making an engine, you probably won't make a game - especially if you're learning along the way. It's technically possible to succeed at both, but having gone through this process many years ago, and having watched dozens of others in our Polish hobbyist gamedev community do the same, chances are under 5%.
"I'll do an engine for my game first, so it's easier to make the next game" - it's a surprisingly strong trap, because you are actually learning important things and winning small victories every day. There's always another win around the corner. Just one more unroll so the scene looks smoothly. Just one more logic layer to the config format, so it's easier to make the scene. Just one more SIGGRAPH paper.
There's always something important to improve. None of that adds up to a complete game. In retrospect, I'd say, writing your own engine is a perfect way for technical people to procrastinate on the hard but necessary parts of making their dream game - "making it fun", as you mentioned. You end up mastering all the skills that add up to building an impressive video game. You never learn how to make a game.
At least personally, the point of graphics programming or making "engines" is not to make games. It is to make real-time graphics, implement interesting approaches etc. If you are making a game instead of an engine, you will probably never focus on the graphics / rendering part, and instead need to focus on gameplay, 3D modeling etc. For many people this is explicitly not what they want: They want to make engines, not games.
Games industry might suck on average, but I'd argue that graphics programming niche does not. There are lots more users of graphics than games, like visualization, simulation, etc. Coupled with the fact that good graphics programmers are extremely rare, it's a surprisingly good career. This is in very stark contrast to game developers, artists and so on, which seem to have much harder time getting good quality jobs. Of course, both job market supply and demand is small, so changing jobs might not be easy.
I'd very much argue against somebody making game development their career, at least from job security grounds. But graphics programming is different.
I think you assume most people want to compete with Unreal on visuals. That's of course borderline impossible. But getting a basic renderer and game loop going isn't really that hard, and likely won't be the majority of your game's code. Just doing drawObject() in a for loop is good enough, you don't have to think hard about optimizng if your game is simple enough, or all the concerns about resource streaming, binding optimizations or parallelism can come later if necessary.
> Building a game engine takes about two years to get to the My First Renderer point.
I wonder what starting point and success criteria do you assume for that 2 year timeline. About a year ago I wrote a deferred renderer with dynamic lights, and shadow mapping, and few post effects in about a month of free time (Less than a week of full time work).
> This is why most indy games now are 2D. That's do-able. Often in HTML/JavaScript.
I don't think that should count against them. Most serious works is done on 2D interfaces. And both WebGL and plain old 2d canvas are very capable nowadays. And quite a few hit indies are 2D.
> If you want a job, be aware that the game industry sucks
Sure but basically everything uses the GPU nowadays. Writing and debugging ML workloads, data visualizations, HUDs and just user interfaces in general requires some understanding more often than not.
> I wonder what starting point and success criteria do you assume for that 2 year timeline. About a year ago I wrote a deferred renderer with dynamic lights, and shadow mapping, and few post effects in about a month of free time (Less than a week of full time work).
i'd assume it means fully understanding what you're building, if you already know 3d graphics or are just copying example code around, sure you can build something quick, but to fully understand the concepts from a background without previous graphics knowledge takes time and dedication
I work professionally on game engines and also have my own custom engine. Watching the space for 15+ years, I would say that nearly _every_ custom game engine exists as a demonstration of graphics capability, not anything to do with improving the actual experience of game development. This is in part to do with the fact that working on 3D at all is a slippery slope towards continuing to work on graphics stuff.
Related to graphics programming in rust: One reason to get into graphics programming is if there aren't any engines that suit your purpose that are available for your programming language of choice.
If I wanted to program a 3D game, I would choose UE5. I'm not, however: I made my own engine (`graphics` crate) and own vector/quatrnion lib (`lin-alg` crate), used by no one other than myself, to view chemistry and molecular bio visuals, with convenience controls etc. And it's suitable as a general purpose engine for rendering simple, non realistic scenes. I'm surprised I had to do this, but didn't find any suitable existing solutions.
> be aware that there are too many bad game engines. In Rust land, where I am, there are three failed renderers, one unfinished one, and the one inside the Bevy engine. Those are the major projects.
I think the advent of world models is going to open up a lot of interesting 3D applications with related graphics & rendering challenges. That intersecting with WebGPU general availability across browsers IMO makes graphics programming a very interesting domain to get into now. I certainly see the need in my dayjob.
My focus area in college was Computer Graphics. There is not enough focus about the math in this article, it just kind of passively mentions it. "Well you can get by with just a little bit of this and that" -- Linear Algebra is huge! So is an Engineering style Calculus course -- not your business calc. Those two require a year of their own to gain mastery.
IMO, pick up:
Linear Alegbra Done Right Calculus Better Explained Concrete Mathmetics.
I come from reading about CRDTs from Evan Wallace and also having built a product used by >40M users.
It applies to software products too!
In their words…
If you want to build products, use React or even vibecode; you will learn higher-level issues of solutions to problems (i.e. people problems rather than machine problems), not how to push data/state/computation around. The problem is solving a need.
Neither is good nor bad; just be clear about your goals and then it’ll be easy to decide if you want to follow Zynga’s cofounder, Jonathan Blow, or Notch! And before you rush to answer… consider whether any of them are happy.
For people who recommend against learning these skills because “what Carmack did is not possible anymore.”… well, if what you look for is money then yeah! But, if you just want to learn for the love of the game, then that would be a very bad advice!
Here's my created list I maintain: https://legends2k.github.io/note/cg_resources/
Do learn if it piques your curiosity and have the time. You're in for a blast and a lot of learning, that'll make you a better engineer, in many other fields of computer science too (you understand hardware, system programming, programmer's machine model, etc.).
Don't learn if you do it with a monetary end goal as it's fleeting, ephemeral and not guaranteed in this day and age.
50 comments
[ 6.2 ms ] story [ 117 ms ] threadGraphics aren't my specialty, but as a musician, sound designer and producer, by far the most effective/influential audio DSP coders I'm aware of understand the basics of music, the physics/acoustics of sounds, and the gotchas at the interface between discrete digital processes and how we perceive and interpret stimuli.
This guy has some good art: https://www.khanacademy.org/profile/kaid_1019042693170894950...
If you want to work with Windows, probably DirectX.
2. Make awesome shaders. Check this out: https://fragcoord.xyz
I would say being a long-time user of Photoshop and Blender helps a lot. It's not a main tool, but supplemental. Maybe AI will take over some of this though.
Hell, maybe that other stuff too, hahaha!
Once you have that intuition, the rest is all figuring out the stages of the graphics pipeline and the frameworks like opengl and their constituent data structures.
I started in 2001, when NVidias first Geforce 1 ("the Gigatexl shadercard") was first announced: The field developed since then with so much speed and innovations, it blows my mind of. Compared to what we could do 25years ago, the tech today is just fu*ing impressive.
Though, with this impressiveness comes a big "but": The space is developing at a speed which is really really scary. Nvidia came up with AI-based effects to influence scene & assets on their own - back then, we wouldnt have even thought about that this will be possible some day in realtime.
I do not know if its possible at all to be a "decent pro" in this field now - let me use other words: "Where is todays Jon Carmack?" - he was famous for squeezing everything out of the hardware, using ideas very hidden in the community etc. - today, there is not any competitive moat for people like him (he actually lives on his legacy), and that is because the field is so vast and evolving so fast that there is no chance to become the next one
The most useful resources I've found for graphics are scratchapixel, UC Davis' graphics lectures, songho's articles, and Essential Math for Games and Interactive Applications. I highly recommend you read this last resource front to back. Seriously, its the best freaking math reference for graphics out there.
But knowing theory is not sufficient. You also need to get your hands dirty by writing code: learn how to build a software rasterizer (check out Tiny renderer) and a ray tracer (Ray Tracing in One Weekend series). Preferably in a language like C++. Then move onto APIs. I recommend you learn OpenGL, but if you're okay with being confused as all hell try Vulkan. Or WebGPU if you're a hipster (/s).
Finally, try to build some stuff. A simple engine. A non-trivial technique. A game. Whatever.
Unfortunately, you're unlikely to get hired working as a rendering engineer without having serious connections, or by having adjacent experience in the industry. Doubly so now that everyone is convinced junior engineers are unnecessary.
I also have some notebooks with SymPy code examples here: https://github.com/minireference/noBSLAnotebooks
The visualizations made things click in a way my Linear 101 course didn't.
I’m going to revisit raycasting with a browser based raycaster from scratch.
I’m just finishing up a webgl + canvas game engine and game for a 2D top down grid strategy game first
Good article on color management in general, which has a great intersection with foundations needed for graphics programming: https://chrisbrejon.com/cg-cinematography/chapter-1-color-ma...
If you want to make games, use an existing engine. Unreal Engine, Unity, Godot, and Bevy are good choices. You'll learn the higher level issues of graphics, not how to push pixels around. The real problem is making it fun.
If you want to do 3D engine programming, be aware that there are too many bad game engines. In Rust land, where I am, there are three failed renderers, one unfinished one, and the one inside the Bevy engine. Those are the major projects. There are many other "I'm going to build a game engine" projects. Building a game engine takes about two years to get to the My First Renderer point. Getting to big, highly detailed, dynamic scenes is a much bigger job. Be aware of the scale difference between the first demo and a useful engine.
If you want a job, be aware that the game industry sucks. Pay is lousy, hours are lousy, jobs end when the project is completed, and, like Hollywood, there's an army of wannabees wanting in. Also, right now, because of the collapse of the Metaverse thing, there's a glut of experienced people.
Then there's mobile. Everything is a cram job. Not enough screen, not enough compute, not enough GPU, not enough battery.
This is why most indy games now are 2D. That's do-able. Often in HTML/JavaScript.
Yup. If you start making an engine, you probably won't make a game - especially if you're learning along the way. It's technically possible to succeed at both, but having gone through this process many years ago, and having watched dozens of others in our Polish hobbyist gamedev community do the same, chances are under 5%.
"I'll do an engine for my game first, so it's easier to make the next game" - it's a surprisingly strong trap, because you are actually learning important things and winning small victories every day. There's always another win around the corner. Just one more unroll so the scene looks smoothly. Just one more logic layer to the config format, so it's easier to make the scene. Just one more SIGGRAPH paper.
There's always something important to improve. None of that adds up to a complete game. In retrospect, I'd say, writing your own engine is a perfect way for technical people to procrastinate on the hard but necessary parts of making their dream game - "making it fun", as you mentioned. You end up mastering all the skills that add up to building an impressive video game. You never learn how to make a game.
I'd very much argue against somebody making game development their career, at least from job security grounds. But graphics programming is different.
It can pay quite well at large studios or tech companies if you work outside of games and the hours aren’t necessarily bad, it depends on the project
If it's all for a hobby, creating your own engine is a lot of fun by itself.
> Building a game engine takes about two years to get to the My First Renderer point.
I wonder what starting point and success criteria do you assume for that 2 year timeline. About a year ago I wrote a deferred renderer with dynamic lights, and shadow mapping, and few post effects in about a month of free time (Less than a week of full time work).
> This is why most indy games now are 2D. That's do-able. Often in HTML/JavaScript.
I don't think that should count against them. Most serious works is done on 2D interfaces. And both WebGL and plain old 2d canvas are very capable nowadays. And quite a few hit indies are 2D.
> If you want a job, be aware that the game industry sucks
Sure but basically everything uses the GPU nowadays. Writing and debugging ML workloads, data visualizations, HUDs and just user interfaces in general requires some understanding more often than not.
i'd assume it means fully understanding what you're building, if you already know 3d graphics or are just copying example code around, sure you can build something quick, but to fully understand the concepts from a background without previous graphics knowledge takes time and dedication
If I wanted to program a 3D game, I would choose UE5. I'm not, however: I made my own engine (`graphics` crate) and own vector/quatrnion lib (`lin-alg` crate), used by no one other than myself, to view chemistry and molecular bio visuals, with convenience controls etc. And it's suitable as a general purpose engine for rendering simple, non realistic scenes. I'm surprised I had to do this, but didn't find any suitable existing solutions.
Where does Fyrox sit in this?
Linear Alegbra Done Right Calculus Better Explained Concrete Mathmetics.
Then you can move on to the low level APIs.
Make the bold choice. Be a game network programmer. Nobody wants to do it, it's really hard and it kinda sucks.
Play the accordion :)
Perhaps I'll write such a book... after I finish my game (dry laughter).
[0]: https://thebookofshaders.com/
I come from reading about CRDTs from Evan Wallace and also having built a product used by >40M users.
It applies to software products too!
In their words…
If you want to build products, use React or even vibecode; you will learn higher-level issues of solutions to problems (i.e. people problems rather than machine problems), not how to push data/state/computation around. The problem is solving a need.
Neither is good nor bad; just be clear about your goals and then it’ll be easy to decide if you want to follow Zynga’s cofounder, Jonathan Blow, or Notch! And before you rush to answer… consider whether any of them are happy.
For people who recommend against learning these skills because “what Carmack did is not possible anymore.”… well, if what you look for is money then yeah! But, if you just want to learn for the love of the game, then that would be a very bad advice!
Don't learn if you do it with a monetary end goal as it's fleeting, ephemeral and not guaranteed in this day and age.