Somewhat surprising there is no mention of basic design principles, or understanding the quirks of human perception. My brother was a production artist for some well-known computer games in the '90s-'00s, and continually complained about programmers and managers with zero visual sense, or curiosity about understanding the artists' side.
Graphics aren't my specialty, but as a musician, sound designer and producer, by far the most effective/influential audio DSP coders I'm aware of understand the basics of music, the physics/acoustics of sounds, and the gotchas at the interface between discrete digital processes and how we perceive and interpret stimuli.
I would say being a long-time user of Photoshop and Blender helps a lot. It's not a main tool, but supplemental. Maybe AI will take over some of this though.
trigonometry->Coordinate Geometry -> Linear Algebra applied to graphics
Once you have that intuition, the rest is all figuring out the stages of the graphics pipeline and the frameworks like opengl and their constituent data structures.
Today, I would not recommend anybody to go into graphics programming:
I started in 2001, when NVidias first Geforce 1 ("the Gigatexl shadercard") was first announced:
The field developed since then with so much speed and innovations, it blows my mind of. Compared to what we could do 25years ago, the tech today is just fu*ing impressive.
Though, with this impressiveness comes a big "but": The space is developing at a speed which is really really scary. Nvidia came up with AI-based effects to influence scene & assets on their own - back then, we wouldnt have even thought about that this will be possible some day in realtime.
I do not know if its possible at all to be a "decent pro" in this field now - let me use other words: "Where is todays Jon Carmack?" - he was famous for squeezing everything out of the hardware, using ideas very hidden in the community etc. - today, there is not any competitive moat for people like him (he actually lives on his legacy), and that is because the field is so vast and evolving so fast that there is no chance to become the next one
The most useful resources I've found for graphics are scratchapixel, UC Davis' graphics lectures, songho's articles, and Essential Math for Games and Interactive Applications. I highly recommend you read this last resource front to back. Seriously, its the best freaking math reference for graphics out there.
But knowing theory is not sufficient. You also need to get your hands dirty by writing code: learn how to build a software rasterizer (check out Tiny renderer) and a ray tracer (Ray Tracing in One Weekend series). Preferably in a language like C++. Then move onto APIs. I recommend you learn OpenGL, but if you're okay with being confused as all hell try Vulkan. Or WebGPU if you're a hipster (/s).
Finally, try to build some stuff. A simple engine. A non-trivial technique. A game. Whatever.
Unfortunately, you're unlikely to get hired working as a rendering engineer without having serious connections, or by having adjacent experience in the industry. Doubly so now that everyone is convinced junior engineers are unnecessary.
Feels like we try to turn anything we do into a career or job, especially with the odd ML angle. How about you "do graphics programming" instead of "being a graphics programmer"? Like start doing simple stuff until it clicks and you see it for being logistics to the GPU, then you can layer on top all the crazy concepts. Its like a small mountain you climb and suddenly everything clicks and you think like "oh my"... the possibilities and things to experiment with...
I created and still maintain A-Frame (aframe.io). It’s been a gentle gateway to learn 3D graphics for a decade. Cool community if I can say so ha ha. Web is a great way to share stuff as you learn, collect feedback and get visibility. Many cases in the community of people that ended up doing 3d graphics professionally.
My only additions to the article would be to study your probability/statistics (can't do efficient path tracing without it) and get comfortable with integrals, especially integrals on a sphere (physically based rendering will be a lot easier to understand).
I had some difficulties making sense of _color_ in context of graphics programming, especially transfer functions (sometimes misleadingly called tone mapping).
Do you want to make games, or do 3D engine programming?
If you want to make games, use an existing engine. Unreal Engine, Unity, Godot, and Bevy are good choices. You'll learn the higher level issues of graphics, not how to push pixels around. The real problem is making it fun.
If you want to do 3D engine programming, be aware that there are too many bad game engines. In Rust land, where I am, there are three failed renderers, one unfinished one, and the one inside the Bevy engine. Those are the major projects. There are many other "I'm going to build a game engine" projects. Building a game engine takes about two years to get to the My First Renderer point. Getting to big, highly detailed, dynamic scenes is a much bigger job. Be aware of the scale difference between the first demo and a useful engine.
If you want a job, be aware that the game industry sucks. Pay is lousy, hours are lousy, jobs end when the project is completed, and, like Hollywood, there's an army of wannabees wanting in. Also, right now, because of the collapse of the Metaverse thing, there's a glut of experienced people.
Then there's mobile. Everything is a cram job. Not enough screen, not enough compute, not enough GPU, not enough battery.
This is why most indy games now are 2D. That's do-able. Often in HTML/JavaScript.
I think you assume most people want to compete with Unreal on visuals. That's of course borderline impossible. But getting a basic renderer and game loop going isn't really that hard, and likely won't be the majority of your game's code. Just doing drawObject() in a for loop is good enough, you don't have to think hard about optimizng if your game is simple enough, or all the concerns about resource streaming, binding optimizations or parallelism can come later if necessary.
> Building a game engine takes about two years to get to the My First Renderer point.
I wonder what starting point and success criteria do you assume for that 2 year timeline. About a year ago I wrote a deferred renderer with dynamic lights, and shadow mapping, and few post effects in about a month of free time (Less than a week of full time work).
> This is why most indy games now are 2D. That's do-able. Often in HTML/JavaScript.
I don't think that should count against them. Most serious works is done on 2D interfaces. And both WebGL and plain old 2d canvas are very capable nowadays. And quite a few hit indies are 2D.
> If you want a job, be aware that the game industry sucks
Sure but basically everything uses the GPU nowadays. Writing and debugging ML workloads, data visualizations, HUDs and just user interfaces in general requires some understanding more often than not.
Related to graphics programming in rust: One reason to get into graphics programming is if there aren't any engines that suit your purpose that are available for your programming language of choice.
If I wanted to program a 3D game, I would choose UE5. I'm not, however: I made my own engine (`graphics` crate) and own vector/quatrnion lib (`lin-alg` crate), used by no one other than myself, to view chemistry and molecular bio visuals, with convenience controls etc. And it's suitable as a general purpose engine for rendering simple, non realistic scenes. I'm surprised I had to do this, but didn't find any suitable existing solutions.
> be aware that there are too many bad game engines. In Rust land, where I am, there are three failed renderers, one unfinished one, and the one inside the Bevy engine. Those are the major projects.
I think the advent of world models is going to open up a lot of interesting 3D applications with related graphics & rendering challenges. That intersecting with WebGPU general availability across browsers IMO makes graphics programming a very interesting domain to get into now. I certainly see the need in my dayjob.
My focus area in college was Computer Graphics. There is not enough focus about the math in this article, it just kind of passively mentions it. "Well you can get by with just a little bit of this and that" -- Linear Algebra is huge! So is an Engineering style Calculus course -- not your business calc. Those two require a year of their own to gain mastery.
IMO, pick up:
Linear Alegbra Done Right Calculus Better Explained Concrete Mathmetics.
I come from reading about CRDTs from Evan Wallace and also having built a product used by >40M users.
It applies to software products too!
In their words…
If you want to build products, use React or even vibecode; you will learn higher-level issues of solutions to problems (i.e. people problems rather than machine problems), not how to push data/state/computation around. The problem is solving a need.
Neither is good nor bad; just be clear about your goals and then it’ll be easy to decide if you want to follow Zynga’s cofounder, Jonathan Blow, or Notch! And before you rush to answer… consider whether any of them are happy.
For people who recommend against learning these skills because “what Carmack did is not possible anymore.”… well, if what you look for is money then yeah! But, if you just want to learn for the love of the game, then that would be a very bad advice!
Here's my created list I maintain: https://legends2k.github.io/note/cg_resources/
Do learn if it piques your curiosity and have the time. You're in for a blast and a lot of learning, that'll make you a better engineer, in many other fields of computer science too (you understand hardware, system programming, programmer's machine model, etc.).
Don't learn if you do it with a monetary end goal as it's fleeting, ephemeral and not guaranteed in this day and age.
37 comments
[ 3.0 ms ] story [ 64.0 ms ] threadGraphics aren't my specialty, but as a musician, sound designer and producer, by far the most effective/influential audio DSP coders I'm aware of understand the basics of music, the physics/acoustics of sounds, and the gotchas at the interface between discrete digital processes and how we perceive and interpret stimuli.
This guy has some good art: https://www.khanacademy.org/profile/kaid_1019042693170894950...
If you want to work with Windows, probably DirectX.
2. Make awesome shaders. Check this out: https://fragcoord.xyz
I would say being a long-time user of Photoshop and Blender helps a lot. It's not a main tool, but supplemental. Maybe AI will take over some of this though.
Hell, maybe that other stuff too, hahaha!
Once you have that intuition, the rest is all figuring out the stages of the graphics pipeline and the frameworks like opengl and their constituent data structures.
I started in 2001, when NVidias first Geforce 1 ("the Gigatexl shadercard") was first announced: The field developed since then with so much speed and innovations, it blows my mind of. Compared to what we could do 25years ago, the tech today is just fu*ing impressive.
Though, with this impressiveness comes a big "but": The space is developing at a speed which is really really scary. Nvidia came up with AI-based effects to influence scene & assets on their own - back then, we wouldnt have even thought about that this will be possible some day in realtime.
I do not know if its possible at all to be a "decent pro" in this field now - let me use other words: "Where is todays Jon Carmack?" - he was famous for squeezing everything out of the hardware, using ideas very hidden in the community etc. - today, there is not any competitive moat for people like him (he actually lives on his legacy), and that is because the field is so vast and evolving so fast that there is no chance to become the next one
The most useful resources I've found for graphics are scratchapixel, UC Davis' graphics lectures, songho's articles, and Essential Math for Games and Interactive Applications. I highly recommend you read this last resource front to back. Seriously, its the best freaking math reference for graphics out there.
But knowing theory is not sufficient. You also need to get your hands dirty by writing code: learn how to build a software rasterizer (check out Tiny renderer) and a ray tracer (Ray Tracing in One Weekend series). Preferably in a language like C++. Then move onto APIs. I recommend you learn OpenGL, but if you're okay with being confused as all hell try Vulkan. Or WebGPU if you're a hipster (/s).
Finally, try to build some stuff. A simple engine. A non-trivial technique. A game. Whatever.
Unfortunately, you're unlikely to get hired working as a rendering engineer without having serious connections, or by having adjacent experience in the industry. Doubly so now that everyone is convinced junior engineers are unnecessary.
I also have some notebooks with SymPy code examples here: https://github.com/minireference/noBSLAnotebooks
The visualizations made things click in a way my Linear 101 course didn't.
I’m going to revisit raycasting with a browser based raycaster from scratch.
I’m just finishing up a webgl + canvas game engine and game for a 2D top down grid strategy game first
Good article on color management in general, which has a great intersection with foundations needed for graphics programming: https://chrisbrejon.com/cg-cinematography/chapter-1-color-ma...
If you want to make games, use an existing engine. Unreal Engine, Unity, Godot, and Bevy are good choices. You'll learn the higher level issues of graphics, not how to push pixels around. The real problem is making it fun.
If you want to do 3D engine programming, be aware that there are too many bad game engines. In Rust land, where I am, there are three failed renderers, one unfinished one, and the one inside the Bevy engine. Those are the major projects. There are many other "I'm going to build a game engine" projects. Building a game engine takes about two years to get to the My First Renderer point. Getting to big, highly detailed, dynamic scenes is a much bigger job. Be aware of the scale difference between the first demo and a useful engine.
If you want a job, be aware that the game industry sucks. Pay is lousy, hours are lousy, jobs end when the project is completed, and, like Hollywood, there's an army of wannabees wanting in. Also, right now, because of the collapse of the Metaverse thing, there's a glut of experienced people.
Then there's mobile. Everything is a cram job. Not enough screen, not enough compute, not enough GPU, not enough battery.
This is why most indy games now are 2D. That's do-able. Often in HTML/JavaScript.
> Building a game engine takes about two years to get to the My First Renderer point.
I wonder what starting point and success criteria do you assume for that 2 year timeline. About a year ago I wrote a deferred renderer with dynamic lights, and shadow mapping, and few post effects in about a month of free time (Less than a week of full time work).
> This is why most indy games now are 2D. That's do-able. Often in HTML/JavaScript.
I don't think that should count against them. Most serious works is done on 2D interfaces. And both WebGL and plain old 2d canvas are very capable nowadays. And quite a few hit indies are 2D.
> If you want a job, be aware that the game industry sucks
Sure but basically everything uses the GPU nowadays. Writing and debugging ML workloads, data visualizations, HUDs and just user interfaces in general requires some understanding more often than not.
If I wanted to program a 3D game, I would choose UE5. I'm not, however: I made my own engine (`graphics` crate) and own vector/quatrnion lib (`lin-alg` crate), used by no one other than myself, to view chemistry and molecular bio visuals, with convenience controls etc. And it's suitable as a general purpose engine for rendering simple, non realistic scenes. I'm surprised I had to do this, but didn't find any suitable existing solutions.
Where does Fyrox sit in this?
Linear Alegbra Done Right Calculus Better Explained Concrete Mathmetics.
Then you can move on to the low level APIs.
Make the bold choice. Be a game network programmer. Nobody wants to do it, it's really hard and it kinda sucks.
Play the accordion :)
Perhaps I'll write such a book... after I finish my game (dry laughter).
[0]: https://thebookofshaders.com/
I come from reading about CRDTs from Evan Wallace and also having built a product used by >40M users.
It applies to software products too!
In their words…
If you want to build products, use React or even vibecode; you will learn higher-level issues of solutions to problems (i.e. people problems rather than machine problems), not how to push data/state/computation around. The problem is solving a need.
Neither is good nor bad; just be clear about your goals and then it’ll be easy to decide if you want to follow Zynga’s cofounder, Jonathan Blow, or Notch! And before you rush to answer… consider whether any of them are happy.
For people who recommend against learning these skills because “what Carmack did is not possible anymore.”… well, if what you look for is money then yeah! But, if you just want to learn for the love of the game, then that would be a very bad advice!
Don't learn if you do it with a monetary end goal as it's fleeting, ephemeral and not guaranteed in this day and age.