As someone with no skin the game in either direciton, it's interesting that you're so triggered by the mere mention of someone's name.
Is it really so difficult to parse out different aspects of a person's existence, such that one could simultaneously appreciate some of the positives (the existence and success of SpaceX, making Tesla a success against the odds, [popularisation of] some engineering leadership concepts) while also acknowledging the negatives (of which, more latterly, there are many)?
Or must the sheer emotion always cloud out rational judgement?
Lots of words and weird analogies to say basically nothing.
What is the status of the project? What can it do? What has it achieved in 5 years?
But no, let's highlight how we follow the "Elon process".
As a side note, whenever someone incessantly focuses on lines of code as a metric (in either direction), I immediately start to take them less seriously.
The third sentence is "The codebase is 18,935 lines not including tests." You can go to the repo and see what it does. Look at issues, PRs, etc. He wasn't saying it to brag about its size, but the opposite.
The mission is to commoditize the petaflop. Basically allow LLMs to be trained efficiently on commodity non-NVIDIA GPUs. Would you prefer some bullshit mission like Meta of "connecting people" or whatever the hell it is?
He said he has a hardware division that makes $2m a year. You can click on the link and buy a computer. He tells you the revenue.
He said he has a deal with AMD which is also public and on X.
The "Elon process" relies specifically to the goal of getting rid of all dependencies. Musk has spoken extensively about building things from the ground up and not relying on other vendors (in this example complex software dependencies). He says he wouldn't be able to build SpaceX competitively if he had just bought rockets or components.
I wish more people focused on public code and shipping publicly. Can I see Toyotas stack? Why does the touch screen perform worse than a 2012 iPad. What the hell is going on
I don't understand this hate someone like Hotz, a true engineer running an ambitious very open company, receives on a ... checks notes ... engineering forum? The whole setup is like a wet dream for engineers from a decade ago.
I hope Mitchell Hashimoto with his billions will join forces with Geohotz. I mean ghostty is fine and all. But we are talking here about minority report UI like future.
It’s hard to argue against hiring contributors, but a bounty system that pays pennies vs. market value for skilled developers shouldn’t be the only interview path, it’s borderline exploitative.
If you think you can do better, the code's right there and you're welcome to try! There's a giant pile of money at the end of the rainbow for you if you can uncomplexify it.
Looking at their boxes, I'd guess 20-ish%, once the labor costs of designing the boxes is out of the picture. I imagine George doesn't take a salary, or at least takes only a small one. They have a contract with AMD as well. I'd bet the company is modestly profitable.
Not sure where it was from but I remember reading a comment about how every modern rocket scientist needs to come to terms with the fact that every novel idea they might have has already been proposed and tried by a Russian scientist in the 20th century.
The risk for Tinygrad is that PyTorch will create a new backend for Inductor, plug in their AMD codegen stuff and walala, PyTorch still king. I mean, they could have easily just taken that route themselves instead of bothering with a new ML framework and AD engine. 99% of the work is just the AMD codegen part of the compiler.
Either way, super cool project and I wish them the best.
tinygrad has a PyTorch frontend, we fully encourage the frontend to stay PyTorch and for them to integrate tinygrad at any level. Our mission is to commoditize the petaflop, all frontends are welcome. We also have a quite complete ONNX frontend.
>"We also have a contract with AMD to get MI350X on MLPerf for Llama 405B training."
Anything to help AMD (and potentially other GPU/NPU/IPU etc. chip makers) catch up with NVidia/CUDA is potentially worth money, potentially worth a lot of money, potentially worth up to Billion$...
Why?
If we have
a) Market worth Billion$
and
b) A competitive race in that Market...
then
c) We have VALUE in anything (product, service, ?, ???) that helps any given participant capture more of that market than their competitors...
(AMD (and the other lesser known GPU/NPU/IPU etc. chip vendors) are currently lagging behind NVidia's CUDA AI market dominance -- so anything that helps the others advance in this area should, generally speaking, be beneficial for all technology users in general, and be potentially profitable (if the correct deals could be struck!) by those that have the skills to do such assisting...)
Anyway, wishing you well in your endeavors, Tinygrad!
Is this the guy who talked a big game about all the things he was going to fix at Twitter, then utterly failed when confronted with a real world codebase and gave up having done nothing of use?
I've looked at the "only 18,935 lines of code" python code and it made me want to poke my eyes out. Not sure what's the point of this extreme code-golfing.
No code golf! While low line count is a guiding light of this project, anything that remotely looks like code golf will be closed. The true goal is reducing complexity and increasing readability, and deleting \ns does nothing to help with that.
If you want to "own" Nvidia, the much more realistic way of doing this then trying to compete with all the data centers that are already being built with Nvidia chips is obviously with open source models. In the case of open source models, inference is much more important to most people not training which a maxed out macbook already does a good job of.
How are ergonomics compared to pytorch, though? Adoption can be also driven by frictionless research (e.g. torch vs. tf comes to mind). Repo is missing proper docs aimed at early adopters imho
Aside from the edgy Unabomber quote and calling both established US political parties racists, everything there is reasonable analysis. What exactly got your goat?
I really hope tinygrad succeeds with their mission of commoditizing the petaflop. We're nearing a future where you own nothing and rent everything, and they are one of few companies pushing back. This combined with the focus on making models efficient through better architecture/training, not just throwing more compute at it, seems like the right direction imo.
I remember when he launched everything and was hoping that the AMD hardware (very capable) was just hamstrung by software. The idea that the easy part is actually the chips is something I never considered though I "know" that "Nvidia succeeded because of their software". Haha, very clever.
I've got a Comma 3X and I'm thankful for the 4090 P2P work too (which is now here? https://github.com/tinygrad/open-gpu-kernel-modules) so I'm excited to see it work. Rooting for the guy. Hope it's true that the chip part is the easy work. Could be that both are "the hard work".
The only thing that George got wrong was failing to realize that he is not an engineer, but an inventor. I am saying this as a fan of his work. He is something that I would dare to call a techno artist. He is the Kasparov of sw engineering, not Karpov. His ideas are like lightning. A powerful strike that shakes the earth (or Earth? maybe). But his agenda and his goals interfered with his God given power.
Ironically, you can hear him saying on a podcast with Friedman (dont' remember which episode) that one should not have a lofty goal, but then you have him talking all about such goals. The Saviour persona takes over (not saying he doesn't have that in him), but this does not suit him.
It should be instinct, rather than intellect that guides him (which is the case with every artist).
Think of the reason he got into hacking. From what I understand it was one summer, now long time ago, him being bored. That's it. Boredom mixed with curiosity and innate talent.
What got Kubrick into making films? Him seeing how bad the films he saw were. Him thinking he can do better. And he did.
A perfect example of motivation that was purely technical in nature. Problem solving skills that needed an outlet. That doesn't mean it is the only reason to get into films, far from it.
In words of Kubrick himself: “Some directors are primarily interested in working with actors, others are more interested in visual composition or camera movement. I don’t think there is one correct emphasis.”
We all know to which group Kubrick gravitated to (the latter of course, just like his idol Max Ophüls). My point? There was something specific that got him stared and not just: "I wanna make films". Take a look at what got him started filming 2001. It was not a book by Clarke, but this:
More precisely it were the special effects he saw from a Canadian documentary in the link above. Barry Lyndon? NASA lenses. Could you say the same for Scorsese? I don't think so. He would never. Ever. Do it for those reasons.
Moving on.
When N.Tesla invented induction motor did he do so for the improvement of the infrastructure and improving the lives of others? No. He did it because he wanted to improve DC motor. Or namely, the fact that he did not like the sparks that came from the brushes and that were caused by electrical arcing on commutator segments.
Did he care to perfect his invention? No.
Mikhail Dolivo-Dobrovolsky did.
N.Tesla could not care less about the materials used or the number of phases that would make the motor perform at it's best (even though he was invested in the production phase initially). Why? Because his mission was complete. He gave the idea. He built a working model. Then he moved on.
Just like Kubrick, his energy came from a desire to solve a problem.
No shame in moving on.
Imagine Kubrick calling it quits after Lolita.
Or N.Tesla devoting his energies on perfecting the induction engine.
Or Elon Musk working on electric cars and not caring about space travel.
Etc...
Not good.
Because there are people in this world made specifically for that role.
That would be the Torvald's of the world (L Torvalds: "I am not a visionary. I'm an engineer. I'm happy with the people who are wandering around looking at the stars but I am looking at the ground and I want to fix the pothole before I fall in"). The Karpov's. The Marcan's. The George R.R Martin's...
Not Nikola's, not Elon's or Steven King's. Not George's.
And just because visionaries look at the stars does not mean they have a lofty goal. Or that they are entitled. Visionary is the wiring of the brain. Those blessed with it should not go against it.
I could be wrong.
I am not that technical.
I am that educated.
I have not achieved much in life.
English is not my first language.
42 comments
[ 0.19 ms ] story [ 70.0 ms ] threadIs it really so difficult to parse out different aspects of a person's existence, such that one could simultaneously appreciate some of the positives (the existence and success of SpaceX, making Tesla a success against the odds, [popularisation of] some engineering leadership concepts) while also acknowledging the negatives (of which, more latterly, there are many)?
Or must the sheer emotion always cloud out rational judgement?
I remember when defcon ctf would play Geohot's PlayStation rap video every year on the wall.
What is the status of the project? What can it do? What has it achieved in 5 years?
But no, let's highlight how we follow the "Elon process".
As a side note, whenever someone incessantly focuses on lines of code as a metric (in either direction), I immediately start to take them less seriously.
The third sentence is "The codebase is 18,935 lines not including tests." You can go to the repo and see what it does. Look at issues, PRs, etc. He wasn't saying it to brag about its size, but the opposite.
The mission is to commoditize the petaflop. Basically allow LLMs to be trained efficiently on commodity non-NVIDIA GPUs. Would you prefer some bullshit mission like Meta of "connecting people" or whatever the hell it is?
He said he has a hardware division that makes $2m a year. You can click on the link and buy a computer. He tells you the revenue.
He said he has a deal with AMD which is also public and on X.
The "Elon process" relies specifically to the goal of getting rid of all dependencies. Musk has spoken extensively about building things from the ground up and not relying on other vendors (in this example complex software dependencies). He says he wouldn't be able to build SpaceX competitively if he had just bought rockets or components.
I wish more people focused on public code and shipping publicly. Can I see Toyotas stack? Why does the touch screen perform worse than a 2012 iPad. What the hell is going on
I don't understand this hate someone like Hotz, a true engineer running an ambitious very open company, receives on a ... checks notes ... engineering forum? The whole setup is like a wet dream for engineers from a decade ago.
Almost completely open source
Interview is completing a paid bounty
CEO is engineer
Has a clear mission, goals, and timeline
What is your issue?
I find this organizational structure compelling, probably the closest to reaching 100% productivity in a week as you can get.
(Love that talk tho)
What's the margin on that? Do 5 software engineers really subsist on the spread from moving $2M/yr in hardware?
That’s not Elon. See Russian TRIZ
https://en.wikipedia.org/wiki/TRIZ
Either way, super cool project and I wish them the best.
Is this like voila or something else?
Anything to help AMD (and potentially other GPU/NPU/IPU etc. chip makers) catch up with NVidia/CUDA is potentially worth money, potentially worth a lot of money, potentially worth up to Billion$...
Why?
If we have
a) Market worth Billion$
and
b) A competitive race in that Market...
then
c) We have VALUE in anything (product, service, ?, ???) that helps any given participant capture more of that market than their competitors...
(AMD (and the other lesser known GPU/NPU/IPU etc. chip vendors) are currently lagging behind NVidia's CUDA AI market dominance -- so anything that helps the others advance in this area should, generally speaking, be beneficial for all technology users in general, and be potentially profitable (if the correct deals could be struck!) by those that have the skills to do such assisting...)
Anyway, wishing you well in your endeavors, Tinygrad!
No code golf! While low line count is a guiding light of this project, anything that remotely looks like code golf will be closed. The true goal is reducing complexity and increasing readability, and deleting \ns does nothing to help with that.
https://github.com/tinygrad/tinygrad?tab=readme-ov-file#cont...
https://geohot.github.io/blog/jekyll/update/2025/04/22/a-way...
We can't just call people we disagree with "absolute nuts" all the time.
I've got a Comma 3X and I'm thankful for the 4090 P2P work too (which is now here? https://github.com/tinygrad/open-gpu-kernel-modules) so I'm excited to see it work. Rooting for the guy. Hope it's true that the chip part is the easy work. Could be that both are "the hard work".
https://en.wikipedia.org/wiki/Universe_(1960_film)
More precisely it were the special effects he saw from a Canadian documentary in the link above. Barry Lyndon? NASA lenses. Could you say the same for Scorsese? I don't think so. He would never. Ever. Do it for those reasons. Moving on. When N.Tesla invented induction motor did he do so for the improvement of the infrastructure and improving the lives of others? No. He did it because he wanted to improve DC motor. Or namely, the fact that he did not like the sparks that came from the brushes and that were caused by electrical arcing on commutator segments. Did he care to perfect his invention? No. Mikhail Dolivo-Dobrovolsky did. N.Tesla could not care less about the materials used or the number of phases that would make the motor perform at it's best (even though he was invested in the production phase initially). Why? Because his mission was complete. He gave the idea. He built a working model. Then he moved on. Just like Kubrick, his energy came from a desire to solve a problem. No shame in moving on. Imagine Kubrick calling it quits after Lolita. Or N.Tesla devoting his energies on perfecting the induction engine. Or Elon Musk working on electric cars and not caring about space travel. Etc... Not good. Because there are people in this world made specifically for that role. That would be the Torvald's of the world (L Torvalds: "I am not a visionary. I'm an engineer. I'm happy with the people who are wandering around looking at the stars but I am looking at the ground and I want to fix the pothole before I fall in"). The Karpov's. The Marcan's. The George R.R Martin's... Not Nikola's, not Elon's or Steven King's. Not George's. And just because visionaries look at the stars does not mean they have a lofty goal. Or that they are entitled. Visionary is the wiring of the brain. Those blessed with it should not go against it. I could be wrong. I am not that technical. I am that educated. I have not achieved much in life. English is not my first language.