Right now we have a LOT of band aids. You want to optimize compute and thinking to a particular problem, sort of like we do. Yes you cannot perfectly predict this but you can do decently well and save a ton of tokens at…
Hey I totally agree I do not want a teleoperator looking into my house, it’s just so deliciously tempting to get in home on policy data. Not sure the reason why they are super interested in home environments vs business…
Totally agree but the idea is this gives you a teleoperation environment that is truly on policy and not some artificial lab. The idea is that these robots, like those Amazon stores, are predominantly just controlled by…
I think the problem is whether you're for or against local models, the training strategies etc are roughly the same. > I think the last month has resulted in a lot of people saying “f u I would rather fail in life than…
Curious why you feel they won’t get the tech down? I think these products are all data plays right now.
I don't have a ball, where's my ball? Did you take my ball? I want my ball.
I agree with everything you're saying. And I would also agree if you can run your business adequately on OSS models you should absolutely do that for exactly the reasons you say. All I mean is: frontier models will…
> Marketing is just an arms race. It is ok to limit how manipulative and harmful it is and force companies to compete on something else. I agree with this, and I think you are right about a lot of this being really just…
> We have observed consistent scalings of language model log-likelihood loss with non-embedding parameter count N, dataset size D, and optimized training computation Cmin, as encapsulated in Equations (1.5) and (1.6).…
They are still a few years away from parity. It's not that they _cannot_ train frontier models its that it costs 6x as much roughly, and thats ignoring the headaches and extra hoops they need to gather that compute.…
by your username I assume you are yourself one of the droids you're looking for lol. But also I think people forget: this is not cut and dried. This is not simple. This is not just "these companies are evil and should…
This makes sense but I am not comfortable with the open source picture either. It depends on the use case and long term strategy. If you're handling customer support tickets or something, beyond some capability I can't…
> Nope. If China were not banned from US-controlled chips, it would be importing chips at much higher prices, therefore getting less bang for its buck while strengthening competitors with its money in the process.…
That is exactly the scenario that I believe and invest in. I peg trouble in Taiwan in the next 2 years at about 30%, which is waaaay higher than is priced into the market right now. If you think intel has gone up in…
Oh no Chinas chip manufacturing efforts have a long and rocky history, they would be pursuing a sovereign stack no matter what. The tradeoff is worth it. They’re even publishing papers which blows me away — their…
The Jensen argument that "oh we get them hooked on our technology and that will actually be better" is bullshit -- they would do the exact same thing they're doing now of building their own supply chains and compute…
Exactly -- the hope for US strategy is that you can slow them down a lot but not forever. That slowing them down is in itself enough to keep a strategic advantage over them both in terms of economic growth and offensive…
Yes but in an equilibrium steady state, compute and data advantages are all you need to first order. China does not yet have a compute advantage. RL is indeed the magic sauce for coding agents but the bottleneck for how…
Definitely hasn’t backfired. Exporting would have just sped up their progress. Instead they had to get clever and lean into the bottleneck which for them, now, is compute efficiency. This is temporary and they’ll figure…
Its like any other sanction, its designed to slow not stop
> To this you can add the specific testable training that pertains to code, but this degrades the weights for more general communication. Im not sure exactly what you’re saying here, is it that you trade off coding…
No, I just have a prior that’s informed by real information and evidence and that’s quite strong. - scaling laws exist - downstream perf trends also exist (epoch capability index) - gpt4 to gpt5 leap in capabilities…
Yea it’s hard for me to think of what the end state equilibrium is. A pile of vibe coded junk today is bad. You need humans. But we’ve made such a ridiculous amount of progress in such a short amount of time and, most…
Good thing we have reams of data on this, holding performance constant the cost goes down 10-40x per year: https://epoch.ai (like the first box) Also, frontier token prices have remained roughly constant: 3.5 sonnet:…
Idk if someone paying attention to how 2025 and 2026 have gone thinks that by 2028 we will be backing off of agenting coding that is wild. Like the other comment says: future models refactor the code of older models.
Right now we have a LOT of band aids. You want to optimize compute and thinking to a particular problem, sort of like we do. Yes you cannot perfectly predict this but you can do decently well and save a ton of tokens at…
Hey I totally agree I do not want a teleoperator looking into my house, it’s just so deliciously tempting to get in home on policy data. Not sure the reason why they are super interested in home environments vs business…
Totally agree but the idea is this gives you a teleoperation environment that is truly on policy and not some artificial lab. The idea is that these robots, like those Amazon stores, are predominantly just controlled by…
I think the problem is whether you're for or against local models, the training strategies etc are roughly the same. > I think the last month has resulted in a lot of people saying “f u I would rather fail in life than…
Curious why you feel they won’t get the tech down? I think these products are all data plays right now.
I don't have a ball, where's my ball? Did you take my ball? I want my ball.
I agree with everything you're saying. And I would also agree if you can run your business adequately on OSS models you should absolutely do that for exactly the reasons you say. All I mean is: frontier models will…
> Marketing is just an arms race. It is ok to limit how manipulative and harmful it is and force companies to compete on something else. I agree with this, and I think you are right about a lot of this being really just…
> We have observed consistent scalings of language model log-likelihood loss with non-embedding parameter count N, dataset size D, and optimized training computation Cmin, as encapsulated in Equations (1.5) and (1.6).…
They are still a few years away from parity. It's not that they _cannot_ train frontier models its that it costs 6x as much roughly, and thats ignoring the headaches and extra hoops they need to gather that compute.…
by your username I assume you are yourself one of the droids you're looking for lol. But also I think people forget: this is not cut and dried. This is not simple. This is not just "these companies are evil and should…
This makes sense but I am not comfortable with the open source picture either. It depends on the use case and long term strategy. If you're handling customer support tickets or something, beyond some capability I can't…
> Nope. If China were not banned from US-controlled chips, it would be importing chips at much higher prices, therefore getting less bang for its buck while strengthening competitors with its money in the process.…
That is exactly the scenario that I believe and invest in. I peg trouble in Taiwan in the next 2 years at about 30%, which is waaaay higher than is priced into the market right now. If you think intel has gone up in…
Oh no Chinas chip manufacturing efforts have a long and rocky history, they would be pursuing a sovereign stack no matter what. The tradeoff is worth it. They’re even publishing papers which blows me away — their…
The Jensen argument that "oh we get them hooked on our technology and that will actually be better" is bullshit -- they would do the exact same thing they're doing now of building their own supply chains and compute…
Exactly -- the hope for US strategy is that you can slow them down a lot but not forever. That slowing them down is in itself enough to keep a strategic advantage over them both in terms of economic growth and offensive…
Yes but in an equilibrium steady state, compute and data advantages are all you need to first order. China does not yet have a compute advantage. RL is indeed the magic sauce for coding agents but the bottleneck for how…
Definitely hasn’t backfired. Exporting would have just sped up their progress. Instead they had to get clever and lean into the bottleneck which for them, now, is compute efficiency. This is temporary and they’ll figure…
Its like any other sanction, its designed to slow not stop
> To this you can add the specific testable training that pertains to code, but this degrades the weights for more general communication. Im not sure exactly what you’re saying here, is it that you trade off coding…
No, I just have a prior that’s informed by real information and evidence and that’s quite strong. - scaling laws exist - downstream perf trends also exist (epoch capability index) - gpt4 to gpt5 leap in capabilities…
Yea it’s hard for me to think of what the end state equilibrium is. A pile of vibe coded junk today is bad. You need humans. But we’ve made such a ridiculous amount of progress in such a short amount of time and, most…
Good thing we have reams of data on this, holding performance constant the cost goes down 10-40x per year: https://epoch.ai (like the first box) Also, frontier token prices have remained roughly constant: 3.5 sonnet:…
Idk if someone paying attention to how 2025 and 2026 have gone thinks that by 2028 we will be backing off of agenting coding that is wild. Like the other comment says: future models refactor the code of older models.