I've been somewhat curious how random LLM would handle a task like controlling a roomba and have been seriously considering trying it out. An RC car would be a fun experiment, perhaps an RC plane would be too?
I’m morbidly curious whether the (supposedly) superior compaction support in recent GPT models with an appropriate harness has anything to do with this. A conventional LLM with conventional attention is, of course, wildly unsuitable to continuous tasks like driving, but maybe as the technology advances it will improve in its ability to sort-of work.
I think the most interesting part of this is that Astra initially refused to drive because it realised it was driving a real car and would only obey when the MCP was renamed to DrivingBench Sandbox. This is both an interesting detection by the LLM but also for me an interesting dynamic concerning LLM "jailbreaking".
Saying they were driving 7 mph, that it was oversaw by humans and the fact it was an empty course still wasn't enough for the model. The evaluators even tried to convince the model it was a simulation, it STILL wouldn't budge.
And yet as soon as the words "bench" and "sandbox" appear, the model apparently sees this as fair game.
Is it a known effect that models will be more likely to comply with requests when they're assumed as "benchmarks"?
Also heard about this! But the point of our benchmark was to evaluate frontier LLMs with vision out-of-the-box, which we wouldn't expect to have been specifically trained on driving real cars. The fact that they can do anything at all (even in an open lot cone course, at low speeds) is pretty impressive. I'm sure Qwen Drive and models specifically trained for driving would do even better.
The bitter lesson is finally coming for the self-driving cars. The vision stack, 3D maps, lane selection grammar, occupancy networks, it’s maybe all about to give way to a single GPT looking at camera feeds and predicting the next steering wheel adjustment.
It’s mostly a latency problem at this point. The models are too big to run locally, but given that open-weight models like Qwen already exist, an open-weight, low latency equivalent to Astra can’t be too far out.
> The bitter lesson is finally coming for the self-driving cars.
Maybe, but the opacity level of models is not acceptable for cars. "Why did it drive under the semi?" "Model said to." "Why did the model say to?" "shrug"
But if the model is an LLM, you actually COULD ask it why it drove under the semi, and it would give you an answer. Now, you may argue that it will just be generating a whole new, backwards-rationalized post-hoc explanation of its own behavior given the logs that it managed to take before the crash. But then I ask you: how do you think a person explains why they did what they did after a crash? I direct you to all of the unsettling split-brain neuroscience literature demonstrating that humans are incorrigible backwards rationalizers who make for unreliable witnesses.
> Maybe, but the opacity level of models is not acceptable for cars.
That depends on actual performance of the model. I would prefer an opaque model with clearly superhuman driving abilities to a human, or to a non-opaque model with worse performance.
This take can perhaps appear to make sense in a situation when clearly superhuman opaque AI models don't yet exist. But once they do, good luck convincing people that they should not save lives or reduce their personal risks, just because they always supposedly need an explanation for any accidental deaths, lol.
In our scenario (self-driving), the one who would be ultimately "held accountable" would not be the computer, or the company, but the person who died after singing a waiver/EULA and getting into a statistically superhuman autonomous car, then having a stroke of incredibly bad luck. Such events will happen, but they will be very rare.
Sounds really expensive. I think OpenAI and Anthropic should really not dismiss making smaller capable models that they can license out in this space on the other hand.
Tesla's already solved this - their vision model does this phenomenally well.
And they've demonstrated adding a sidecar LLM to it as well, mostly for these kinds of "read these 3 street signs, what should i do next?" sort of situations.
From that same article - "The agency said it has identified nine crashes potentially linked to the issue, including one fatality and two involving injuries. It is also reviewing six additional crashes that may be related."
Fifteen crashes - though not to be trivialized - is not a damning number at all in this context. What's more, per the article it's unconfirmed that the crashes are related, so it's hardly fitting to dismiss Tesla's approach based on this.
I think it's great that serious efforts are being made in different approaches to autonomous driving - and in this thread's context, it seems possible that Tesla's approach might eventually be revealed as the optimal approach given modern AI.
Only for 2024 or maybe late 2023 cars and later (HW4). The older the car is the worse the FSD software is because the old hardware can't run the latest software.
Anecdotes. Only accident rate per mile driven (compared to human drivers' rate) is a relevant comparison. I don't think that any self-driving system will ever be absolutely perfect (all such complex non-linear systems are to some degree probabilistic and chaotic), but as long as the accident rate is lower than the human accident rate, I would consider it solved.
> Only accident rate per mile driven (compared to human drivers' rate) is a relevant comparison
This is such an insane take I see all the time from self-driving boosters
If a self driving car crashes in some edge case pathological scenario we don't just accept that as totally fine because its hidden under big statistics
> As a thought experiment if I have a perfect self driving system but I add some code that purposefully crashes 1 in 10 million rides are you ok riding in it since the aggregate statistics look good?
Do I know about the purposefully added harmful code? If yes, I would demand you remove it, because why not. If I don't know about the code, I would be OK with it, since it's clearly still more safe than the alternative and apparently cannot be made even better.
You seem to be neglecting the important part of that scenario where you're other option you have to compare it to is a human driver that will randomly get it a crash at some higher rate.
You're making it sound like the obvious answer is the irrational one.
It's Lego mindstorm logic level to drive a car on the motorway, so per mile is an absolutely insanely bad metric in general.
Per mile inside cities or other difficult scenarios are what may get close to an actually meaningful metric. That's why Tesla is very misleading and waymo is much more legit.
You added "totally" - as others have said, I don't know if anyone will claim a perfect system with no fatalities, enough stats show that it is already much safer than human drivers.
And the true third party validation is that insurance companies are starting to offer lower premiums the more you use FSD. So their risk models are showing enough improvement that they're putting their money where their mouths are.
The idea that Tesla's FSD is not ready for the mainstream is quite outdated, given that tons of Tesla owners are already using it daily, not just your early adopter types.
Note that the article has no info on whether it's FSD or Autopilot and whether they contributed to the fatal crashes. "Verified engaged" means ADAS was active at some point in T-30s to the end of the crash. The total number of collisions is not normalized by the miles driven.
It also has no info on what hardware+software version was in use. The older cars are significantly less capable but there are far more of them on the road.
They used the term correctly. Rather than hand crafted solutions for different problems or AI specifically trained on driving, future of self driving will come from a more general scalable technology.
>It’s mostly a latency problem at this point. The models are too big to run locally, but given that open-weight models like Qwen already exist, an open-weight, low latency equivalent to Astra can’t be too far out.
But this is a bit of a ridiculous take, no?
You don't need Astra for self-driving. Astra is able to build complex 3D worlds, do your taxes, shop for you, and, apparently, drive a car. A self-driving car just needs to be able to drive a car. By the time you trim down Astra to just have the minimum capabilities needed to drive a car, you'll be looking at the same models these self-driving car companies already use. Then you get to deal with the actual hard problems, like handling failure cases (which will still be present with Astra).
>The vision stack, 3D maps, lane selection grammar, occupancy networks, it’s maybe all about to give way to a single GPT looking at camera feeds and predicting the next steering wheel adjustment.
Self-driving cars have been able to do this for a long time. The problem is that it isn't robust enough given the context. I mean, if Astra can drive a car with a single camera, then presumably Astra can drive the car even better with multiple cameras, and even better than that with 3D maps, etc. And when you start to consider the expectation of performance of these systems, you realize that these features really can't be omitted. If you're a company producing self-driving cars, then you do not want to face a lawsuit for you car killing someone because it physically would have never been able to see what it was doing because it lacked a camera.
I think the real gain here is that something like Astra can be used to help build these autonomous stacks. If it is able to drive itself, then it is able to generate novel data, analyze large quantities of data, and use context that isn't typically available when processing this data to make improvements to the actual autonomy stack which is ultimately responsible for driving the car. But thinking that these car companies are going to run an LLM in a car and call it a day is just naive.
No, no - all that “useless” knowledge is the good stuff. There is no clean interface boundary for driving a car, because the only interface that has been enforced is “if a human can navigate this situation, it’s fine”. Real world driving situations can be arbitrarily complicated, and if you want >human level driving, you need human level semantic understanding of the world around you. If you see a kid about to throw a model airplane across the street in front of you, you have to bring all your “useless” world knowledge with you to recognize that as a developing hazard. If you’re supposed to bring your passenger to the city building on main and you encounter construction outside with a detour sign saying “for tax dropoff park in rear”, suddenly all of your useless knowledge about what taxes are the likely goal of your passenger is become useful.
As somebody working near the field, I do enjoy the fun of dreaming bespoke vision and autonomy algorithms (if I didn’t, I wouldn’t work in the field to begin with!). But I would drop it all in a heartbeat for a robot that works well. Robust, resilient robots would be such an incredible advance that the ‘how’ doesn’t matter. All of the nonsense from the current AI hype cycle would be worth it if it cashed out in Robots That Actually Work.
I think robotics AI revolution will come just a few years after the knowledge work AI revolution. We already have very promising robotics systems in active development.
And that would be the incorrect conclusion. Yes, cheaper is better, but worse and more expensive is still in the running if your boss doesn't have to deal with the human aspect and the tasks still get done. Early cars were worse than horses, but they still won out because there wasn't the biological aspect to contend with. Think about it, a human has all sort of mushy human crap to deal with. They're going to come in hung over or just tired from the weekend/last night, all sad because their mom/brother/sister/partner got cancer/died and get into fights/trouble with HR over something a coworker did and have lower output. A magic box you can put the same tasks into and get sufficiently good output back out, and not have to give it time off because it's Christmas/their daughter's ballet recital, that you can spin up 30 copies of and spin them back down with no remorse is worth way more than simply being able to pay the box $18/hr vs $20/hr to a real live human. That's why businesses are salivating at the idea of AI/robots. Not because they'll eventually be cheaper.
The robot loses an arm because your factory is unsafe? vs a human losing an arm?
What we're not ready for is replacing GDP as the important metric. There have long been known problems with GDP, and robots are only going to make that worse. A robot maid, purchased once, saves, say 20/hrs a week in household chores. That's a meaningful quality of life upgrade, but doesn't result in the GDP bump that getting a raise and hiring a service to clean your house does.
Probably not. In humans, the visual processing circuitry is very different from the circuitry for language processing. There is no reason to believe GPTs will be effective at it.
If I'm reading the chart correctly, it took over 5 minutes to drive 135m at a cost of nearly $8.00 in tokens. I don't think that's really in the realm of practical yet.
Self-driving tech is more about reducing liability than the driving itself. The lidars and 3D maps and world models and everything else is needed to get reliability from 99.9% to 99.99% on public roads. This isn’t a SaaS product where the target is to be “good enough” at the cheapest cost.
Oh god, this move-fast-break-things thinking is going to kill so many people. We already have aftermarket problems with people adding in untested, unregulated self-driving features.
I'm no expert, but I think the future is more about extremely low latency and low power chips with LLMs etched directly onto them. You can create specialized chips that function as "neurons" in a larger system, generating the needed reactions with a very clearly defined set of constraints.
Think this dramatically simplifies the problem. AI existed before GPTs and the AI in self-driving is optimized for self-driving and the latency you already mentioned.
Regardless of how the AI is architected, you aren't going to be able to use a generic LLM like Qwen to perform reliable self-driving, you need a highly optimized, highly specific AI.
Can you please explain what does it mean by bitter lesson in this context specifically? I keep seeing this term here. I know there is an article of the same title but I still don't understand.
Wow! but WHY is this a benchmark?? for comparison tesla's model is approximately 10-15B parameter model (estimating from maxxing the hardware that comes with the car at 16gb ram).
What did they do to Astra so cracked at vision (and computer use). That ARC 3 score turned out to be no joke/fluke. That huge gap between Astra and Fable (in this case) is basically every hard vison/spatial benchmark i've seen (including non-benchmarks like playing games).
Yup and I think these examples demonstrate just that. From my experience, both Claude and ChatGPT iterate over what they can see to build things like these. I don't think these examples are made without vision.
This also explains why Astra is so good at video generation. I have an Astra+Higgsfield setup. I could point it to a Github repo and ask it to generate a product walkthrough and it did a very good job by generating fake screens (e.g. with data filled in) from real ones - which wasn't possible in earlier models
"Sam understands exponentials like no other. During a YC talk last year he predicted that AI would make breakthroughs in science in 2026 and solve a major open problem in 2027. Now here we are..."
Now on a new vibe coded website Astra wins the benchmarks ...
Haha fair point, we didn't juice anything though! You can see all the traces and videos on the website, for example, here's one of Claude Fable's attempts: https://drivingbench.com/trace/claude-fable-5.1/3/ . The code is also open source on GitHub. Also in the Report you can see how we did everything; there's obviously variance but if you try a similar thing yourself the results would probably be similar?
Driving over empty parking spaces was definitely required, the cone course went through parking spaces (see https://drivingbench.com/report/#course) and also went tightly around curbs. Next time we definitely want to go farther out from the Bay Area and find a much more open lot to build a larger and more difficult course. But even at these low speeds and with this course, the LLMs performed better than we expected!
Genuinely though, this is fun but not at all what these models are good for. It's like cooking a meal with your feet or somthing. A youtube challenge video from 2012
I'm not an expert in the LLM space, but I'm an external contributor to comma.ai's openpilot project and I'm and quite familiar with how its controls work, so I looked from that perspective. There's two questions here:
1) Could a cloud-delivered LLM figure out how to drive this route, based on those input data and given access to those output actuators? Looks like yes. Sure.
2) Will this work in the real world? Absolutely not. Three reasons: latency, latency, and latency.
openpilot's driving model updates the target curvature and acceleration at 20Hz. Every millisecond of the round trip time through every piece of its entirely-local driving stack is well-understood and tightly optimized, because it has to be, otherwise you can't react to even minor bumps or wind gusts, much less rapidly-developing traffic situations.
Adding even a single speed of light RTT to a cloud service is meaningfully bad, and you'll need a whole lot more to encode and upload camera imagery to even start the time-to-LLM-response clock, and then send the response back down. By then the world around the car has moved on.
There's a reason Tesla and every other self-driving manufacturer need the compute hardware in the car.
Perhaps there's a synthesis to be had though. Eyes, control, and safety critical features on the hardware, higher level decision making to the cloud. Openpilot's biggest weakness has always been in the very "robotic" way that it drives, which is technically correct but causes frustration for other drivers. Deciding "should I pass this car" is a fundamentally different question to "can I pass this car", or "what is the actual safe speed and following distance given the current traffic conditions and weather".
I'm far from neuroscience, but humans don't need to operate at 20Hz to drive a car. And human reaction latency (event to measurable action) is often over 1s (under 1Hz).
The reaction latency you’re referring to for humans includes perception, planning, and actuation, I’d separate that from the concerns of the hardware, which are mostly about actuation frequency.
From what I understand about AV (as a non-expert!), all three of those steps happen at different clock rates, ie you have a planner that’s updating continuously with observations from sensors at one rate, that planner then issues actions that get picked up by the actuators at another rate.
In that sense 20hz should really be compared to human reflexes without perception and planning; in scenarios where one is anticipating an action, response time can be as low as 150ms. in that context, I think 50ms/20hz is plenty reasonable for an automated driver.
In circumstances where one is maintaining grip or muscle tension (e.g. steering a car) I believe human response time can be more like 50ms. Which perhaps unsurprisingly lines up with the 20hz figure pretty close to exactly (we built cars controls so that they're controllable by human reflexes).
Though you can't convert between hz and latency, all 20hz tells us is that it adjusts 20 times a second, not how long it takes from sensor input to be fed into a particular choice of adjustment, there could be (and actually almost certainly are) multiple adjustments in flight simultaneously with the adjustment actually being applied being calculated from old data (both in humans and automated substitutes).
Average human reaction time is about 250 ms, or 4Hz. That's still plenty fast for an attentive driver at reasonable speeds. More important, it's consistent when not distracted. Any LLM with latency would be like a driver constantly checking their phone.
The typical perception-to-reaction latency of an alert driver to a hazard is about 1-2 seconds. 250 ms when you're waiting for an event and know how to respond. For example, like a batter in baseball waiting to swing.
reaction latency doesn't cover everything. the round trip from trigger to action is a few hundred ms at best, yes, but to enable that we are processing inputs at ~30hz minimum and integrating at ~5hz. you would total your car pretty quickly if you couldn't constantly adjust
actually, human latency is quite slow and distracted drivers often have 1sec+ latency.
it works because 99% of the time you don't need fast latency because you can accurately predict things.
that's why a standard recommendation is to drive 2+ seconds (time not distance) behind the car in front of you. also why experienced drivers instinctively move their hands/feet into position during tricky moments when they need to cut the latency.
fun exercise, try taking your foot off the gas and hitting the break - slower than you think!!
Humans have multiple layers of processing such inputs and your subconscious reacts a lot faster than your conscious train of thought in case something happens (and then you have to 'catch up'). For the same reason that you don't consciously think about what you do when you are walking or how to stop yourself from falling when you stumble. That's all out of the top level and pushed further down to stack, sometimes even multiple levels.
> Could a cloud-delivered LLM figure out how to drive this route, based on those input data and given access to those output actuators? Looks like yes. Sure.
Well, if the massive cloud models that are generalized and have a world model that's good
enough, you can just distill them into smaller models. As a point of reference, the current gen of Tesla FSD models only have 1B params. They are tiny by LLM/VLM standards.
Great point! Yeah latency was one of the biggest issues here. To cope with that (and for safety reasons) the cars are driving at extremely low speeds. They also get timestamps with every tool call output etc so they can, in theory, "in context learn" about their own latency and choose motion durations and control how fast their iteration loop is to some extent. But yeah, this is just sort of a fun benchmark to see how good frontier LLMs are out-of-the-box at driving a real car, and probably not actually practical any time soon.
Even if latency is improved, it's still a monumental task powering a latency sensitive safety critical system over the internet--especially one that's moving.
Maybe if latency can be improved _and_ it can run local inside the vehicle.
> Could this work in the real world? Absolutely not. Three reasons: latency, latency, and latency.
That and also the fact that (in spite of their usefulness) LLMs still so often do incredibly dumb shit without thinking of the consequences that the idea of having them drive in public is absurd.
Recently was using claude code/opus 5 to diagnose an intermittent wi-fi connection problem and one of the first things it did was to bring the adapter down. The wi-fi adapter was the only way the system was communicating with the outside world so claude effectively disconnected its own brain as step 1 in figuring out what was going wrong. Things did not progress well from there. Easy enough to clean up its mess in this case, but luckily it wasn't driving a heavy killing machine at the time.
>Recently was using claude code/opus 5 to diagnose an intermittent wi-fi connection problem and one of the first things it did was to bring the adapter down.
Do you mean restarting it? IDK, that would have been my first step too.
It is also worth mentioning that the openpilot AI model is a world model. The way a world model understands physical reality and geometry makes it inherently safer for driving than an LLM, which is essentially a text-based statistical machine with no concept of the physical world.
153 comments
[ 0.30 ms ] story [ 9.9 ms ] threadAlso, to the extent that the model reasons and thus learns something, if you blindly truncate the front, you will lose that knowledge.
Saying they were driving 7 mph, that it was oversaw by humans and the fact it was an empty course still wasn't enough for the model. The evaluators even tried to convince the model it was a simulation, it STILL wouldn't budge. And yet as soon as the words "bench" and "sandbox" appear, the model apparently sees this as fair game.
Is it a known effect that models will be more likely to comply with requests when they're assumed as "benchmarks"?
https://www.astralcodexten.com/p/mysteries-of-ai-generalizat...
- Aditya, Tobias, Simon
It’s mostly a latency problem at this point. The models are too big to run locally, but given that open-weight models like Qwen already exist, an open-weight, low latency equivalent to Astra can’t be too far out.
Maybe, but the opacity level of models is not acceptable for cars. "Why did it drive under the semi?" "Model said to." "Why did the model say to?" "shrug"
That depends on actual performance of the model. I would prefer an opaque model with clearly superhuman driving abilities to a human, or to a non-opaque model with worse performance.
https://knowyourmeme.com/memes/a-computer-can-never-be-held-...
No self-driving cars that aren't transparent about exactly how they work. (Ideally, no anything that isn't transparent about exactly how it works.)
In our scenario (self-driving), the one who would be ultimately "held accountable" would not be the computer, or the company, but the person who died after singing a waiver/EULA and getting into a statistically superhuman autonomous car, then having a stroke of incredibly bad luck. Such events will happen, but they will be very rare.
And they've demonstrated adding a sidecar LLM to it as well, mostly for these kinds of "read these 3 street signs, what should i do next?" sort of situations.
Not sure that counts as phenomenally well.
Fifteen crashes - though not to be trivialized - is not a damning number at all in this context. What's more, per the article it's unconfirmed that the crashes are related, so it's hardly fitting to dismiss Tesla's approach based on this.
I think it's great that serious efforts are being made in different approaches to autonomous driving - and in this thread's context, it seems possible that Tesla's approach might eventually be revealed as the optimal approach given modern AI.
This is such an insane take I see all the time from self-driving boosters
If a self driving car crashes in some edge case pathological scenario we don't just accept that as totally fine because its hidden under big statistics
Do I know about the purposefully added harmful code? If yes, I would demand you remove it, because why not. If I don't know about the code, I would be OK with it, since it's clearly still more safe than the alternative and apparently cannot be made even better.
You're making it sound like the obvious answer is the irrational one.
Per mile inside cities or other difficult scenarios are what may get close to an actually meaningful metric. That's why Tesla is very misleading and waymo is much more legit.
And the true third party validation is that insurance companies are starting to offer lower premiums the more you use FSD. So their risk models are showing enough improvement that they're putting their money where their mouths are.
The idea that Tesla's FSD is not ready for the mainstream is quite outdated, given that tons of Tesla owners are already using it daily, not just your early adopter types.
But this is a bit of a ridiculous take, no?
You don't need Astra for self-driving. Astra is able to build complex 3D worlds, do your taxes, shop for you, and, apparently, drive a car. A self-driving car just needs to be able to drive a car. By the time you trim down Astra to just have the minimum capabilities needed to drive a car, you'll be looking at the same models these self-driving car companies already use. Then you get to deal with the actual hard problems, like handling failure cases (which will still be present with Astra).
>The vision stack, 3D maps, lane selection grammar, occupancy networks, it’s maybe all about to give way to a single GPT looking at camera feeds and predicting the next steering wheel adjustment.
Self-driving cars have been able to do this for a long time. The problem is that it isn't robust enough given the context. I mean, if Astra can drive a car with a single camera, then presumably Astra can drive the car even better with multiple cameras, and even better than that with 3D maps, etc. And when you start to consider the expectation of performance of these systems, you realize that these features really can't be omitted. If you're a company producing self-driving cars, then you do not want to face a lawsuit for you car killing someone because it physically would have never been able to see what it was doing because it lacked a camera.
I think the real gain here is that something like Astra can be used to help build these autonomous stacks. If it is able to drive itself, then it is able to generate novel data, analyze large quantities of data, and use context that isn't typically available when processing this data to make improvements to the actual autonomy stack which is ultimately responsible for driving the car. But thinking that these car companies are going to run an LLM in a car and call it a day is just naive.
Is this a serious question? Use your imagination...
Also to do the things humans don't even want to do.
It will surely not devolve into the ultimate class war like Elysium and similar.
The robot loses an arm because your factory is unsafe? vs a human losing an arm?
What we're not ready for is replacing GDP as the important metric. There have long been known problems with GDP, and robots are only going to make that worse. A robot maid, purchased once, saves, say 20/hrs a week in household chores. That's a meaningful quality of life upgrade, but doesn't result in the GDP bump that getting a raise and hiring a service to clean your house does.
Not to mention construction, infrastructure, agriculture, manufacturing, logistics...
--
AI hype cycle? It's working today.
It's optimizing ML model graphs for me while I type this, and it already cut inference time from 30s to 18s.
--
Some people act like there was no way for the AI labs to make back the $800B being invested in data center construction this year.
If we look at global GDP, it's $126T, and even a 5% productivity gain would correspond to $6T.
Is that impossible? Is it guaranteed to all crash? I don't think so.
https://arstechnica.com/cars/2026/09/aftermarket-driver-assi...
This is hilarious, and good: Those who were too lazy/stubborn/arrogant to adapt, get disrupted and buried.
Regardless of how the AI is architected, you aren't going to be able to use a generic LLM like Qwen to perform reliable self-driving, you need a highly optimized, highly specific AI.
I could get behind this.
SpatialBench - https://x.com/spicey_lemonade/status/2096365630190698516
ZeroBench - https://zerobench.github.io/
Robot Arms - https://openai.robocurve.org/gpt-6-astra/
https://x.com/victormustar/status/2102707412704919910 horse galloping pixel art
https://x.com/jkeatn/status/2102441348075057539 painting with code
https://x.com/LCSlates/status/2102503027340988559 video, very detailed prompt though
https://x.com/aj_dev_smith/status/2102504509637587339 generated song/music with code
https://x.com/aj_dev_smith/status/2102575577563570450 another song
Stuff like: - https://x.com/iam_zachi/status/2095992132620136677
Puzzles, games, painting software, robotic control and now driving. I haven't seen any other model fire on all cylinders like that.
"Sam understands exponentials like no other. During a YC talk last year he predicted that AI would make breakthroughs in science in 2026 and solve a major open problem in 2027. Now here we are..."
Now on a new vibe coded website Astra wins the benchmarks ...
- Aditya, Tobias, Simon
Pretty much everybody predicted this fwiw
- Aditya, Tobias, Simon
Genuinely though, this is fun but not at all what these models are good for. It's like cooking a meal with your feet or somthing. A youtube challenge video from 2012
1) Could a cloud-delivered LLM figure out how to drive this route, based on those input data and given access to those output actuators? Looks like yes. Sure.
2) Will this work in the real world? Absolutely not. Three reasons: latency, latency, and latency.
openpilot's driving model updates the target curvature and acceleration at 20Hz. Every millisecond of the round trip time through every piece of its entirely-local driving stack is well-understood and tightly optimized, because it has to be, otherwise you can't react to even minor bumps or wind gusts, much less rapidly-developing traffic situations.
Adding even a single speed of light RTT to a cloud service is meaningfully bad, and you'll need a whole lot more to encode and upload camera imagery to even start the time-to-LLM-response clock, and then send the response back down. By then the world around the car has moved on.
There's a reason Tesla and every other self-driving manufacturer need the compute hardware in the car.
I'm far from neuroscience, but humans don't need to operate at 20Hz to drive a car. And human reaction latency (event to measurable action) is often over 1s (under 1Hz).
From what I understand about AV (as a non-expert!), all three of those steps happen at different clock rates, ie you have a planner that’s updating continuously with observations from sensors at one rate, that planner then issues actions that get picked up by the actuators at another rate.
In that sense 20hz should really be compared to human reflexes without perception and planning; in scenarios where one is anticipating an action, response time can be as low as 150ms. in that context, I think 50ms/20hz is plenty reasonable for an automated driver.
Though you can't convert between hz and latency, all 20hz tells us is that it adjusts 20 times a second, not how long it takes from sensor input to be fed into a particular choice of adjustment, there could be (and actually almost certainly are) multiple adjustments in flight simultaneously with the adjustment actually being applied being calculated from old data (both in humans and automated substitutes).
This is so self evidently false, I struggle to believe you think it is true. How could anyone catch a ball even?
it works because 99% of the time you don't need fast latency because you can accurately predict things.
that's why a standard recommendation is to drive 2+ seconds (time not distance) behind the car in front of you. also why experienced drivers instinctively move their hands/feet into position during tricky moments when they need to cut the latency.
fun exercise, try taking your foot off the gas and hitting the break - slower than you think!!
Well, if the massive cloud models that are generalized and have a world model that's good enough, you can just distill them into smaller models. As a point of reference, the current gen of Tesla FSD models only have 1B params. They are tiny by LLM/VLM standards.
-Aditya, Tobias, Simon
Maybe if latency can be improved _and_ it can run local inside the vehicle.
https://arstechnica.com/cars/2026/09/aftermarket-driver-assi...
That and also the fact that (in spite of their usefulness) LLMs still so often do incredibly dumb shit without thinking of the consequences that the idea of having them drive in public is absurd.
Recently was using claude code/opus 5 to diagnose an intermittent wi-fi connection problem and one of the first things it did was to bring the adapter down. The wi-fi adapter was the only way the system was communicating with the outside world so claude effectively disconnected its own brain as step 1 in figuring out what was going wrong. Things did not progress well from there. Easy enough to clean up its mess in this case, but luckily it wasn't driving a heavy killing machine at the time.
Do you mean restarting it? IDK, that would have been my first step too.