The articulate bus is modeled as two connected vehicles. The back of the bus is presumed to follow the front of the bus. When the front of the bus is out of view, the software predicts its trajectory. The bug was that the rear of the bus is considered to follow that predicted location even if sensors show differently.
The problem is the bus front was occluded and thus its position and vector were estimates of growing uncertainty, not direct readings. It is not understandable that an estimate with high uncertainty took precedent over a fully reliable direct reading from multiple sensors
This is an outstanding postmortem of this incident. I really did not expect it from Forbes.
Accidents will happen. One thing that will be important going forward is a proper operational response to the problem and this is likely to become more complicated as time goes on. After all if these have been on the road for 5 years, would it really be necessary then to stop operations?
> After all if these have been on the road for 5 years, would it really be necessary then to stop operations?
I imagine they wouldn't be running the same software for 5 years. Whenever there's an update, the "days without accident" for that particular version is set to zero, so I suppose until you know why the accident happened, it's the right choice to assume that all vehicles running that version might be affected.
It seems obvious that the Cruise vehicle should have detected it was going to hit the back of the bus and stop in time, but that can lead to overcautious driving. My 2020 Honda Civic has automatic braking to prevent crashes and it generally works well but would quite often brake suddenly when there was going to be an intentional near miss on my part as the driver. For example in a left turn lane, the vehicle ahead slows right down since they are going to do a U-turn and I aim to just miss their rear bumper to the right by a foot or two when turning left and the Honda might automatically brake when it sees the nearby car even though I know I am not going to hit it.
A fairly straight forward algorithm I thought of and have been tempted to test. Take the last few frames of video. Scale and skew them according to perspective transform based on steering wheel angle and road speed. Average them together. What should be left is exactly the objects in frame that are truly on a collision path with the camera, all else should average away to grey blur.
Collision avoidance in nearly all cars is radar based. And it includes steering angle input.
The problem is when the car you're going to narrowly miss is still in the path of your car but going to move out of it. That's something a human could predict based on judgement, but it's very hard for a basic collision avoidance algorithm to predict.
Humans also get this wrong. You see a car turning, you assume they keep turning and you'll pass just behind their rear bumper in that case. Now for a reason invisible to you the turning car hits their brakes and stops partially blocking your lane. You can't stop in the remaining distance. But would usually be able to steer around it by going to the edge of your lane or overlapping the next lane. But this isn't something a simple collision avoidance algorithm can do.
You haven't understood my simple algorithm which does exactly what you say a human driver is doing, no advanced ml required just basic understanding of projective geometry. If you have frames of video of the previous few fractions of a second and information on how the car moved in 3d space over those frames, I'm saying you can construct a dead simple filter that preserves precisely the in scene objects which are on a collision course with the camera while everything irrelevant in the scene gets washed out.
The simplest case is with no steering. The projective transform is just blowing up the image by a multiplicative factor dependent on the distance moved per frame. So the x, y position of each pixel gets stretched the further back in time it is to compensate for the cars cha ge in parallax. After stretching, all these frames are averaged together. Anything on a collision course with the camera is going to be at the same angular position each time just getting larger. So averaging over will highlight its presence. The rear bumper which you are actually due to miss will be drifting in angular position in each compensated frame, so it gets averaged into the background blur of irrelevant objects.
Tldr simple video frame filter highlights exactly which parts of image frame relevant to possible collision, tells computer what can safely ignore, implements exactly your near miss driving algorithm, no fragile machine learning bs.
It does because the motion of the other objects is embedded in their position over the previous few frames. On the contrary, it is precisely filtering on moving objects such that the velocity vector is directly on a collision path with the camera. Objects which are moving but ultimately not on a collision course will be in different locations once the perspective transform is applied and thus average out to background. Objects which are moving on a collision course relative to the observer get amplified by this simple filter (irrespective of if the motion is absolute or not relative to ground).
The perspective transform needs the depth of each pixel from the camera (or equivalent 3D Cartesian coordinates of objects). An affine image transform that ignores depth won’t project objects correctly.
Let's assume ego and other vehicle are on a straight lane and move with constant speed along the same line. All trajectories where the other vehicle is not faster than ego are eventually colliding. But for each velocity that other vehicle projects to different positions, doesn't it? You can accumulate for one realtive motion only, but there are many dangerous ones. Taking ego speed and steering will accumulate standstill objects on the road plane (if the homography between image and road is accounter for properly). Clearly standstill objects in your path are on a collision course but the original problem was about moving and especially decelerating ones.
Also it takes a large amount of frames to really average out.
Pre-ML We used to implement multi-hypothesis Kalman filters and took the one with the least invention as best prediction.
18 comments
[ 4.3 ms ] story [ 49.8 ms ] threadI'm missing why "reasons not explained" amounts to articulated bus confusion.
All in all, a very understandable coding mistake.
Accidents will happen. One thing that will be important going forward is a proper operational response to the problem and this is likely to become more complicated as time goes on. After all if these have been on the road for 5 years, would it really be necessary then to stop operations?
I imagine they wouldn't be running the same software for 5 years. Whenever there's an update, the "days without accident" for that particular version is set to zero, so I suppose until you know why the accident happened, it's the right choice to assume that all vehicles running that version might be affected.
Nope. In the uk they're RTC's - road traffic collisions - someone will be at fault.
> Even so, for reasons not explained, the Cruise vehicle did brake
It broke alright
The problem is when the car you're going to narrowly miss is still in the path of your car but going to move out of it. That's something a human could predict based on judgement, but it's very hard for a basic collision avoidance algorithm to predict.
Humans also get this wrong. You see a car turning, you assume they keep turning and you'll pass just behind their rear bumper in that case. Now for a reason invisible to you the turning car hits their brakes and stops partially blocking your lane. You can't stop in the remaining distance. But would usually be able to steer around it by going to the edge of your lane or overlapping the next lane. But this isn't something a simple collision avoidance algorithm can do.
The simplest case is with no steering. The projective transform is just blowing up the image by a multiplicative factor dependent on the distance moved per frame. So the x, y position of each pixel gets stretched the further back in time it is to compensate for the cars cha ge in parallax. After stretching, all these frames are averaged together. Anything on a collision course with the camera is going to be at the same angular position each time just getting larger. So averaging over will highlight its presence. The rear bumper which you are actually due to miss will be drifting in angular position in each compensated frame, so it gets averaged into the background blur of irrelevant objects.
Tldr simple video frame filter highlights exactly which parts of image frame relevant to possible collision, tells computer what can safely ignore, implements exactly your near miss driving algorithm, no fragile machine learning bs.
Also it takes a large amount of frames to really average out.
Pre-ML We used to implement multi-hypothesis Kalman filters and took the one with the least invention as best prediction.