No. The system uses 8 cameras. Three forward facing (narrow, wide, standard), two rear facing sides (the two showed here), one rear, and two side cameras.
Impressive to watch. The Tesla drives quite politely, but not annoyingly so (read that as impeding traffic). It actually stopped for the joggers who wasn't actually in the road, but very close. It also waited for a bicyclist on the right to turn left in front of the car.
Looks pretty promising, not bad, especially since the trend in software (and sensors) is to get better over time.
On perfectly marked roads, in fair weather, with no bad behaviour seen from other drivers in the video. Impressive, but the technology has a long way to go yet before it can replace human drivers.
I do not understand why the focus on self driving is so much on the whole trip.
90 - 95% of the time I am driving long stretches, I spend on highways. No problem if only that part would be done selfdriving. Those few minutes in busy town centers I do not mind to do myself. No problem of driving short stretches (10 - 15 min) either.
maybe your driving behavior is different from other people. I live in a city and do about half time on city streets and half time on highways. Anyway, consistent performance on all parts of the trip is needed to convince buyers.
s/convince buyers/avoid mass accidents caused by drivers becoming even less attentive than they're now/. Cruise control doesn't cause that, but a "90% there" self-driving solution will.
It shows that the car would have been able to drive the route completely without a driver. This is handy if you want to pickup somebody, want to avoid high parking fees, need to charge your car without actually doing it.
Doing those things right now in a Tesla is probably not safe just now, but it shows that level 5 autonomy is not so far fetched and 2020 is a realistic date.
On that particular day, under favorable conditions. There's a lot of work between 80% and 100%. Showing the 80% doesn't show anything about how long the last 20% will take.
There's virtually zero chance we'all have level 5 autonomy within 3 years.
The tech is impressive but it has to be absolutely bulletproof. I like Tesla but I find it annoying that Musk doesn't deliver anything on time and it seems like Tesla delivers way more demos than real products.
Autodriving 90% of the time would be a fantastic feature.
Autodriving 100% of the time is where it gets revolutionary (mass-layoffs of truck drivers, taxi drivers, summon features that park cheaply outside of downtown).
The former is great, but there's a lot of hype going around right now so it's not strange for people to be cynical.
I think anywhere between 80% and 99.999% would cause a lot of accidents... "good enough to not pay attention and be inattentive when it needs your help, not good enough to not need your help."
It depends on where the 80% is. If it's "80% of the roads in the country" that's one thing, if it's "at random points, at any time and on average for 20% of your journey you'll be taking control" that's another.
It's all about predictability. Even just full motorway / major road driving would be pretty awesome, and the car would know when it's coming up to a section you need to drive and you can have plenty of warning. Not "HELP DRIVER I SAW EITHER A LEAF OR A CHILD QUICK TAKE OVER"
If I have to pay attention to the road while the car is on autopilot then it's basically useless -- I might as well drive the car just for the stimulation.
If/When it's good enough that I can sleep during a long highway trip (and pull into the last rest stop before my exit if I don't wake up) then I'll be happy.
Forget parking, the cars would be driving, ferrying people and goods around, as much as possible when not charging. No need to own one, just hail one when you need a ride.
It will be interesting to see how people take to not owning cars. People like customizing their spaces, and I can see people with young children not wanting to deal with all that setup multiple times a day...
That's practical, and for people like you and me who aren't bothered by the idea of not owning our own vehicle, but it's very culturally desirable in the US to own a personal vehicle. Not an unsolvable problem, but it will definitely be a bump in the road.
I do wonder how many people living in cities would still choose to own a car if they could just get a self-driving one on demand.
For example, friends of mine recently bought a summer cottage. It's a little over an hour's drive. They don't have a car, so they currently rent one or try to borrow one when they want to go there. Once there, the car will probably sit still for a week.
Currently, there's no really good solution to this. Renting is expensive if you just need to drive a couple of hours a week. Buying a car might make more economic sense in the long term, but it's still pretty expensive for the purpose. Taxi rides like that cost an arm and a leg too. But if the car could drive itself out of the woods...
Of course it's a question of pricing, and I'm guessing we're still some way off from having self-driving cars navigate dirt paths that pass through people's yards as is the case here.
It is so important to do the whole trip because switching costs for the driver from "totally inattentive" to "grasping the wheel and driving" are measured in seconds.
Do you assume that the car, after recognizing that human intervention is required, continues to move forward into the path of danger while it waits for the human to take over?
Volvo takes the position for their 2017 that they cannot demand that the driver take over. The car can ask, but must be prepared to pull over and stop if the driver doesn't push the "Manual Drive" button.
Here's the first of 100 self-driving cars Volvo will put in customer hands in 2017.[1] This shows close-ups of all the sensors - high radar and camera behind the windscreen, low LIDAR and camera in the nose, and side and down-looking cameras on the door mirrors. No high-mounted LIDAR, though. These vehicles are intended for self-driving in all weather conditions in Gothenburg, Sweden.
NHTSA levels: [1] Level 3: "Vehicles at this level of automation enable the driver to cede full control of all safety-critical functions under certain traffic or environmental conditions and in those conditions to rely heavily on the vehicle to monitor for changes in those conditions requiring transition back to driver control. The driver is expected to be available for occasional control, but with sufficiently comfortable transition time. The Google car is an example of limited self-driving automation."
Tesla's old system was Level 2. Google and Volvo have level 3 in test. Tesla's new system is supposed to be Level 3. There's a bright line between level 2 and 3 - NHTSA says that at Level 3 and above, the manufacturer is responsible for accidents and cannot blame the driver.
However statistics say that most of the accidents happen at the beginning or towards the end of the trip. There are environmental reasons (city vs. sub-urbs vs. cross-country) and there are human reasons.
Agreed. Throw in some fog, high-speed stop-and-go interstate fun, and moving from the interstate into a crowded city full of people jaywalking and running stoplights trying to get to where they're going.
Absolutely yes. I can't wait for that day. We'll end up saving more lives per year than if we somehow banished war, at least in industrialized countries.
What's weird is the period we're entering, where the technology is becoming more and more capable, but we're still a very long ways away from full auto. Yet the hype machine says it's just around the corner. I'm betting we're still 10-15 years out. So how do we handle situations where the driver has to pay a little attention, but not that much? In airplanes, this has been a very dangerous area -- and things in airplanes usually happen much, much slower than they do in traffic.
> What's weird is the period we're entering, where the technology is becoming more and more capable, but we're still a very long ways away from full auto. Yet the hype machine says it's just around the corner.
I think it would be worthwhile to figure out how to shut down the hype machine, or at least tune it down a few notches. Dreams and visions are fine, but every time I switch the current tab from HN to Facebook, I get to see overhyped bullshit predictions about AI, nanotech, self-driving cars, etc. being just about the corner, as evidenced by whatever misinterpreted paper or video news sites could find that day. And the non-tech crowd is totally buying that. Not just regular folks, but also investors and politicians and all the other people who then end up causing resources to be wasted because they buy into the bullshit.
It's interesting, though, to see rise of the robots talk breach tech circles and enter into mainstream economic and political discussions. Between the arrival of imagenet in 2011 and now the drums have been steadily getting louder.
In 2013 an oft-cited and alarming study out of oxford suggesting most jobs will be automated over the next decade was released. It was followed by a series of influential books whose authors ran the lecture circuit: 'The Second Machine Age', 'The Zero Marginal Cost Society', 'Superintelligence', and 'The Rise of the Robots'.
There were some bold bold and well publicized statements from respected luminaries such as Bill Gates, Elon Musk and Stephen Hawking.
Aggressive maneuvering to hoover up machine learning talent, and bold investments from automakers pursuing autonomous driving has only added gasoline to the fire.
Automation became the talk of the town at Davos in Switzerland. There's been a rising chorus from Basic Income supporters.
Now the hype is out of control. Nobody is actually looking at the technology. It's okay, though. Hype is self-correcting.
Throw in some electric scooters diving in and out, bicycles, crazy couriers, cars parked on the road, someone pushing a hand-cart, an ambulance, and at least one of each of these kinds of vehicles going the wrong way and disregarding all traffic laws.
yes. If Tesla really wants a good test, try autopilot in Morocco!
all at once: moped taking over from the right, in a round about, truck entering the roundabout without slowing down. donkey & cart on the side. woo hoo!
Well, give them time :). Maybe they can use this video to leverage the generated PR into getting themselves allowed to shoot the next video at more difficult conditions :).
Tallahassee, FL in the summer would be perfect: heavy rain storms daily, college students everywhere, and everyone drives with a supreme sense of arrogance!
This is pretty impressive. Still, at 00:54, getting a closer look at the female runners? Someone following could have gotten caught out if driving too close / playing Pokemon.
A human driver (well, at least myself) would simply have lifted the foot off the accelerator somewhat and drifted closer to the centre line when passing - seeing as there was no oncoming traffic.
Perhaps the car knows it's own capabilities better than we humans do.
Maybe it knows that if the joggers were to suddenly jump out, doing what you or I would do isn't enough, so it slowed down to a near stop just in case.
Also, it's a bit of a "blind corner", so drifting out toward the middle of the lane might not be the best idea (again, even though it's probably something I as a person in that scenario would have done).
I've wondered about this. Totally defensive driving would reduce a car to 5mph - somebody might jump out from behind every blind corner or mailbox.
Truth is, we all drive fairly irresponsibly, trusting in the rules of the road and human self-preservation to keep most stuff out of our way. And if we do hit somebody who dashes out on the freeway, well that's an accident or 'their fault' or 'nobody's fault' so no foul.
I think it's going to take some societal change, but we will have to.
Computers are just an extension of ourselves. The people making these systems will make mistakes, these systems will kill people (and arguably already have), but hopefully they will kill less than we currently do by not having them.
There's this stigma around "computers" where people treat them differently than everything else. Yes, you're getting into a car whose programming could cause you to die, but you are always relying on this kind of thing. Getting in a regular car, you are relying on the engineer who designed the braking system to have been correct, you are relying on the factory worker to have been paying attention that day, and the QC robot to have been programmed correctly, etc...
We need to get away from that, we need to treat the computer in your car as you would treat it's braking system. Accidents will happen, people will get hurt, bugs will occur. If they are systemic, there will need to be changes, recalls, and fixes. But the car will never operate in 100% of conditions flawlessly, and neither will you.
The big difference is, if I make a mistake and hurt myself, my family, or others, I have myself to blame. If my car makes a mistake though, I have someone else to blame.
Even though the statistics may point to self-driving cars being safer in the aggregate, would an individual be willing to give up control to software which might make a mistake and kill them? Even if, statistically, they might be safer?
Will people make the rational choice, or the emotional choice?
just film and show a complete flight from an airliner from the cockpit. flights are fully autonomous except take off and landing, which still are computer assisted.
all crashes occur when the pilot overrides the flight system or the runway has no support for computer aided landing (ils).
thats the only reason flying is statistically safer than driving.
Yeah. We do (or should) typically drive "defensively." But in practice that doesn't mean that we defend against everyone potentially pulling some utterly crazy stunt at any moment. For example, we watch for pedestrians but we don't actually worry too much that there might be someone hiding behind a tree who is going to jump into traffic. And we may be somewhat cautious at cross-streets but we still have a general expectation that a car isn't going to just randomly blow through a red light in the middle of its cycle.
Perhaps the AI also needs to be able to perceive the motivations of the humans ie read body language. If the joggers suddenly turned their heads to the left and twisted their bodies etc, then that would to a greater degree warrant the extreme caution. There's a lot of work cut out for the machine learning engineers.
I agree, but until then I'm guessing they are going to play it safe.
That being said, looking at that portion of the video again it looks like this may have just been the system having trouble figuring out if the people were in the road until it was closer.
I was wondering if stopping for the joggers was more of a bug than a feature. I don't drive much, so I'm definitely not an expert about what is best in that situation, but I would have drifted over the middle line to give them a nice berth and kept going.
That's really weird choice, who are picking these songs? :) I can't imagine they've licensed the music either... I guess next up is the "Highway to Hell" video
Haha, I find the image of him doing that, and no one daring to try change his mind, hilarious for some reason. I can see it escalating:
"So for our next video let's set up smoke machines on the road here, and then we'll have a camera team in front capturing it coming out of the smoke in slow-motion with "Smoke On the Water" as soundtrack. It's going to be amazeballs!"
I think the musical choices are awesome. It's so much better than choices that seem like all the marketing people debated in endless meetings who it might offend and so on. The company has some personality (cult of personality, maybe, but whatever :) )
...so...going on basic understanding of IP rights management...
If I was an investor I'd be super fucking pissed off Tesla is squandering capital on NOT CHEAP sync rights for Rolling Stones songs to be used in their PR materials.
If I was an investor I'd be super fucking pissed off Tesla is ignorant enough about Intellectual Property that they'd use a Rolling Stones song in their PR materials without paying the NOT CHEAP sync rights.
No credits at the end? Yeah, I get the feeling they didn't pay. Before you want to dog pile me for doubting if Tesla did things right, keep in mind this is a company that blows its own deadlines and wants to lock in its IP...well, I've seen a thing or two over the past 20 years of online life and I remember when Microsoft paid out the nose to get the Rolling Stones to license "Start Me Up" for a Windows campaign. The rights holder to some Rolling Stones material broke up The Verve following "Bittersweet Symphony" case.
Yes, I'm an IP asshole, but I'm a lot better at it than most.
I tweeted the Vimeo link to the Rolling Stones so hopefully we'll get to the bottom of whether Tesla paid or not. Why is this of any importance? Because if it was my song they were using I'd pursue the $250,000 per infringement max penalty for being a billion dollar company and shamelessly ripping off an artist. Just because it's the Rolling Stones, who I wish would retire, doesn't change the rules on the books, as much as Elon & some of the culture might wish was the case...
The $250k penalty doesn't apply, because sound recordings made prior to 1972 (like Paint It Black) aren't subject federal copyright law, but to earlier state copyright laws. In this case, that's supposedly California Civil Code, specifically articles §980 and subsequent. At least that's what it seems to me - IANAL.
The Benny Hill theme tune was a pretty unfortunate choice of background music, not sure if that's the impression they wanted to make (maybe it was, not sure).
When the last video came out, there were quite a few people saying "wait, that video's sped up, right? It doesn't seem to be taking turns at a safe speed."
I expect Tesla got feedback indicating that some people didn't notice that the last video was sped up, so they decided to make this video a lot more obvious by speeding it up even more and adding the music. I agree, it's not really a good choice, but I can see how they might have come to that decision.
The tech community tends to be a bit dour and pompous. We tend to overblow every tiny announcement into some world-changing event. Look at Apple's recent announcement video for their $300 book for a particularly gratuitous example.
It's healthy for Tesla to retain a sense of self-deprecating humour. Self-driving cars are massively important, but they're also a bit silly and strange. There's no harm in acknowledging the absurdity of a car that drives itself. There's no harm in bursting your own bubble.
I don't know about other countries, but at least in the Netherlands the Benny Hill song is used mainly in videos to ridicule the subject. I remember when funny 'Home Videos' were a thing on TV (before cats took over on the interweb) this song was used in about 90% of them. So to me, using this song in something as serious as a self driving car demonstration is pretty weird.
This song is also often used in a recurring absurd segment in cartoons when some people are running after each other and they go into / come out of random doors in a hallway.
If you look at it from the perspective of the sane and rational (Tesla autopilot/a cartoon viewer) viewing down to the ridicule (Human drivers and their emotions/absurd cartoon scenario) I think it is well placed :)
On the Internet, specifically on the Something Awful forums, it was was infamously paired with footage from 9/11 and posted either the day of, or the day after, the attacks. The video is called "tribute.wmv": https://youtu.be/2AYujWCCHRk
I suspect it has to do with the video being sped up the same as the segments on the Benny Hill were, which masks one of the challenges of the self driving car.
From my experiences, at least the northeastern us, driving the speed limit, in the traffic density shown in the demo, almost instantly attracts tailgating drivers who want to go faster.
Large parts of cities will be turned into roads for driverless cars to ferry themselves to and from the office 2x as many times to save the owner $10 on parking. Traffic will go slowly because driverless cars don't care about speed when no one is inside. And then all the plebs still driving their own cars will be stuffed.
This is kind of true, as much as this tech is fascinating, I can't help but feel trains and better planned cities are the real solution to the car problem.
Even in parts of Europe I noticed they put big trucks on trains to automate a large part of the journey and a driver finishes the last leg of the trip.
Trains are also much easier to automate, I also imagine a train track costs less to maintain than a road.
I never said there wasn't, however; What is obvious is that if taxis and trucks (privately owned or not) are now so cheap to run because there are no labor cost involved in having them on the road, then the next problems we will face are increased congestion, increased pollution (electric cars still consume a lot of power), increased road degradation. The only way to control this might be through increased taxes or car registration fees.
Jaw dropping.
I as a European born in a not so fortunate part of Europe (EE) have to really admire America. Americans deliver. Cocky europeans keep on whining how much culture and quality of life they have but at the end of the day they are just insecure about themselves.
1/5 of my countrymen, more than a 1m people escaped from communism to this great country.
* DISCLAIMER : I am aware it's not all wine and roses ;)
So it appears it's reading the road signs? I see it stops on the intersection with a stop sign where's no traffic lights. But what if the sign is damaged or removed? The car will go straight without stopping?
I hope they are using GPS/Map data for existing signs along with visual identification of the signs. Even when the gps is not accurate enough, the system should amplify the accuracy of the location based on permanent objects and the topography.
Either on the first video or some news, i think i heard the ability to drive where the lanes are not clearly defined. Things such as snow would absolutely requires such reconstruction.
Yes, i mean what happens more often- if sign is not fully removed, but bent by collision or turned sideways by hooligans or something. Human would recognize that. Car will not.
Sounds like the cars should share a centralized database of critical sign locations, and cross reference their GPS location with the database. When they approach a location that the database says should have a sign, and one is not detected, it should behave as if there was a sign, and report the incident to somebody.
> But what if the sign is damaged or removed? The car will go straight without stopping?
Probably the same thing a new-to-the-area human would do if the sign was removed. Where I live there are many intersections where only 1 road has stop signs. I'd imagine there are other areas were the norm is a 4 way stop.
As for damaged, I would imagine it can still recognize the sign as often in image recognition you do not need the whole picture to confirm a match.
If there is no STOP sign, I don't see how it would know to stop. However, what's interesting is if it goes through an intersection with a STOP sign and the next day the sign has been knocked over by strong wind - will it stop? What if the sign is missing, but there's a STOP sign on the opposite end of the intersection - a human driver will infer that his own side also has a STOP sign, but will the car?
A stop sign is paired with a stop line in the road which can be seen. It will also see other cars and react accordingly if they are not following their own stop signs. An SDC doesn't really need a stop sign.
There seem to be some issues. E.g. it recognizes the joggers (0:55) as obstacles even though they're obviously (to a human) on the sidewalk. And when taking a right turn (1:01) it mistakes a hydrant and a parked car in a parking bay on the other side for an obstacle and stops for a moment before noticing that they're not actually in its lane.
I think a polite driver may have slowed down gradually to not startle the joggers, but not stopped if it were avoidable, especially since there was a car behind the tesla.
In the latter case i guess the field of view of the cameras is not wide enough to assess those things during a turn in advance?
It passes a single jogger without an issue beforehand, but they were two when the car stopped and left one was relatively close to the lane when the car was approaching. Maybe it wouldn't stop if it was a single jogger.
If you pause it at 0:54 it clearly made a mistake. The foot of the left person is outside the green box, the car thought the person was further forward and to the left of where they really were. Good progress but not ready for prime time.
i don't know much about self-driving, but i would not over-interpret the green boxes. Also i don't know if your assessment about the range is true. Seems like a bug.
The car might have been playing on the safe side. If you look at the joggers carefully, only the left one is marked as an "in-path object", the other one has just an "object" box. They could probably tune down the collision prediction algorithm just a little bit, but that has to be done very carefully.
Why bother, err on the side of caution with pedestrians. The driver is already benefiting from auto pilot, a minor delay due to safety of other road users would be acceptable to most.
It's inconveniencing other drivers on the road and much more importantly this erratic stopping from a human perspective might make it more likely to cause an accident. Granted the accident would legally be the meat driven car's fault who bumps into the suddenly stopped Tesla. It would still be nice to play well with human drivers who don't have the luxury of relaxing in a self driving car.
> Granted the accident would legally be the meat driven car's fault who bumps into the suddenly stopped Tesla.
Exactly. If you bump into the car in front, because they stopped for a percieved hazard, it is your fault for tailgaiting, not their fault for percieving a hazard.
That's all very nice in theory but, in the actual real world, if you do a lot of unpredictable braking or other unexpected behaviors, you will basically get hit all the time even if those accidents aren't technically your fault.
Soon people will either adapt and end up following the law, or they will be unable to pay for insurance, because of the amount of claims that they're taking out against it.
I generally not a fan of social darwinism, but in this specific case, I think that something of the sort will have a positive effect. How many people have been injured because the car they were hit by was being forced to go faster, because of tailgaiting?
I agree, legal or not, I very much rely on other drivers "signals" when driving. Over the years you learn to predict what the driver in front of you would do in a normal situation and adjust your driving in advance. I'm still able to break on emergency but when you can clearly see way ahead of the car in front there is no obstacle and yet they still brake it's extremely risky and frustrating. Young drivers here usually mark the car some way to warn you to be super aware of their unexpected behavior, I'd expect self driving cars would to the same.
It means that you would have been surprised if the jogger was making a jump to her left (avoiding hole in her path for example), forcing any driver to hit the brake.
Instead of relying on other drivers' signals, you could instead use them as extra warning, and instead of driving so close to them you're putting yourself and them and potentially others in danger if they brake when you didn't expect, you could back off to a safe distance.
Well, not sure where you read I drive close to anyone. What I'm trying to say is that erratic drivers are dangerous because you learn to predict what a normal driver would do in a typical situation (e.g. jogger too close to the lane => slow down to a safe speed and navigate towards the center).
If a self driving car doesn't reproduce the most common driving style the other drivers should be extra careful around it, some kind of mark on the outside could help.
I wish there were some external visual cue like a light or something to indicate a car is using autopilot. That way you could give them a wider buffer.
The problem is that pedestrians on the sidewalk could potentially become an unavoidable obstacle at any time if they walk into traffic. For this reason there can never be a 100 percent safe system until vehicles and pedestrians are physically separated by barriers or separate structures which may happen eventually in some new urban designs.
Interestingly, it behaved just like a young learner-driver would - play it safe, be cautious, and give extra space to the joggers because a new driver is a little uncertain about controlling a 6,000 lbs hunk of moving steel so close to delicate humans.
It's astounding to realize the AI will learn and change it's habits, just like a young human driver learning.
I think the hydrant is the same problem: head on you don't have any parallax effect to do know the distance, so if you don't detect where the thing is rooted to the ground you might guess it's in the way.
and it completely fails do detect the right end of the lane (the moments before I was amazed how it still drove so fluid)
I think 0.25x is the right speed if you want to watch it in realtime
Does this new Telsa autopilot use stereo cameras? Would stereo camera help to have a better 3D vision? Or is a video stream enough to get the 3D data.
From the video it looks like lane detection and 2D object recognition.
It might use a structure-from-motion (SfM) running on CUDA to get a 3D point cloud out of the 8 cams. Does someone know more about how their new auto pilot works?
They do not. They have 3 forward cameras (1), but they all point directly forward and are for separate use cases. I've always thought it would make more sense to have two cameras at either side of the windshield rather than one in the middle. The downsides are you need a new method to clear water and debris since the wipers won't hit that area, and there might be limited FOV.
Stereo might require some level of calibration to account for camera alignment, maybe they want to avoid that? I would think that implementing stereo would be fairly cheap both computationally and in hardware- if they can get by this well without it, they could pare the stereo computations down to a few frames per second, on specific objects in the scene. Maybe a future update will add binocular cameras in the dash, but they are pretty confident about their current hardware.
I would guess that the car couldn't predict the direction the joggers were moving in. They could be jogging towards the road. We as humans would know that they were running along the road because we see their backs but not necessarily from extrapolating their movement
I would guess that the car couldn't predict the direction the joggers were moving in. They could be jogging towards the road. We as humans would know that they were running along the road because we see their backs but not necessarily from extrapolating their movement
Wow so many comments about the Benny Hill music. I for one love that Elon Musk's quirky sense of humour. So many people take themselves far too seriously.
I think Elon Musk had a lot to do with the video, I imagine the music was his choice. Very few other people could suggest this tune due to its previous comedic use.
While watching Harry Potter. Perhaps the Harry Potter theme would have been more appropriate, but I'm sure that licensing would have been prohibitive. I can hear some similarities, so maybe it was an NN based suggestion
That's still just an allegation from the driver of the semi (he said he heard harry potter playing, and there was a portable DVD player found, nothing more was released).
But what is confirmed is that he was speeding at the time as well, and after that incident Tesla put out an update preventing autopilot from being set for more than 5mph over the speed limit.
Very cool demonstration (despite curious choice of music!).
I did notice that the lane markings are suspiciously bright and well-painted throughout the video. Almost as if they'd been touched up by Tesla! Or maybe this area just has particularly diligent road-painting crews.
In many parts of the real world you'd be lucky to get such clear lane markings throughout the journey, certainly not here in the UK or in places that get winter ice & snow.
In my autopilot gen 1 Tesla, when the software was first released, lane markings had to be really clear, but as the software has matured, it now does less than perfect lane marking in pretty heavy rain without issue.
I think the current-gen autopilot vision code is a completely different codebase from a different supplier running on different hardware than the gen 1, though, so all those improvements have probably been thrown away.
I know I'm probably gonna get downvoted, but I cannot not bite. I'm super happy I kept my Saturn games. I still play them occasionally and some of them are very valuable now. I also hope you meant PS3 or PS4 when you said "PlayStation" and were not trying to revive a long gone console war that was very painful for some of us.
Actually, i think in AP 1 they used MobileEye and Tesla has their custom software to compliment it and with the switch to AP 2 "It runs a Tesla-developed neural net for vision, sonar, and radar processing." https://cleantechnica.com/2016/10/25/inside-nvidias-new-self...
So it seems like the analogy is more akin to Xbox 360 and Xbox One, hardware got an upgrade but they were able to run Xbox 360 games on it with some work.
Despite the different codebases, they still have the datasets that they've collected over the years, and that's really what matters (for things like lane detection.)
It would be quite weird if they didn't retrain their new models with all that data.
What does this mean? So they send all 8 video cams data to Telsa? This would mean ould mean very high upload traffic and costs - sounds unrealistic. I assume Tesa sends the GPS data, and sensor data and maybe occasional photos back to HQ, but not full videos of all 8 cameras - or do you have a source/reference?
Holistic path planning uses the entire scene to identify the lane, not just the lane markings.[1] This is exactly how a human drives on a rural road.
Once enough fleet learning data is gathered, high resolution maps will mark out the road surface to high accuracy (cm).
Of course that's no good unless the car knows where it is within cm! For this Tesla uses sensor fusion with GPS, an onboard IMU (similar to a smartphone), four wheel wheel odometry (via the ABS sensors), steering angle, and torque delivery. Owners report that the Autopilot 1.0 cars know their position in a parking space to within about 10 cm. And it has demonstrated high accuracy and low drift in no-GPS situations[2].
This type of sensor fusion (Kalman filtering) was originally developed for the Apollo spacecraft, and was one of the responsibilities of the famed Apollo Guidance Computer.
The first self-driving video came out a few days after Episode 1 of Westworld, where an a cover of this song featured prominently (I'm trying not to spoil it).
Elon confirmed on Twitter it was inspired by this. [1]
I believe Tesla's AP 2.0 is built on NVIDIA's platform (with probably a lot of custom stuff, of course). If you want to learn more about how that kind of system works, that CES 2016 demo is very instructive, especially from 42:50 onwards: https://www.youtube.com/watch?time_continue=2573&v=KkpxA5rXj...
That's really impressive, but I really want to see it driving through rush hour in a densely populated old Italian city. Let's continue pushing the envelope.
I would love to see how autopilot works in different cities (e.g Mexico City, Rome, Buenos Aires). I Just found "I tested Tesla Autopilot in Manhattan traffic — and lived to tell about it" http://mashable.com/2015/11/06/tesla-autopilot-new-york/
What's the most amazing is that Elon mentioned on twitter that it's only using data gathered in the past month since they announced the hardware/first video.
That's a little bit of a misdirection. They certainly learned a lot about how to build their models and what data is important, and this results in more effective training.
Fastest speed clocked in at 35mph during the entire trip with an average of 15-25mph for remainder of the time. Does this suffice a realistic use case? Specifically in US?
In town, with the level of traffic seen? Yeah, it's keeping to the posted speed limits. Realistically, I'd be doing 5 to 10 over except in explicit residential zones, when there are cops (limit to 5 over), and when traffic doesn't allow it.
I think watching this in real-time would be more informative. At 1:32, it pull out on what looks like could be a busy road, then stops before starting again. It doesn't look so bad sped up in the video, but not great in reality.
I agree about real-time. I get that it would be more boring but I find it hard to judge how the performance differs from a typical human driver in the sped-up version.
Well it was certainly fun to watch. Aren't there more detection methods than just cameras? This is certainly the best one I have seen from them but I still want to see it at night, in rain, and such, times when people need the most help with driving. This was still too nice of weather to get a good impression of how far its come.
Also, if the driver wants to exceed the speed limit of a road will autopilot allow them? If so, who becomes liable since the system will obviously know the speed.
If the road had not been curved when it came upon the joggers would it have passed without stopping? Is that something that needs work on? I understand its erring on the side of safety.
So next drives, a night run, a rain run, and if they are really ballsy then both at one time. Would be cool to have someone throw something in front of it while its going through the parking lot, beach ball, etc.
It seems like the right-bound of the lane (the right red line in the midview camera) was not calculated, so it thinks the joggers were inside a big lane. that'd be my guess as to why they would think the joggers were not "outside" of the lane for the system to breeze by. At 1:23-1:24, there's a person walking their pet and it looks like they were really close, but the camera was able to detect the right boundary.
254 comments
[ 2.8 ms ] story [ 237 ms ] threadAnybody knows if these 3 cameras on the right are the only ones used by the auto pilot?
Looks pretty promising, not bad, especially since the trend in software (and sensors) is to get better over time.
90 - 95% of the time I am driving long stretches, I spend on highways. No problem if only that part would be done selfdriving. Those few minutes in busy town centers I do not mind to do myself. No problem of driving short stretches (10 - 15 min) either.
Buyers were convinced by "cruise control", which just (barely) keeps a set speed.
Doing those things right now in a Tesla is probably not safe just now, but it shows that level 5 autonomy is not so far fetched and 2020 is a realistic date.
The tech is impressive but it has to be absolutely bulletproof. I like Tesla but I find it annoying that Musk doesn't deliver anything on time and it seems like Tesla delivers way more demos than real products.
Autodriving 100% of the time is where it gets revolutionary (mass-layoffs of truck drivers, taxi drivers, summon features that park cheaply outside of downtown).
The former is great, but there's a lot of hype going around right now so it's not strange for people to be cynical.
It's all about predictability. Even just full motorway / major road driving would be pretty awesome, and the car would know when it's coming up to a section you need to drive and you can have plenty of warning. Not "HELP DRIVER I SAW EITHER A LEAF OR A CHILD QUICK TAKE OVER"
If/When it's good enough that I can sleep during a long highway trip (and pull into the last rest stop before my exit if I don't wake up) then I'll be happy.
live in the gonnies? own a car. hailing one would take forever. having cars near you in the gonnies,that you dont own wouldn't be economical.
basically, there would be less vehicule owners but not that many less.
For example, friends of mine recently bought a summer cottage. It's a little over an hour's drive. They don't have a car, so they currently rent one or try to borrow one when they want to go there. Once there, the car will probably sit still for a week.
Currently, there's no really good solution to this. Renting is expensive if you just need to drive a couple of hours a week. Buying a car might make more economic sense in the long term, but it's still pretty expensive for the purpose. Taxi rides like that cost an arm and a leg too. But if the car could drive itself out of the woods...
Of course it's a question of pricing, and I'm guessing we're still some way off from having self-driving cars navigate dirt paths that pass through people's yards as is the case here.
Anything less than 4 is asking for trouble.
I'm not talking about "oh hey, here's a situation I can't handle" and more "take over now".
Here's the first of 100 self-driving cars Volvo will put in customer hands in 2017.[1] This shows close-ups of all the sensors - high radar and camera behind the windscreen, low LIDAR and camera in the nose, and side and down-looking cameras on the door mirrors. No high-mounted LIDAR, though. These vehicles are intended for self-driving in all weather conditions in Gothenburg, Sweden.
[1] https://www.youtube.com/watch?v=nwcSeRE-XEY
They certainly are saying they'll only release a Level 4 vehicle - the Tesla is still expecting a human to take over.
Tesla's old system was Level 2. Google and Volvo have level 3 in test. Tesla's new system is supposed to be Level 3. There's a bright line between level 2 and 3 - NHTSA says that at Level 3 and above, the manufacturer is responsible for accidents and cannot blame the driver.
[1] http://www.nhtsa.gov/About-NHTSA/Press-Releases/U.S.-Departm...
However statistics say that most of the accidents happen at the beginning or towards the end of the trip. There are environmental reasons (city vs. sub-urbs vs. cross-country) and there are human reasons.
What's weird is the period we're entering, where the technology is becoming more and more capable, but we're still a very long ways away from full auto. Yet the hype machine says it's just around the corner. I'm betting we're still 10-15 years out. So how do we handle situations where the driver has to pay a little attention, but not that much? In airplanes, this has been a very dangerous area -- and things in airplanes usually happen much, much slower than they do in traffic.
I think it would be worthwhile to figure out how to shut down the hype machine, or at least tune it down a few notches. Dreams and visions are fine, but every time I switch the current tab from HN to Facebook, I get to see overhyped bullshit predictions about AI, nanotech, self-driving cars, etc. being just about the corner, as evidenced by whatever misinterpreted paper or video news sites could find that day. And the non-tech crowd is totally buying that. Not just regular folks, but also investors and politicians and all the other people who then end up causing resources to be wasted because they buy into the bullshit.
In 2013 an oft-cited and alarming study out of oxford suggesting most jobs will be automated over the next decade was released. It was followed by a series of influential books whose authors ran the lecture circuit: 'The Second Machine Age', 'The Zero Marginal Cost Society', 'Superintelligence', and 'The Rise of the Robots'.
There were some bold bold and well publicized statements from respected luminaries such as Bill Gates, Elon Musk and Stephen Hawking.
Aggressive maneuvering to hoover up machine learning talent, and bold investments from automakers pursuing autonomous driving has only added gasoline to the fire.
Automation became the talk of the town at Davos in Switzerland. There's been a rising chorus from Basic Income supporters.
Now the hype is out of control. Nobody is actually looking at the technology. It's okay, though. Hype is self-correcting.
Or, as we call it in China, "driving".
all at once: moped taking over from the right, in a round about, truck entering the roundabout without slowing down. donkey & cart on the side. woo hoo!
A human driver (well, at least myself) would simply have lifted the foot off the accelerator somewhat and drifted closer to the centre line when passing - seeing as there was no oncoming traffic.
Maybe it knows that if the joggers were to suddenly jump out, doing what you or I would do isn't enough, so it slowed down to a near stop just in case.
Also, it's a bit of a "blind corner", so drifting out toward the middle of the lane might not be the best idea (again, even though it's probably something I as a person in that scenario would have done).
Truth is, we all drive fairly irresponsibly, trusting in the rules of the road and human self-preservation to keep most stuff out of our way. And if we do hit somebody who dashes out on the freeway, well that's an accident or 'their fault' or 'nobody's fault' so no foul.
Will we allow automated drivers that privilege?
Computers are just an extension of ourselves. The people making these systems will make mistakes, these systems will kill people (and arguably already have), but hopefully they will kill less than we currently do by not having them.
There's this stigma around "computers" where people treat them differently than everything else. Yes, you're getting into a car whose programming could cause you to die, but you are always relying on this kind of thing. Getting in a regular car, you are relying on the engineer who designed the braking system to have been correct, you are relying on the factory worker to have been paying attention that day, and the QC robot to have been programmed correctly, etc...
We need to get away from that, we need to treat the computer in your car as you would treat it's braking system. Accidents will happen, people will get hurt, bugs will occur. If they are systemic, there will need to be changes, recalls, and fixes. But the car will never operate in 100% of conditions flawlessly, and neither will you.
Even though the statistics may point to self-driving cars being safer in the aggregate, would an individual be willing to give up control to software which might make a mistake and kill them? Even if, statistically, they might be safer?
Will people make the rational choice, or the emotional choice?
all crashes occur when the pilot overrides the flight system or the runway has no support for computer aided landing (ils).
thats the only reason flying is statistically safer than driving.
A video where a car manages to stop because a drunk falls into your lane will make the stock soar.
A video where a car hits an old lady who jumps in front of your car will make the stock crash.
There are people on either side who want this technology to succeed or fail (not self-driving, i mean electric cars)
getting there 30s later on average is worth killing grandmas? with automated cars you dont even have to pay attention or care. get in get out.
That being said, looking at that portion of the video again it looks like this may have just been the system having trouble figuring out if the people were in the road until it was closer.
"So for our next video let's set up smoke machines on the road here, and then we'll have a camera team in front capturing it coming out of the smoke in slow-motion with "Smoke On the Water" as soundtrack. It's going to be amazeballs!"
"Umm, ok Elon..."
"Put in a lion roar as well"
"I quit."
[1] https://news.ycombinator.com/item?id=12995428
If I was an investor I'd be super fucking pissed off Tesla is squandering capital on NOT CHEAP sync rights for Rolling Stones songs to be used in their PR materials.
If I was an investor I'd be super fucking pissed off Tesla is ignorant enough about Intellectual Property that they'd use a Rolling Stones song in their PR materials without paying the NOT CHEAP sync rights.
No credits at the end? Yeah, I get the feeling they didn't pay. Before you want to dog pile me for doubting if Tesla did things right, keep in mind this is a company that blows its own deadlines and wants to lock in its IP...well, I've seen a thing or two over the past 20 years of online life and I remember when Microsoft paid out the nose to get the Rolling Stones to license "Start Me Up" for a Windows campaign. The rights holder to some Rolling Stones material broke up The Verve following "Bittersweet Symphony" case.
Yes, I'm an IP asshole, but I'm a lot better at it than most.
Appropriate in bringing light heartedness to a typically sped up video.
Can everyone chill the fox out about a theme tune?
I expect Tesla got feedback indicating that some people didn't notice that the last video was sped up, so they decided to make this video a lot more obvious by speeding it up even more and adding the music. I agree, it's not really a good choice, but I can see how they might have come to that decision.
The tech community tends to be a bit dour and pompous. We tend to overblow every tiny announcement into some world-changing event. Look at Apple's recent announcement video for their $300 book for a particularly gratuitous example.
It's healthy for Tesla to retain a sense of self-deprecating humour. Self-driving cars are massively important, but they're also a bit silly and strange. There's no harm in acknowledging the absurdity of a car that drives itself. There's no harm in bursting your own bubble.
Come to think of it, I just feel like making a montage.
Even in parts of Europe I noticed they put big trucks on trains to automate a large part of the journey and a driver finishes the last leg of the trip.
Trains are also much easier to automate, I also imagine a train track costs less to maintain than a road.
This still means the car problem needs solving.
1/5 of my countrymen, more than a 1m people escaped from communism to this great country.
* DISCLAIMER : I am aware it's not all wine and roses ;)
IIRC it would automatically make a note of your location when you pass speed bumps or bad patches of road and adjust the suspension accordingly.
Either on the first video or some news, i think i heard the ability to drive where the lanes are not clearly defined. Things such as snow would absolutely requires such reconstruction.
Probably the same thing a new-to-the-area human would do if the sign was removed. Where I live there are many intersections where only 1 road has stop signs. I'd imagine there are other areas were the norm is a 4 way stop.
As for damaged, I would imagine it can still recognize the sign as often in image recognition you do not need the whole picture to confirm a match.
[0] https://en.wikipedia.org/wiki/Priority_to_the_right
I understand that whether the driver was paying attention or not, he or she would be at fault, I'm just pointing it out.
I think a polite driver may have slowed down gradually to not startle the joggers, but not stopped if it were avoidable, especially since there was a car behind the tesla.
In the latter case i guess the field of view of the cameras is not wide enough to assess those things during a turn in advance?
Exactly. If you bump into the car in front, because they stopped for a percieved hazard, it is your fault for tailgaiting, not their fault for percieving a hazard.
I generally not a fan of social darwinism, but in this specific case, I think that something of the sort will have a positive effect. How many people have been injured because the car they were hit by was being forced to go faster, because of tailgaiting?
If a self driving car doesn't reproduce the most common driving style the other drivers should be extra careful around it, some kind of mark on the outside could help.
"by your command"
It's astounding to realize the AI will learn and change it's habits, just like a young human driver learning.
If you don't see the legs and don't have very accurate 3d vision, the object might just as well stand further away in the middle of the road: http://malea.lacerta.uberspace.de/up/e92e6492-ce56-468b-8e09...
I think the hydrant is the same problem: head on you don't have any parallax effect to do know the distance, so if you don't detect where the thing is rooted to the ground you might guess it's in the way.
and it completely fails do detect the right end of the lane (the moments before I was amazed how it still drove so fluid)
I think 0.25x is the right speed if you want to watch it in realtime
From the video it looks like lane detection and 2D object recognition.
It might use a structure-from-motion (SfM) running on CUDA to get a 3D point cloud out of the 8 cams. Does someone know more about how their new auto pilot works?
Stereo might require some level of calibration to account for camera alignment, maybe they want to avoid that? I would think that implementing stereo would be fairly cheap both computationally and in hardware- if they can get by this well without it, they could pare the stereo computations down to a few frames per second, on specific objects in the scene. Maybe a future update will add binocular cameras in the dash, but they are pretty confident about their current hardware.
1: https://www.tesla.com/en_EU/autopilot
I'd have a hard time believing that, to be honest.
But what is confirmed is that he was speeding at the time as well, and after that incident Tesla put out an update preventing autopilot from being set for more than 5mph over the speed limit.
That's not correct. I routinely drive with autopilot set to +10mph over the speed limit.
I did notice that the lane markings are suspiciously bright and well-painted throughout the video. Almost as if they'd been touched up by Tesla! Or maybe this area just has particularly diligent road-painting crews.
In many parts of the real world you'd be lucky to get such clear lane markings throughout the journey, certainly not here in the UK or in places that get winter ice & snow.
There are some testing scenarios which Tesla should be able to use, ut none of the software from MonilEye has relevance to Nvidia's platform.
I also read another quote from the Tesla engineering team that "nearly all of it" (the data collected from AP 1) can be used for the new AP system https://electrek.co/2016/11/07/tesla-bought-1-million-data-s...
So it seems like the analogy is more akin to Xbox 360 and Xbox One, hardware got an upgrade but they were able to run Xbox 360 games on it with some work.
It would be quite weird if they didn't retrain their new models with all that data.
Holistic path planning uses the entire scene to identify the lane, not just the lane markings.[1] This is exactly how a human drives on a rural road.
Once enough fleet learning data is gathered, high resolution maps will mark out the road surface to high accuracy (cm).
Of course that's no good unless the car knows where it is within cm! For this Tesla uses sensor fusion with GPS, an onboard IMU (similar to a smartphone), four wheel wheel odometry (via the ABS sensors), steering angle, and torque delivery. Owners report that the Autopilot 1.0 cars know their position in a parking space to within about 10 cm. And it has demonstrated high accuracy and low drift in no-GPS situations[2].
This type of sensor fusion (Kalman filtering) was originally developed for the Apollo spacecraft, and was one of the responsibilities of the famed Apollo Guidance Computer.
[1] http://wccftech.com/tesla-autopilot-story-in-depth-technolog...
[2] http://motherboard.vice.com/read/how-teslas-model-s-tracks-y...
I really cannot wait for next video, featuring Jean Jacques Perrey's "Little Ships".
The first self-driving video came out a few days after Episode 1 of Westworld, where an a cover of this song featured prominently (I'm trying not to spoil it).
Elon confirmed on Twitter it was inspired by this. [1]
e: Added twitter thread link.
[1] https://twitter.com/elonmusk/status/789020178969276416
Something you wouldn't wish on anyone who is even remotely your friend.
And then there's the magic roundabout in Hemel Hempstead! https://www.youtube.com/watch?v=ih223jffmek
Also, if the driver wants to exceed the speed limit of a road will autopilot allow them? If so, who becomes liable since the system will obviously know the speed.
If the road had not been curved when it came upon the joggers would it have passed without stopping? Is that something that needs work on? I understand its erring on the side of safety.
So next drives, a night run, a rain run, and if they are really ballsy then both at one time. Would be cool to have someone throw something in front of it while its going through the parking lot, beach ball, etc.