Increasing the diversity of the team members might be good for reducing bias, but the fact remains that darker skinned people are harder for machine vision systems to work with due to the lack of contrast.
Like that fuss a few years ago with the laptops (I want to say it was Toshiba but I'm not certain) not being able to use facial recognition to unlock the OS.
Also, this seems to be only vision systems - you'd hope LIDAR would be able to detect a pedestrian regardless of skin color.
I would a real surprise if models with LIDAR inputs can detect your skin color.
I also do not see what team diversity should do. Adding such a bias actually on purpose is way harder than one would think (and not the issue), and simply making a better model should be the goal of any team anyway.
In the study they do say that at night the dark skinned people could actually be detected better (according to their very small validation set). Something nobody would expected. So I'd really be worried if their model makes it into production. They still have some issues at least with sample size.
The thing is these types of models are really tricky to get right. When they are wrong/biased it is just because it is difficult to get right. I think it is stupid to frame this problem as a racial issue.
For those interested in learning more about these problems there are two interesting book to read.
"Technicaly Wrong" and "Weapons of Math Destruction"
> The report [...] hasn’t yet been peer-reviewed. It didn’t test any object-detection models actually being used by self-driving cars, nor did it leverage any training datasets actually being used by autonomous vehicle manufacturers.
1. The study talks about how some vision-based object detection systems don't work as well on black people.
2. The articles makes the leap to self driving cars, ignoring that self driving cars uses Lidar and probably custom made algorithms. To their credit they do say that it's a "potential risk".
3. Everybody shares (including this HN submission) or rewrite the article dropping the "potential" part.
how do idiots blame bias for everything? if you have a dark room of course it’ll be harder to detect a dark skin person! the same with a couch that’s black instead of white.
For all those not bothering to check the study. It actually says "At night the model was BETTER at DETECTING DARK skinned people than LIGHT".
Questionable result but still in there.
Sounds like a physical issue, about how dark surfaces reflect less light and the car can't analyze that information. They'll find a way to fix this eventually.
15 comments
[ 2.4 ms ] story [ 45.3 ms ] threadIncreasing the diversity of the team members might be good for reducing bias, but the fact remains that darker skinned people are harder for machine vision systems to work with due to the lack of contrast.
Like that fuss a few years ago with the laptops (I want to say it was Toshiba but I'm not certain) not being able to use facial recognition to unlock the OS.
Also, this seems to be only vision systems - you'd hope LIDAR would be able to detect a pedestrian regardless of skin color.
Is it also harder for human drivers?
I also do not see what team diversity should do. Adding such a bias actually on purpose is way harder than one would think (and not the issue), and simply making a better model should be the goal of any team anyway.
In the study they do say that at night the dark skinned people could actually be detected better (according to their very small validation set). Something nobody would expected. So I'd really be worried if their model makes it into production. They still have some issues at least with sample size.
The thing is these types of models are really tricky to get right. When they are wrong/biased it is just because it is difficult to get right. I think it is stupid to frame this problem as a racial issue.
1. The study talks about how some vision-based object detection systems don't work as well on black people.
2. The articles makes the leap to self driving cars, ignoring that self driving cars uses Lidar and probably custom made algorithms. To their credit they do say that it's a "potential risk".
3. Everybody shares (including this HN submission) or rewrite the article dropping the "potential" part.
If a automated car is better at detecting threats on the road than humans then there is little point to using them.
Step 2 - show that the model detects dark-skinned pedestrians 5% less often than light-skinned pedestrians
Step 3 - write a clickbait title implying that self-driving cars will run down dark-skinned pedestrians with precisely zero evidence
Step 4 - post the article on HN with an even more inflammatory title than the actual article
Take this up with HN