Our privacy is under attack on all fronts by our own government. It seems hard to imagine a feasible way to defend against this. Even well thought out legislation will likely just be ignored with impunity. Is privacy of any sort a lost cause at this point?
Pretty much. There are things you can do, but the majority of people won't do them. Our political system is showing its age, being completely gerrymandered, bought by major corporations, and is incapable of changing unless it benefits the rich in some way.
Wow, calling the system 'race-blind' is the stupidest thing I have heard in a while. Of course the computer has no conception of race, but that has no bearing on whether the system is accurate for all skin shades.
Yeah, imagine your Trumpista ICE agrent walking down the street with a camera that flags people who aren't in the database (ie, no valid driver's license or passport). Welcome to 2017
This doesn't mean much. They can flag you for whatever, but not having ID isn't a crime, so a flag is the most they can do. Let's say they think that not having an ID means you're an illegal immigrant. They have to prove that in court, while you don't have to prove that you aren't.
Police can do whatever they want. Here's an article about several people who were arrested (and booked into custody) for "failure to identify" themselves[0]
More accurately, police "will" do whatever they want; they aren't lawyers and frankly barely understand the laws they're supposed to enforce. They think they're allowed to do things they're not allowed to do all the time, because the reality is they're mostly thugs and simply aren't that bright.
As noted in the article, its only a violation if you fail to identify yourself after being arrested. What is not clear to me is whether it is sufficient to just give your name? I often go out without my wallet and I'm not sure why there would be any legal presumption that a citzen would produce some type of government document to serve as "ID".
If you're detained for a suspected immigration violation they don't even have to give you a hearing for 6 months. Criminal justice procedure does not apply to the immigration detention system.
> Database contains photos of half of US adults without consent, and algorithm is wrong nearly 15% of time and is more likely to misidentify black people
I think technology is often a very effective way to describe what "systemic racism" is.
Racism propagates easily, and race-blind / race-agnostic / race-indifferent behavior does not combat it or counteract it.
Imagine a majority-white tech team, largely color-blind, yet still (by virtue of economic and geographic segregation) in a mostly white-centric bubble. The training set might be largely white. If some preprocessing or feature-detection is taking place on the raw images, it might be tracking features that could have higher variability among white people while overlooking other features.
If you don't know to go out of your way to check for, include tests, and give thought to mitigating imbalances caused by race (because you "see no race") then you allow this problem to propagate. In the case of computer vision, you will inherit the "racism" of every library you import from.
This reminds me of gender biased found in Word2vec[1]. It is hard to accuse the dataset of being sexist, especially when viewing it as a reflection of language in news article. Yet, people using Word2vec use it as a representative of language, not a representative of language in news, with all the baggage in today's (and past) culture. As a result, sexism can propagate, autocomplete and query suggestions might be biased. Then spending habits might be biased, etc.
> In some cases, according to tests with female mannequins, small women were almost three times as likely as their average male counterparts to be seriously injured or killed. A study of actual crashes by the University of Virginia’s Center for Applied Biomechanics found that women wearing seatbelts were 47 percent more likely to be seriously injured than males in similar accidents.
That is a very interesting point regarding Word2vec, but it's inappropriate to call this example racism based on the evidence provided.
Only 16% of the US population is Black, while 70% is white. If the error rates between Caucasians and Blacks were the same that would, in fact, be racist - or at the very least undemocratic.
It's like asking that everything in the world be symmetric when only 10% of the world is left handed.
But then the humans can't be "color blind". They must explicitly go out of their way to undo the effects of the rest of the system by, E.g. Displaying extra care when dealing with a black suspect.
Your chance of getting confused BY a criminal is probably more connected to your ability to comprehend and communicate.
Perhaps you not understand the metaphor. Developing takes time and money to produce accuracy. You're advocating greater per-capita investment for an ethnic group. That's racial discrimination. You are advocating racism.
It should come as no surprise that there's a difference. It is therefore suspect that they avoid mentioning the magnitude of the difference; it must not support their case otherwise they would highlight it.
Critical reading, logic, and reason will help protect you from this type of manipulation and inadvertent racism.
In my naive mind, I can't help but think it might simply be a case of the contrast ratio being greater in lighter skinned people - therefore making the hit ratios higher. Greater contrast ratios will make facial features stand out more.
Exactly my thinking. I worked with a company that developed on a medical device that took subcutaneous readings extracutaneously, and despite being developed almost entirely by Asian engineers, it got the best data quality from fair-skinned Caucasians, and the worst from dark-skinned Africans, as it totally relied on the transmission of light through skin tissue.
Claiming racism without understanding the problem is both unfair to the people involved, and an unwise undermining of one's own credibility for the times when people really are acting in bad faith. That may be in this case, but that may not be, and I don't like guilty-until-providen-innocent no matter whom it's leveled against (even a Federal agency of which I'm not the world's biggest fan, and a database the likes of which make me think we need a constitutional amendment to protect the privacy of the citizens).
Even if it was no-one's intent that the system is racially biased, allowing such a system to continue operate after learning of such a bias is a racist act.
I'm inclined to agree with you, but what would you say if it turned out that after much effort the engineering team was unable to make performance equal across all skin tones?
What if the technical challenges of identifying a person's face mean it will always be challenging to identify someone when the light and/or contrast levels are lower? What then? Manually cripple the system and misidentify white suspects, too?
This post's comments are a great example of why technologists are fairly terrible at engineering social solutions, much the same way sociologists tend to be terrible at engineering technical solutions.
Ignoring known technical deficiencies in the system and assigning the same confidence levels in the output regardless of race could work out to be pretty racist, but what are they supposed to do, manually cripple the accuracy of the system down to the lowest common denominator?
In the medical device I mentioned above, it was medically useful for caucasians and lighter-skinned Hispanics, so-so with Asians, and almost useless for Africans.
So, what–don't go to market until the technology has parity across all skin tones?
but I actually wrote a pretty successful 3d radiology/cardiology package. So I understand medical devices.
I just wanted to hop in here and say that there is NO WAY the FDA would give you clearance to use the device you describe on African Americans under the circumstances that your posts have postulated.
It seems a no brainer that the facial recognition software should not be used on African Americans either. At least, if it was MEDICAL software... it would DEFINITELY NOT be cleared with that kind of a failure rate. And any provider or delivery network using the facial recognition software on African Americans would be on his or her way to prison.
But the regulations on legal software may be more lackadaisical than the regulations on medical software. Getting things exactly right may not be as important. And the penalties for misuse of medical devices are, without question, far more draconian.
> I just wanted to hop in here and say that there is NO WAY the FDA would give you clearance to use the device you describe on African Americans under the circumstances that your posts have postulated.
The FDA has approved drugs that are specifically intended to target medical issues in specific ethnicities, so I'm not sure about your assessment. That said, this product's market was outside the scope of the FDA, so it's irrelevant.
> It seems a no brainer that the facial recognition software should not be used on African Americans either.
Any data source which produces data that's better than a random guess should be considered when trying to solve a problem. That it offends some people's social sensibilities is a separate issue.
Look, I don't like the mere existence of the FBI's system, and I don't think the public's continued lack of concern for placing constitutional restraints on systems like it is a wise course forward, but to say it's a "no-brainer" when it might produce valuable leads is to focus on one aspect of the problem without considering the rest of it.
Just as it would also be a mistake to consider it more accurate than it is when trying to identify dark-skinned suspects. Witness accuracy is not a new problem.
It's also worth noting that the human eye is just as bad at facial recognition when the light and/or contrast is lower.
If it's better than a random guess–which it is, majorly–then it could constitute a valid data point in narrowing a search. There's nothing racist about using it to the level of accuracy it has demonstrated. If the system worked to the same degree of accuracy on Caucasians as it does on Africans nobody would be calling it racist, they'd be calling it a tool that can help build a list of potential suspects to further investigate.
To argue anything else is making it a matter of social sensibilities.
Because in Eyewitness identification, you're never truly dealing with "random" guesses. Again, my experience is on the medical side of this whole thing. I know, and understand, the rules as my company is obliged to play by them. And there is just no question that we would not be allowed to use that system on african americans. Review boards would look on it the same way you might look on someone using leeches in the middle ages.
Depends on the camera settings too. Older movies with mostly white casts messed this up a lot, by not properly calibrating for the darker skinned characters. Conversely, movies with all black casts get this right: they tune their camera/color settings to make the faces more distinguishable.
I ran into this a few weeks ago where I took a picture of a couple friends. The camera decided to pick the friend with dark skin, resulting in him looking great, and my white friend (who isn't even super pale or anything) getting washed out.
Yeah, I shoot semi-manual on my old D700 SLR. If you want high contrast ratios in dark areas of an image, everything else will be blown out - and you'll have to use a longer shutter speed - or suffer iso degradation or blur.
> If you want high contrast ratios in dark areas of an image, everything else will be blown out
Conversely could you say that in order to capture subjects with lighter skin, you need to increase lighting in the background or else the background will clip?
> he was able to photograph black actors of dramatically different skin tones in a nighttime interior scene using just everyday house lamps, thanks to a sophisticated digital camera. “I just changed the wattage of the bulb, used a dimmer, and I didn’t have to use any film lights. That kind of blew me away,” Patterson says. “The camera was able to hold both of them during the scene without any issues.”
It seems like there might have been tech limitations initially, but with today's technology you can light your backgrounds and expose for either dark or light subjects well. The big challenge is having a photo with 2 subjects in the same exposure who you want to both look good.
Edit: by the way I too have an old Nikon camera. I usually shoot in Aperture priority with some intentional underexposure, and then edit it up in post :)
If it's a machine learning system (which virtually all face recognition systems are as far as I know) it depends more than anything on the training data.
Thank you for sharing that article. However, I don't see how the algorithm is itself racist.
That is like, if I give you glasses to put on, and through those glasses, you observe racism. That doesn't mean that the glasses made racism appear. And I don't think attaching blinders so that you can't see the racism would be helpful, either.
Word2vec is a lens that observes human output. And people saw things that they didn't want to see.
This is the thing about "systemic" racism. No one thing is racist. No need for racist intent. Just a combination of systems producing an outcome unfair to a particular group.
No disease is. No algorithm is. No specific policy is. Systems can exhibit systemic racism. Other than hateful extremism coming from an individual, most forms of injustices are emergent properties.
In a hypothetical world where evolution doesn't care about morals and therefore doesn't make sure to only create differences in morally insignificant features, how would you be able to tell what is "systemic discrimination" and what is the state of genetic differences across groups?
Is the second also systemic discrimination? Can Joe the Down syndromer talk about systemic discrimination in having no chance at being the POTUS?
I'm pointing out a statistical fact from the article that black people are more hurt by this technology. I'm not arguing for racist intent, but against accepting unfairness towards one group as acceptable.
Dynamic range is of a photo is something controllable by humans building this technology. Contrast is a technical limitation and not a physical one. Lookup the articles on how early stage photography couldn't capture darker skin colors. Again, not racism, but related to incomes and advertising, but still had effects that disproportionally helped certain groups and harmed others.
My camera's eye and face detection are also pretty racist. Works well with white and asian people, not so much with black people. It must have been written by women though because it recognizes women much more than men.
I wrote a class paper about the threat to privacy/freedom from data mining and advanced algorithms in 2003.
Unfortunately the risks they highlight exist without facial recognition. It's interesting to see facial recognition as a point of debate because the battle has been largely lost.
The quantity and intimacy of personal data already captured is much more damaging. Amazon figured out a teen girl was pregnant in 2012 [0]. IMO face recognition isn't even the most invasive modern technology. Sure it helps close the loop on a few Luddites but most of us willingly handed over the keys to our lives long ago.
"Western Civilization" has operated as Imperialist and Totalitarian regimes for centuries. "Modern" democracy is the exception, not the default mode of our societies.
To be fair that is misleading. Even in the most extreme cases, Feudal Europe arguably struggled to maintain authoritarianism as it was often government and military power was decentralized. Let's be honest though, I was never talking about feudal Europe. We're talking about America, and yes, modern democracy that built the Western world. Your characterization of it as always authoritarian is plainly erroneous.
68 comments
[ 4.7 ms ] story [ 129 ms ] threadSection 4.1 of this https://www.fbi.gov/services/records-management/foipa/privac... shows that state and local authorities are not the only ones with direct access.
Edit: forgot a word
http://www.zerohedge.com/news/2017-03-27/coming-inner-city-u...
I don't feel strongly either way, I just think that there is more nuance around the issues of privacy.
I can (trivially!) choose not to use Google's services, including their DNS servers.
[0]http://www.citylab.com/crime/2014/02/yes-police-can-arrest-y...
I think technology is often a very effective way to describe what "systemic racism" is.
Racism propagates easily, and race-blind / race-agnostic / race-indifferent behavior does not combat it or counteract it.
Imagine a majority-white tech team, largely color-blind, yet still (by virtue of economic and geographic segregation) in a mostly white-centric bubble. The training set might be largely white. If some preprocessing or feature-detection is taking place on the raw images, it might be tracking features that could have higher variability among white people while overlooking other features.
If you don't know to go out of your way to check for, include tests, and give thought to mitigating imbalances caused by race (because you "see no race") then you allow this problem to propagate. In the case of computer vision, you will inherit the "racism" of every library you import from.
This reminds me of gender biased found in Word2vec[1]. It is hard to accuse the dataset of being sexist, especially when viewing it as a reflection of language in news article. Yet, people using Word2vec use it as a representative of language, not a representative of language in news, with all the baggage in today's (and past) culture. As a result, sexism can propagate, autocomplete and query suggestions might be biased. Then spending habits might be biased, etc.
1: https://mic.com/articles/124899/the-reason-this-racist-soap-...
> In some cases, according to tests with female mannequins, small women were almost three times as likely as their average male counterparts to be seriously injured or killed. A study of actual crashes by the University of Virginia’s Center for Applied Biomechanics found that women wearing seatbelts were 47 percent more likely to be seriously injured than males in similar accidents.
But it was developed by Japanese and tested only on Japanese people. The host had to wear a black wig to make it work.
Only 16% of the US population is Black, while 70% is white. If the error rates between Caucasians and Blacks were the same that would, in fact, be racist - or at the very least undemocratic.
It's like asking that everything in the world be symmetric when only 10% of the world is left handed.
Perhaps you not understand the metaphor. Developing takes time and money to produce accuracy. You're advocating greater per-capita investment for an ethnic group. That's racial discrimination. You are advocating racism.
It should come as no surprise that there's a difference. It is therefore suspect that they avoid mentioning the magnitude of the difference; it must not support their case otherwise they would highlight it.
Critical reading, logic, and reason will help protect you from this type of manipulation and inadvertent racism.
Software is subject to the same effects institutions of racism, sexism, etc. and is a good example of just how pervasive these institutions are.
Claiming racism without understanding the problem is both unfair to the people involved, and an unwise undermining of one's own credibility for the times when people really are acting in bad faith. That may be in this case, but that may not be, and I don't like guilty-until-providen-innocent no matter whom it's leveled against (even a Federal agency of which I'm not the world's biggest fan, and a database the likes of which make me think we need a constitutional amendment to protect the privacy of the citizens).
This post's comments are a great example of why technologists are fairly terrible at engineering social solutions, much the same way sociologists tend to be terrible at engineering technical solutions.
In the medical device I mentioned above, it was medically useful for caucasians and lighter-skinned Hispanics, so-so with Asians, and almost useless for Africans.
So, what–don't go to market until the technology has parity across all skin tones?
Not everything is about intent.
but I actually wrote a pretty successful 3d radiology/cardiology package. So I understand medical devices.
I just wanted to hop in here and say that there is NO WAY the FDA would give you clearance to use the device you describe on African Americans under the circumstances that your posts have postulated.
It seems a no brainer that the facial recognition software should not be used on African Americans either. At least, if it was MEDICAL software... it would DEFINITELY NOT be cleared with that kind of a failure rate. And any provider or delivery network using the facial recognition software on African Americans would be on his or her way to prison.
But the regulations on legal software may be more lackadaisical than the regulations on medical software. Getting things exactly right may not be as important. And the penalties for misuse of medical devices are, without question, far more draconian.
The FDA has approved drugs that are specifically intended to target medical issues in specific ethnicities, so I'm not sure about your assessment. That said, this product's market was outside the scope of the FDA, so it's irrelevant.
> It seems a no brainer that the facial recognition software should not be used on African Americans either.
Any data source which produces data that's better than a random guess should be considered when trying to solve a problem. That it offends some people's social sensibilities is a separate issue.
Look, I don't like the mere existence of the FBI's system, and I don't think the public's continued lack of concern for placing constitutional restraints on systems like it is a wise course forward, but to say it's a "no-brainer" when it might produce valuable leads is to focus on one aspect of the problem without considering the rest of it.
Just as it would also be a mistake to consider it more accurate than it is when trying to identify dark-skinned suspects. Witness accuracy is not a new problem.
It's also worth noting that the human eye is just as bad at facial recognition when the light and/or contrast is lower.
the system doesn't work. At least not for african americans. You can't "socialize" away the fact that it doesn't work.
Use it for the races that it works for. DON'T use it for the races it does NOT work for. That is common sense... also known as a "No Brainer".
To argue anything else is making it a matter of social sensibilities.
Because in Eyewitness identification, you're never truly dealing with "random" guesses. Again, my experience is on the medical side of this whole thing. I know, and understand, the rules as my company is obliged to play by them. And there is just no question that we would not be allowed to use that system on african americans. Review boards would look on it the same way you might look on someone using leeches in the middle ages.
Only worse... because we KNOW it doesn't work.
I ran into this a few weeks ago where I took a picture of a couple friends. The camera decided to pick the friend with dark skin, resulting in him looking great, and my white friend (who isn't even super pale or anything) getting washed out.
Conversely could you say that in order to capture subjects with lighter skin, you need to increase lighting in the background or else the background will clip?
Here's an article about the cinematography and technology choices: https://www.washingtonpost.com/entertainment/movies/12-years...
> he was able to photograph black actors of dramatically different skin tones in a nighttime interior scene using just everyday house lamps, thanks to a sophisticated digital camera. “I just changed the wattage of the bulb, used a dimmer, and I didn’t have to use any film lights. That kind of blew me away,” Patterson says. “The camera was able to hold both of them during the scene without any issues.”
It seems like there might have been tech limitations initially, but with today's technology you can light your backgrounds and expose for either dark or light subjects well. The big challenge is having a photo with 2 subjects in the same exposure who you want to both look good.
Edit: by the way I too have an old Nikon camera. I usually shoot in Aperture priority with some intentional underexposure, and then edit it up in post :)
It's just the same as with people: https://en.wikipedia.org/wiki/Cross-race_effect
That is like, if I give you glasses to put on, and through those glasses, you observe racism. That doesn't mean that the glasses made racism appear. And I don't think attaching blinders so that you can't see the racism would be helpful, either.
Word2vec is a lens that observes human output. And people saw things that they didn't want to see.
Is the second also systemic discrimination? Can Joe the Down syndromer talk about systemic discrimination in having no chance at being the POTUS?
It's physics, not racism. Darker colors absorb more light and reflect less. In a photo the contrast is difficult to bring out with darker subjects.
I know this because I had a black dog and all her pictures came out like crap until she got old and turned salt and pepper.
Some people just have to see racism everywhere. That's too bad because it takes meaning away from real racism.
Dynamic range is of a photo is something controllable by humans building this technology. Contrast is a technical limitation and not a physical one. Lookup the articles on how early stage photography couldn't capture darker skin colors. Again, not racism, but related to incomes and advertising, but still had effects that disproportionally helped certain groups and harmed others.
Unfortunately the risks they highlight exist without facial recognition. It's interesting to see facial recognition as a point of debate because the battle has been largely lost.
The quantity and intimacy of personal data already captured is much more damaging. Amazon figured out a teen girl was pregnant in 2012 [0]. IMO face recognition isn't even the most invasive modern technology. Sure it helps close the loop on a few Luddites but most of us willingly handed over the keys to our lives long ago.
[0] https://www.forbes.com/sites/kashmirhill/2012/02/16/how-targ...
Where this--and other troubling revelations from both the Surveillance State and the Deep State--leads us to is authoritarianism.
Once we're there, nothing will matter anymore. It will be the end of Western Civilization as we know it.