ML systems are biased when data is biased. This face upsampling system makes everyone look white because the network was pretrained on FlickFaceHQ, which mainly contains white people pics. Train the exact same system on a dataset from Senegal, and everyone will look African.
I think that the Twitter thread is less about bias from datasets used to train ML apps and is more in reference to the application of biased ML apps that are supposed to identify criminals (which have been historically bad at identifying non-white individuals apart from each other).
I feel as though the general message that the tweet threads author is trying to convey isn’t very clear. I wish they would clarify what they mean.
> I’m sick of this framing. Tired of it. Many people have tried to explain, many scholars. Listen to us. You can’t just reduce harms caused by ML to dataset bias.
Is pointing out the technical reason for a malfunction "reducing harms caused to dataset bias"?
Actually, in this specific example, what form would the harm take place? Would the ML system upscale a surveillance photo of a black person into a white person, causing the white person to be wrongfully arrested? Does facial recognition working better on white faces mean white protesters are at greater risk of surveillance?
> Train the exact same system on a dataset from Senegal, and everyone will look African.
In other words it would still be broken. That's not much of a defense.
I don't think there is dataset that would make the PULSE algorithm that produced that image deal well with skin color, the hill climbing it does is going to find it easier to take an averagish face and tweak the lighting.
If you could take a Bayesian perspective toward the super-resolution problem, things will make sense: given a low-res image, it corresponds to a distribution of corresponding high-res images. Which one is more likely? It depends on the prior and the likelihood. The right figure is a possible outcome, however, if we have strong prior toward the possibility of well-known people, we would be biased toward those people. It's not wrong, it is just not comprehensive.
I think you're being a little too charitable to the AI in this case.
Even without "famous person bias", it's fair to say that the skin tone of the resolved image is slightly but measurably lighter than the skin tone of the blurred original (I'm curious what the blurred version of the resolved image looks like, btw).
Occam's razor says the model simply wasn't trained with enough ethnically diverse images, for whatever reason.
Anyone knows why "fastMRI" [0] would not suffer from problems like this, especially if there is something on the images that has not been on any images it was trained on (e.g. foreign matter)? Enhancing faces going wrong is one thing, getting medical images wrong a whole another.
[0] "fastMRI is a collaborative research project between Facebook AI Research (FAIR) and NYU Langone Health. The aim is to investigate the use of AI to make MRI scans up to 10 times faster. By producing accurate images from under-sampled data, AI image reconstruction has the potential to improve the patient’s experience and to make MRIs accessible for more people.", see https://fastmri.org
There is almost always some way to do more work and get more accuracy (e.g. getting MRIs from 2 different places, having 2 groups analyze them, etc), but we don't generally do all these things because there are costs involved.
Conversely reducing costs is often valuable, even if accuracy degrades a bit, depending on how steep the trade off is
I'm not sure why shitty AI always gets used as evidence against the field. We typically dont see shitty software and say, well all software must be shitty then.
This is a nightmare if it ever gets used in trials. "We took this low-res picture from the burglary, used our high-tech Artificial Intelligence to enhance it, and now it looks just like you!"
17 comments
[ 2.6 ms ] story [ 53.2 ms ] threadI feel as though the general message that the tweet threads author is trying to convey isn’t very clear. I wish they would clarify what they mean.
Is pointing out the technical reason for a malfunction "reducing harms caused to dataset bias"?
Actually, in this specific example, what form would the harm take place? Would the ML system upscale a surveillance photo of a black person into a white person, causing the white person to be wrongfully arrested? Does facial recognition working better on white faces mean white protesters are at greater risk of surveillance?
I'm also sick of this framing.
This is a big issue in Science, actually. One guy speaks and thousands follow. We are still in the dark ages.
In a tweet he identified the issue with the model and how to fix it. Then he got mobbed for it. [0]. But he was, and still is, right.
[0] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
In other words it would still be broken. That's not much of a defense.
I don't think there is dataset that would make the PULSE algorithm that produced that image deal well with skin color, the hill climbing it does is going to find it easier to take an averagish face and tweak the lighting.
Even without "famous person bias", it's fair to say that the skin tone of the resolved image is slightly but measurably lighter than the skin tone of the blurred original (I'm curious what the blurred version of the resolved image looks like, btw).
Occam's razor says the model simply wasn't trained with enough ethnically diverse images, for whatever reason.
[0] "fastMRI is a collaborative research project between Facebook AI Research (FAIR) and NYU Langone Health. The aim is to investigate the use of AI to make MRI scans up to 10 times faster. By producing accurate images from under-sampled data, AI image reconstruction has the potential to improve the patient’s experience and to make MRIs accessible for more people.", see https://fastmri.org
Conversely reducing costs is often valuable, even if accuracy degrades a bit, depending on how steep the trade off is