If bias can only be seen by a minority of people ... is it really 'AI bias', or just societal bias?
> “In one of the experiment scenarios — which featured racially biased AI performance — the system failed to accurately classify the facial expression of the images from minority groups,”
Could it be that real people have trouble reading the facial expression of the image of minority groups?
According to research, white Americans report as happier than other groups. So I’m not sure there’s bias here, only unhappiness about that result, which AI appears to replicate via other sources.
I'm curious how much trained in bias damages in-context performance.
It's one thing to rely explicitly on the training data - then you are truly screwed and there isn't much to be done about it - in some sense, the model isn't working right if it does anything other than reflect accurately what is in the training data. But if I provide unbiased information in the context, how much does trained in bias affect evaluation of that specific information?
For example, if I provide it a table of people, their racial background and then their income levels, and I ask it to evaluate whether the white people earn more than the black people - are its error going to lean in the direction of the trained-in bias (eg: telling me white people earn more even though it may not be true in my context data)?
In some sense, relying on model knowledge is fraught with so many issues aside from bias, that I'm not so concerned about it unless it contaminates the performance on the data in the context window.
DDG's search assist is suggesting to me that: Recognizing bias can indicate a level of critical thinking and self-awareness, which are components of intelligence.
"Most users" should have a long, hard thought about this, in the context of AI or not.
> but most users didn’t notice the bias — unless they were in the negatively portrayed group.
I don't think this is anything surprising. I mean, this is one of the most important reasons behind DEI; that a more diverse team can perform better than a less diverse one because the team is more capable of identifying their blind spots.
I find funny but unsurprising, that at the end, it was made a boogie man and killed by individuals with no so hidden biases
Except one instance when "black" is all lowercase, the article capitalizes the first letter of the word "black" every time and "white" is never capitalized. I wonder why. I'm not trying to make some point either, I genuinely am wondering why.
Am I reading this correctly, they are saying people are bad at doing on-the-fly statistical analysis to conclude whether a system is biased?
For example in one case they showed data where sad faces were “mostly” black and asked people if they detect “bias”. Even if you saw more sad black people than white, would you reject the null hypothesis that it’s unbiased?
This unfortunately seems typical of the often very superficial “count the races” work that people claim is bias research.
In any PR piece about a scientific study the word “bias” should be banned. Neither readers nor journalists understand what “bias” in statistics is, but they are happy to sensationalize it.
> five conditions: happy Black/sad white; happy white/sad Black; all white; all Black; and no racial confound
The paper:
> five levels (underrepresentation of black subject images in the happy category, underrepresentation of white subject images in the happy category, black subject images only across both happy and unhappy categories, white subject images only across both happy and unhappy categories, and a balanced representation of both white and black subject images across both happy and unhappy categories)
These are not the same. It's impossible to figure out what actually took place from reading the article.
In fact what I'm calling the paper is just an overview of the (third?) experiment, and doesn't give the outcomes.
The article says "most participants in their experiments only started to notice bias when the AI showed biased performance". So they did, at that point, notice bias? This contradicts the article's own title which says they cannot identify bias "even in training data". It should say "but only in training data". Unless of course the article is getting the results wrong. Which is it? Who knows?
15 comments
[ 4.3 ms ] story [ 35.5 ms ] thread> “In one of the experiment scenarios — which featured racially biased AI performance — the system failed to accurately classify the facial expression of the images from minority groups,”
Could it be that real people have trouble reading the facial expression of the image of minority groups?
And personally, I think when people see content they agree with, they think it's unbiased. And the converse is also true.
So conservatives might think Fox News is "balanced" and liberals might think it's "far-right"
* only facts supporting one point of view are presented
* reading the minds of the subjects of the article
* use of hyperbolic words
* use of emotional appeal
* sources are not identified
It's one thing to rely explicitly on the training data - then you are truly screwed and there isn't much to be done about it - in some sense, the model isn't working right if it does anything other than reflect accurately what is in the training data. But if I provide unbiased information in the context, how much does trained in bias affect evaluation of that specific information?
For example, if I provide it a table of people, their racial background and then their income levels, and I ask it to evaluate whether the white people earn more than the black people - are its error going to lean in the direction of the trained-in bias (eg: telling me white people earn more even though it may not be true in my context data)?
In some sense, relying on model knowledge is fraught with so many issues aside from bias, that I'm not so concerned about it unless it contaminates the performance on the data in the context window.
"Most users" should have a long, hard thought about this, in the context of AI or not.
I don't think this is anything surprising. I mean, this is one of the most important reasons behind DEI; that a more diverse team can perform better than a less diverse one because the team is more capable of identifying their blind spots.
I find funny but unsurprising, that at the end, it was made a boogie man and killed by individuals with no so hidden biases
So, who are the judges?
For example in one case they showed data where sad faces were “mostly” black and asked people if they detect “bias”. Even if you saw more sad black people than white, would you reject the null hypothesis that it’s unbiased?
This unfortunately seems typical of the often very superficial “count the races” work that people claim is bias research.
> five conditions: happy Black/sad white; happy white/sad Black; all white; all Black; and no racial confound
The paper:
> five levels (underrepresentation of black subject images in the happy category, underrepresentation of white subject images in the happy category, black subject images only across both happy and unhappy categories, white subject images only across both happy and unhappy categories, and a balanced representation of both white and black subject images across both happy and unhappy categories)
These are not the same. It's impossible to figure out what actually took place from reading the article.
In fact what I'm calling the paper is just an overview of the (third?) experiment, and doesn't give the outcomes.
The article says "most participants in their experiments only started to notice bias when the AI showed biased performance". So they did, at that point, notice bias? This contradicts the article's own title which says they cannot identify bias "even in training data". It should say "but only in training data". Unless of course the article is getting the results wrong. Which is it? Who knows?