The video lecture calls for validation of ML results with regard to “appropriate social behaviour”. Is this actually a call to hide results that are politically inconvenient? A concrete example would help here.
A bias is simply a non uniform distribution among the values of a feature of what you're trying to predict. Since the data used by Google in training AI comes from users, it's likely to contain all sorts of human and social bias adding noise to what they're trying to predict.
Indeed, I know what a bias is and what the common cognitive biases are, though they always bear repeating. I’m referring to a specific statement at the end of the video lecture, where the viewer is encouraged to validate ML results against “socially appropriate behaviour”. This is somewhat ambiguous, and can be interpreted as meaning that the viewer should suppress any result that may appear taboo. I’m wondering if this is the intention behind the statement.
If so, this is not to be encouraged. It would be the equivalent of the sciences rejecting results when they don’t conform to current theory. Go to extra lengths to validate the results, sure, but don’t throw them away out of hand.
It's becoming increasingly clear that in the current environment, algorithms trained on real-world data that produce politically inconvenient results will be decried as X-ist, and either disregarded or fiddled with until the results they produce are inoffensive.
I think it’s important to bear in mind that most machine learning models still aren’t very good, and the main problem you’ll encounter with them isn’t bias, it’s whether or not they’re producing gibberish. In this thread we seem to have assumed that ML is generally trustworthy, and I doubt very much that it is.
What we may be debating is whether gibberish is more or less morally good depending on whether it includes race and gender.
-against our own biases- we are evaluating and validating models all the time. There are many, many models with acceptable accuracy that we rely on every day.
I think this is a myopic view of the current state of ML.
This isn't about "appearing taboo", it's remembering that data is always the product of the environment that created it. If you are using e.g. housing data, naively applying the data directly bakes in, for example, the decades of redlining, blockbusting, restrictive covenants, etc. Many cities are still segregated today, decades after redlining/etc ended.
Lets say you are writing ML related to loan applications or deciding if someone gets parole. Do you want to "accurately" look at the historical data? Or should the decision also include the context of that data with an eye towards making a decision that is more socially appropriate than perpetuating the racism of the past?
I take your point about misleading historical data, but I’m not sure that was the sentiment behind the statement. I took it more to mean “if this feels transgressive to you, don’t publish it!”. Your explanation is a valid observation of a phenomenon, but the “socially appropriate” part feels shoehorned in.
There was a time when tech companies were driven by data, inspired by the scientific method. There was a time when the scientific method was about uncovering the 'what is' truth, as opposed to 'what ought be' decrees, which are by definition political. Today it seems that we are more than happy to interject arbitrary political biases in the equation, in the name of elusive fairness. Food for thought, Harrison Bergeron [0]. Teaser:
> THE YEAR WAS 2081, and everybody was finally equal. They weren't only equal before God and the law. They were equal every which way. Nobody was smarter than anybody else. Nobody was better looking than anybody else. Nobody was stronger or quicker than anybody else. All this equality was due to the 211th, 212th, and 213th Amendments to the Constitution, and to the unceasing vigilance of agents of the United States Handicapper General.
Residential segregation is a good example of the difficulty of quantifying the causality of fairness. On one hand, residential segregation is a human universal. From the Papuan jungle to the London asphalt jungle, humans tend to stick with their own kin, for some definition of own kin. People have built simulators demonstrating that even a weak preference for own kin leads to significant residential segregation [1]. On the other hand, there is a history of forced segregation, well documented in many places [2][3] etc.
Furthermore there are more than one forms of segregation, read literally as 'separation from others'. There is the well studied economic and racial residential segregation, the lack of easy access to more economically desirable living quarters. On the flip side, USA is already a very atomised and isolating society. Some would point to the addiction and homelesness crises as direct consequences of social isolation [4]. Living isolated from one's own kin is yet another factor increasing social isolation, and a different kind of segregation.
At what point does enforced fairness morph from an attempt to fix the past ills into the tyranny of today, steering people toward actions that are contrary to their natural preferences and well being? Or is there no such point and we're willing to bludgeon our way into an all out harrisonbergeronian future?
Is it really imperative to treat reality as if it was uniformly distributed? Why would you want to cast a non-linear environment into a linear shape? Are we trying to get rid of non-uniformity (which is realistically impossible, let's be honest) or are we just correcting towards the falsehoods of current thinking?
Moreso, it's a reminder that "training an ml model" isn't a controlled experiment. It's a process for constructing a tool.
You aren't hiding anything, training wasn't (and isn't ever really) rigorous. You're more than likely discovering issues in your data set. It's like polling. You don't cover up politically inconvenient polls, but there's often a level of post processing you need to do to account for biases in your sample.
Same with ml, except often times the first data set, or three, are useless because no one has ever analyzed them before and doesn't know how to post process correctly.
Unequal outcome != bias. Most SJWs don‘t seem to understand this. Of course it can be an indicator that there may be bias, but it is FAR from a proof thereof.
I can easily write you a completely unbiased “model” predicting who comitted crimes:
The problem is that people view pure statistical based crime probability as "unbiased" because... math. It's not! Most bias is deeply hidden, and our training data comes directly or indirectly from some form of human subjectivity.
The more autonomy plays a role in our lives, the more aware we need to be of how internal bias from human source has affected things like predictions.
And please, there is nothing more flippant and off-putting than railing against "SJWs," this is a topic that demands thoughtfulness.
> The problem is that people view pure statistical based crime probability as "unbiased" because... math. It's not!
Can you explain what you mean? “Biased” usually means not using just statistics but also extra-statistical personal convictions. What does “unbiased” mean for you?
Sure, which is why the fair machine learning literature has developed more nuanced (and IMO better) definitions of fair predictors.
For example, suppose you have some sensitive attribute A (say race) and want to know some outcome Y. A predictor \hat Y satisfies equalized odds if \hat Y and A are conditionally independent given Y. In binary classification, this means that the classifier has identical false positive rates and false negative rates across races. So e.g. your false negative rate for white people must be the same as your false negative rate for black people. Note that a perfectly accurate classifier satisfies equalized odds -- roughly, its a way of guaranteeing that errors are not concentrated in some group.
I think it's unfair to characterize fair ML research as the work of "SJWs". There's a lot of smart, technical work done by smart, technical people thinking hard about these problems.
Oh that’s interesting! So you can have your “creditworthiness AI” \hat Y) predict that disproportionately more black (A) people are not creditworthy, as long as that’s (roughly) in line with the actual creditworthiness (Y) as determined by the “supervisor”.
That's also a good point -- evaluating fairness against labelled data breaks down if the labels themselves are biased in some way. IMO (and I am not familiar with the entire fair ML literature), we still lack technical tools to address biased data well.
I didn’t mean that they’re biased, just that being “black” correlates with “being poor”, which is not biased in any way but might still appear so based on superficial view(well, the data/AI isn’t biased, though the society might be but that’s not the bank’s problem...)
That correlation is a "bias", or maybe better an "unfairness".
Such a system would end up requiring black people have higher credit scores than otherwise equal white counterparts, in order to be granted whatever thing it is. That's, as modern feminist/race theory might define it, an implicit bias. Nowhere is anyone explicitly stating "black implies bad", but the way the model is built creates unequal representation.
No no, what I mean, and is AFAIK visible in the data, is that the poor/low credit people are disproportionately black. So even if the AI/statistics ignore race (as they should), the result will be that blacks have lower credit scores (on average, because they poorer).
That part is to be expected, the important thing is that you don't then recorrelate, either explicity or implicitly, race with the lower credit scores, that can essentially double count race (via income and then via the second check) and that's what you want to avoid.
Ok yeah then we agree. You definitely shouldn’t feed “irrelevant” features to the AI (i.e. those you don’t want it to discriminate/regress upon) otherwise it will overfit, in the best case to the noise, in the worst to the bias.
Is this for anyone or is it just posturing? I mean anyone doing ML knows that if you train an algorithm on a biased dataset that it will, surprise, learn to be biased. It's not at all inherent or restricted to unfair treatment of any group. It's a more general problem. By focusing on "fairness" you are basically overfitting the larger problem of not selecting good datasets or not having good data.
Huh? Humans make the conscious decision to override instinctual and observational impulses all the time in response to empathy and emotion. It is a huge foundational factor in society. Why would we NOT want autonomous agents to have the same capacity? There's a reason things like The Trolley Problem come up in AI courses: our agents have to play reasonably in a human world.
I find the classic Trolley Problem interesting mostly because it exposes the extremely limited capacity that humans have to act rationally. I would expect a trivial ML/AI implementation to "correctly" solve the Trolley Problem and I would not want to consciously program our future AI systems to suffer from the same limitations that cause humans to struggle with the Trolley Problem.
By programming them this way (avoiding programming in human frailties for the purpose of better "fitting in"), I think we can more quickly expose and correct certain areas where us bags of mostly salt water act suboptimally.
The whole point of the trolley problem is there is no correct solution. It's a negative sum game. It's an ethical problem that has no solution. There is also no guarantee that a machine would kill one in lieu of five. Why? Training data.
The bigger point I'm making is people think AI acts in some form of posthuman reason. At this time no such thing exists: decisions are still bound to human bias, and we should at least consider optimizing with regard to the rules of a modern society.
My intentionally off-hand use of "correctly" was meant to indicate a belief that killing one innocent is strictly better than killing five and so if a machine is faced with choosing exactly one of those, the problem boils down to a comparison and a conditional jump, IMO.
Humans are constrained by the knowledge that they would dread for the rest of their life the knowledge that they optionally and intentionally took an overt action which resulted in the death of an innocent person (to save five, but they would know they killed one when they could have done nothing and had "clean hands"). Machines don't need to suffer from this.
I would hope that if we program safety-controlling systems that we wouldn't replace the compare and conditional-jump with "halt and catch fire".
No offense here, but none of that has anything to do with the dilemma. Human response would be biased in myriad ways. Agent response would be biased in myriad way. There is no correct solution. The lasting impact on a person (and lack thereof in an agent) has no bearing on the decision.
Really? Why is the original/classic trolley problem in any way interesting if the future weight of the decision on the human has no bearing? (That's a serious question.)
In the problem statement, an actor has control over whether this 1 person or those 5 people die. If they choose red, then 1. If they choose blue, then 5. (Failing to make a choice is choosing blue.) If after playing the scenario, they knew someone would use a Men In Black pen on them to completely forget what happened, is there a rational argument against choosing red?
(I realize this might read like trolling, but I genuinely don't find the original problem a difficult one if you eliminate the human guilt aspect.)
If you choose to switch the track to kill 1 innocent to save 5, you open the door to things like a society where 1 innocent healthy person is killed to harvest the organs and save 5 innocents. The calculation is the same.
Utilitarianism is very dangerous. Our current societal system isn't really based on utilitarianism as I see it, and I don't think we want to live in a world where a pure version of that school of thought is allowed to have any power.
Actively choosing an action to deliberately kill someone else could be weighed negatively by the machine, it would depend on the programmer.
I don't think you can apply the word "rational" here, if you don't take a position of utilitarianism, not deliberately killing the person could be axiomatically rational. It's not just about guilt.
So let's suppose you were riding in your self-driving car and the AI in charge of it encountered a trolley problem:
The brakes had suddenly failed approaching an intersection and the car had the choice of either running into a group of five pedestrians crossing the street or slamming into the concrete wall to your right, killing only you. Which is the correct answer?
Now suppose it was not you in the car but your wife and two daughters. Would you want your car, the car you bought and paid for, to sacrifice your family to save the lives of five strangers? What if it were only four strangers? Or ten?
There is no right answer. But as humans we're biased to favour, if not our own lives, the lives of our closest family members over strangers. I, personally, would never buy a car if I knew it would make the choice to sacrifice my family to save strangers.
Yes, I agree humans are selfish. Given the enormous number of situations where personal or in-group selfishness would confer a survival and propagation advantage, it would be surprising if we didn't have it. I hope we don't take explicit pains and effort to encode this into ML/AI systems.
Concretely, the car should take the course of action to minimize the global harm based on the knowledge it has. If that's killing me in the event of a brake failure, so be it. The spectacularly unlikely probability of that is outweighed by the million more times when it and others like it will make a less selfish decision than today's human drivers.
> I, personally, would never buy a car if I knew it would make the choice to sacrifice my family to save strangers.
The point I find interesting here is "this consumer preference, if common, confers a propagation advantage to systems which behave selfishly".
But should we be deprived of our instinct? Should we not get to apply our own personal ethics to our lives in the future, for the greater good? Its not a welcome encroachment on free will, even if it may be net positive from an objective view, removing the sense of agencey from humanity may end up hurting us worse.
Humans make the conscious decision to override instinctual and observational impulses all the time in response to empathy and emotion.
This statement is misleading.
"Instinctual and observational impulses" is poorly defined, but the low latency cognitive systems I think you are describing (such as object categorization) are inputs to higher executive functioning systems which determine behavior.
The fact of and ability to "override" generalizes too much about how these low latency systems are prioritized a-priori. In other words, some of them seem to be able to be able to be re-prioritized fairly easily (positive/negative response to specific olfactory stimuli), while others approach impossible without motor system intervention (Not reacting to severe pain stimuli).
It's categorically misleading to separate these from all other functions of Perceive > Process > Respond as all decisions are all built on the same reinforcement learning foundations.
Not sure where the strawman is. I'm trying to describe that the cognitive process you think exists, doesn't actually exist the way you seem to think it does.
The “American Inventor” result is easily explained by being a substring of “African American Inventor”. No conspiracy necessary, it’s just a substring hit.
It’s a trope that’s supposed to show that google searches are biased in favour of black people. If you search for American inventor you get lots of image hits for black inventors. The real reason for this is that American inventor is a substring of African American inventor, so google shows results containing the superstring.
But it does operate this way, of course. It is just the same effect that causes "earth society" to mostly show results about the superstring "flat earth society"
No, it is actually that flat Earth just got really popular in the last few years. There is a lot of money and promotion force behind it. It seems like at least one group is trying hard to make their opponents view points be seen as synonymous with believing the Earth is flat... Pretty good disinfo strategy!
Because there aren’t that many pages about it. Also a white American inventor would probably just be referred to as an inventor, without reference to race or nationality. A black inventor would be seen as out of the ordinary, and so be referred to as an African American inventor, stacking the deck in favour of those results.
Strangely it's a "substring hit" that only works for scientists and inventors. The substrings stop matching when you look for "American economists" or "American philosophers".
Works for American doctor, gives a decent number of black results for American soldier. Exactly what is it going to take for your conspiracy to be disproved? Does Eric Schmidt need to float down on a cloud and walk you through Google’s algorithm?
Some explanation from Product Expert (Google Search)
> Regarding the search for [white american inventors], the results would reflect the way the images are labelled. Check the image descriptions and you will find that the image results all contain some of the words you searched for. Maybe the webmasters who label images of inventors have no interest in the colour of the inventor's skin!
Not even a paragraph in and I see problems with this. Seems somebody bias has already gotten in their own way.
This is the first step - redefine the meanings of words to fit your agenda.
I could be wrong about the intent. But when you can redefine fairness the you can make it fair to take from one person and give to another. Or deny one person and Grant access to another based on things that don't matter.
Also. The attack on human bias is totally silly. It's a key trait that had made us and other species successful. Without these biases we might pick the wrong fruit eat it and die. We might try to cuddel into a cave with bears in the winter.
Our entire existence and reasoning skills is based off bases.
Also. Never in my life have a seen ripe or unripe bannans and described them by their color. Is this a education problem?
But even if I had there is nothing wrong with pointing out when something is not atypical. Besides if I plan on eating bananas in a few days I might want the "green" ones. Or if I was going to make a yummy smoothie the "brown" ones are the ones I want.
Also bananas are a bad analogy for what they were getting at. Every banana has the potential to be green, then yellow, then brown. So none of these states are out of the ordinary.
I don't need or want to be unconsciously retrained by ML. I think we are going to need laws against big cooperations using their power and tech to change socal constructs. It's not their place and we can never be sure they are doing it for good or even the outcome will be good for human kind. It's also a power that can be abused. We already have internal documents comming from Google suggesting swaying public opinion on on political canadates. And this type of research is exactly where you would start.
What? This isn't about redefining fairness, but defining it in the first place. Seriously, try to come up with a rigorous technical definition of what a fair decision process means. It's not easy. But we need such a definition (or definitions).
It mattered less in the past because humans made all the decisions so we could fall back on fuzzy qualitative notions. We can't do that when algorithms make decisions.
> I think we are going to need laws against big cooperations using their power and tech to change socal constructs.
Maintaining such constructs is as much a use of power as changing them is. It’s completely impossible to outlaw a corporation’s or any agent’s influence on society. ‘Society’ as we think of it (the nation, a community, bureaucracy, you name it) itself is a massive construct with tons of subjectivity attached.
> I don't need or want to be unconsciously retrained by ML.
If it’s a thing in your world, which it probably already is, it will have an influence on you. This type of research is how we make that visible. IMHO the social sciences are a good thing.
> It’s completely impossible to outlaw a corporation’s or any agent’s influence on society.
Honestly. We would have to ban all marketing and mass media. The only other solution I can think of is to subject all relevant industries to some democratic process, and I don't think anybody really wants that.
I am down for a ban on all marketing and mass media :p
But really, think there is a difference in trying to get me to buy your microwave and a small group of people attempting to retrain the masses on social issues -- according to their views -- "for the good of everybody".
It's really hard for me to understand how anybody can think it would be good for any small group, no matter how much good they think they are doing, or how pure their intent is to have the power to change the masses, little alone be allowed to use such power.
Things are complicated one wrong step and these people can break society.
Ultimately I agree with you, I don't think this is a good thing. But I just don't see a way that we as a society can realistically address the problem of a few powerful entities having a disproportionate influence on mass culture, because that has already been happening long before google was around. You're basically wanting to eliminate large-scale propaganda, social engineering, and eliminate the culture industry. That's not going to happen.
Ugly short men in a corporate environments have to swim against the tide because everyone instinctively thinks less of them. Tall handsome men get automatic adoration. Prejudice is irrational and harmful. Bill Gates and Bezos are too short to be CEOs. The world is filled with irrational biases that do more harm than good.
Hah! So their intro video for this idea states that we are basically biased because we assume bananas to be yellow...
I wonder how their code is going to read...
If white, then reduce privileges
If black, then give reparation as structRep
If binary gender, then increase non-binary sexual imagery; equalize all norms
If young female wants to be a mother, then show videos of females "succeeding professionally"
Again, at the top of HN with the most political article, but commenters like myself are censored from being able to even give our take on why they're extremely misleading.
How is this not exactly like Brave New World?
I think google should take a bit of its own advice here. Google "american scientists", "american mathematicians" and tell me how these are not hugely biased results. I work in NLP and there is no way you get this without forcing your data/algorithms to return these types of results.
This isn't a normative statement, just descriptive. Whether or not google should bias its results is a completely different discussion.
Can you explain how this is collaborative filtering as opposed to a classic IR ranking problem? CF would suggest they are somehow getting user ratings of these scientists, but either way its going to boil down to a similarity metric basically. So I guess for me, I can't imagine how user data is creating these rankings and I'm pretty confident using IR techniques on the datasets they have would not return these either, ergo, they are likely tweaking the factors themselves to return results that are "less biased" i.e. less representative of the underlying distribution and more normally distributed aka politically correct.
But If you have a better theory of how the 10 of the first 20 "american scientists" are black and 5 are women, I'd be interested to hear it.
Result #6 is the list of African-American inventors and scientists on Wikipedia. Unless Baidu has the same ideological biases as Google (would be strange), the most likely explanation is that it's driven by n-gram frequencies.
Yes, precisely that's what I would expect from an NLP system b/c it will find "African American" and, I would expect "Chinese American", etc. in documents more frequently than for a plain "American", much like what this article mentions with Banana and no one ever mentioning yellow. Still, the algorithm would have to be pretty approach would have to be pretty naive not recognize that "X-American" is a subset of "American". It would be like not recognizing that a query for "anonymous function" is something different than a query for "function".
I'm still interested in a possible technique which could lead to this type of bias without it being explicit (or requiring google to have an extremely naive approach).
> Yes, precisely that's what I would expect from an NLP system
> I'm still interested in a possible technique which could lead to this type of bias without it being explicit (or requiring google to have an extremely naive approach).
I don't understand. If it's what you expect, then what's left to explain?
I don't see why you can't just accept that that naive approach is their approach? Those two words almost always occur together as part of "-American scientist." This happens to work very well in general for search engines. I don't think Google or DuckDuckGo is hoping their image page for American Scientist just returns African Americans and are therefore subtly changing their algorithm to that end.
> I don't see why you can't just accept that that naive approach is their approach?
I very strongly doubt their approach is based on substring search. They're obviously using a knowledge graph. And if you try a search for "American economists" or "American philosophers" the results look much more expected, either the "American" in this case is not a substring of "African-american" or they simply thought that economy and philosophy aren't as worth of an equality boost as STEM disciplines.
Have you considered that there may not be as many African American economists as there are scientists, doctors and inventors and that is the reason the search behaves differently? Do you have any substantive basis for your claim that google is racially biasing their results?
It's strange that you say there is no bias in the results, because in another comment you have even proposed a mechanism to explain it:
"A black inventor would be seen as out of the ordinary, and so be referred to as an African American inventor, stacking the deck in favour of those results."
So the results look biased. Now, it might just be by chance; however all this talk from Google about fairness and equal representation makes me a bit suspicious. Not certain, just suspicious. I would have preferred a company that just said "look, we're engineers, this is the data, these are the algorithms, we don't care about the outputs, deal with it". But it looks like that stopped happening at Google (I'm not saying in this specific case, I'm saying in general) a long time ago.
So apparently Google, while understanding the search query, still orders the results by the words used to express it- and in the second case clearly privileges African-Americans because of the "American" substring. My bad.
As someone else has pointed out, it's because "American scientists" is a substring of "African American scientists." And it's easy to check that's the case. Search for "African American scientists" and those first results are the same. Makes sense: exact match vs inferred connection. Then go search for "English scientists", "French scientists", "German scientists."
I'll point out that broad categories of fairness have nothing to do with AI/ML and everything to do with institutions ensuring fair outcomes.
For instance, we would be appalled if a court found that the conviction rate for people like the defendant was extremely low, so the trial was going to start with the presumption of 80% guilt.
What's the problem? It's not one fixed by better modelling or a better bias percentage. The problem is that fairness in many contexts is about processes and institutions. We expect courts to follow principles like due process and equal protection under the law.
Companies like Google can't jump right to social science and social engineering without thinking about fairness and justice in these terms.
Can a user confirm that the data about them is accurate? If not, what can they do about it?
If a business has its account algorithmically cancelled, what is the appeals process? How is it fair?
When companies like Google start getting into social science, the problem is extremely complicated fast. For example, it starts looking like a judicial system in parts, but one that isn't established and maintained by a legitimate government.
I was talking about how to even engage with Google about untrue or unhelpful data.
If Google wants to be fair, it's not just a matter of data, demographics, and statistics. It needs to think about what philosophers like Kant and Rawls would say about treating even outliers and exceptions equitably.
Which begs the question: how do you identify unfair data? If the data is not false or inaccurate, you shouldn't tamper with it.
On the other hand, if you already know at the outset the result that you would like your algorithm to produce, then why bother with machine learning at all? Just hard-code your output.
I read HN with showdead on. There's some pretty unjustifiable censorship going on in this thread.
I'll just call their attention to their own rule on this: "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."
The weak interpretation of these dead comments is that they're just racists who hate dark-skinned people. There are much stronger interpretations and those should be discussed.
Especially in the context of Google, where it's been demonstrated over and over through video leaks, court filings and the Damore firing, that a single and very heavy-handed ideology has power and will not allow discussion. From a centrist position is seems obvious Google's internal thought process will be itself heavily biased, and the possibility of certain factual hypothesis being true about the physical world (not normative statements on what to do) will be denied for ideological reasons. It's like a group of fundamentalist Christian scientists studying geology. This needs to be acknowledged in any discussion of Google, ML, and 'bias'.
Exactly. And I've spoken with several people who have been emailing the admins about this - and they just continue censoring without providing reasonable explanations (incorrectly citing abuse). HN has become extremely political - and how many aren't posting anymore because they were banned? Probably a lot! I've met at least 3 people who have reacted like I said, "do you watch Fox news," when I had actually asked, "do you visit HN?" There's a leftist article like this one at #1 nearly daily.
So why isn't there anything about teaching fairness in statistics?
To sport the classic over simplification, ML is just glorified curve fitting. But in reality, it's not like we created a special new species with intelligence that we have to careful teach not to be racist. _We_ just have to be careful not to be racist with it.
So what's happening lately is that tech giants have gotten into the business of dictating how the world should be. No one was consulted about it, it was not put to a vote, it simply began, and over the last few years it's increasing in amplitude.
The problem with them using words like 'fairness' is anyone who disagrees with what they're doing is immediately put on the defensive and labelled as 'unfair'. This is far from the truth though, as what society today is actually doing, is trying to figure out exactly what 'fair' should mean. Google et al just took it upon themselves to start pushing their particular brand of fairness onto society and acting as if it's the most natural widely-accepted common sense.
Google's idea of fairness is to build ML solutions that don't accurately predict the world, but rather inject Google biases into the predictions in order to bias some real-world process, and ultimately change the distribution of some factors in the real world according Google's image of how things should be. Not enough <demographic> represented in the workforce? Bias the candidate selection model to equalize it. Some <demographic> is over-represented in crime statistics? Bias jail sentencing model to be more lenient on them and hope it equalizes their representation in stats over time.
This is not inherently a bad mechanism, but it's an extremely powerful mechanism with far-reaching consequences, and it is currently controlled by a small number of people at the top of the US tech industry, with no checks or oversight. I worry that there's a strong possibility that the long term effects of messing around with these levers have not been considered. We have done serious unintentional damage with far smaller levers like the Australian Rabbit Plague.
I don't know what 'fairness' should actually mean in modern society, but we definitely need to be aware that the US tech industry is hijacking the term to fit their agenda, and need to be checked. And the exact nature of the biases they're injecting should definitely be made transparent and put to public debate.
Your viewpoint is not correct. Many class of decisions we make needs to be independent of race, color, religion and gender. This is not Google’s idea, it’s US Constitution + laws in many states. As ML is increasingly used for making decisions such as whom to give loans or who do you hire, many models may predispose a human based on exactly these attributes violating laws.
Well, yes. That's exactly why the debate around 'fairness' has begun.
ML will easily infer gender/color/religion from any other features about an individual. It should not be too surprising that those features play a big role in identity. Consider the recent news about Amazon throwing out a model which kept learning to discriminate on gender no matter how much they tried to blacklist gender-related features. They ended up with a model that could determine the gender of a job applicant just from subtle language cues in how they write their CV - usage of words like 'captured' and 'executed' which were used more frequently by one gender.
ML is worthless if blinded to anything that might give away a sensitive feature about an individual because that's virtually everything. And the current generation of ML is making it it really clear that the world as a whole discriminates on these features all the time, whether directly or indirectly.
These are the facts, the world is not evenly distributed. The question for society now is where do we go from here, and what exactly should 'fair' mean. Do we introduce reverse bias into processes to equalize the output distribution? Do we change the criteria considered by various processes to try to influence the output distribution? Do we look at the underlying social causes of the non-uniform distribution and try to address those? Do we just leave things as they are? Do we legislate to get ML out of critical social processes? Should we maybe first address much more obvious cases of unfairness that don't even require ML to identify, like massive and growing wealth disparity? Google's idea of fairness is only one of these choices, and it does not represent a public consensus on the subject - the public was never consulted. And yet the public is increasingly being impacted by this issue.
> ML will easily infer gender/color/religion from any other features about an individual. It should not be too surprising that those features play a big role in identity. Consider the recent news about Amazon throwing out a model which kept learning to discriminate on gender no matter how much they tried to blacklist gender-related features. They ended up with a model that could determine the gender of a job applicant just from subtle language cues in how they write their CV.
Ah yes, subtle clues like the word "women's" and "girl's".
> ML is worthless if blinded to anything that might give away a sensitive feature about an individual because that's virtually everything. And the current generation of ML is making it it really clear that the world as a whole discriminates on these features all the time, whether directly or indirectly.
It's not a problem that the network could identify that fact. It's a problem that the training data for Amazon contained this, because it's pretty clear there is an illegal and unfair bias in Amazon hiring. The data, from the stories we have, didn't include anything but successful hire data. No credible notion of "success" for post-hires has been indicated here.
> Google's idea of fairness is only one of these choices, and it does not represent a public consensus on the subject - the public was never consulted. And yet the public is increasingly being impacted by this issue.
Have you looked at the courseware? The entire courseware is about how these systems optimize to fit data, not to fit truth. They explore numerous ways in which that subtlety creates surprising or undesired outcomes.
It's very difficult to imagine a world where more awareness of the limitations of mathmatical models will lead to an undesirable outcome. Which one of these core tenants from the syllabus are YOU opposed to?
* Engage with a diverse set of users and use-case scenarios, and incorporate feedback before and throughout project development. This will build a rich variety of user perspectives into the project and increase the number of people who benefit from the technology.
* Model potential adverse feedback early in the design process, followed by specific live testing and iteration for a small fraction of traffic before full deployment.
* Consider augmentation and assistance: producing a single answer can be appropriate where there is a high probability that the answer satisfies a diversity of users and use cases. In other cases, it may be optimal for your system to suggest a few options to the user. Technically, it is much more difficult to achieve good precision at one answer (P@1) versus precision at a few answers (e.g., P@3).
* Ensure that your metrics are appropriate for the context and goals of your system, e.g., a fire alarm system should have high recall, even if that means the occasional false alarm.
Most are equally best practice. It's a course designed to supplement the depth and breadth of education in a world increasingly staffed by ML practitioners who learned the entirity of their courseware from 2 free Coursera courses and a bunch of Siraj Raval videos.
You're talking about "fairness" like this course has ML-based policy decisions or even addresses that substantially. This is a category error.
Fair doesn't mean the same as legal. And actually, positive discrimination is also legal and constitutional, and is not just "making decisions independent of race/characteristic".
I think the problem is the shift from 2 different kinds of independent. In case of stealing, for example:
The constitution means (this has been clarified enough) that you get convicted if you steal, irrespective of other factors like skin color or religion.
The modern interpretation is that every race and religion must steal at the same rate, irrespective of the truth in any particular case.
This is the shift people seem to have trouble with.
> The problem with them using words like 'fairness' is anyone who disagrees with what they're doing is immediately put on the defensive and labelled as 'unfair'. This is far from the truth though, as what society today is actually doing, is trying to figure out exactly what 'fair' should mean. Google et al just took it upon themselves to start pushing their particular brand of fairness onto society and acting as if it's the most natural widely-accepted common sense.
I'm sorry, but this comment seems reactionary in the extreme. Could you identify aspects of the courseware itself that are "unfair" in the sense you're using? Or give a misleading definition of "fair"?
> Not enough <demographic> represented in the workforce? Bias the candidate selection model to equalize it. Some <demographic> is over-represented in crime statistics? Bias jail sentencing model to be more lenient on them and hope it equalizes their representation in stats over time.
Further, followups like this suggest you're afraid that this is a play by Google to lead to the (presumably undesirable, from your tone) outcome of fairness. You're not-very-subtly suggesting it'd be unfair for black people to do less jail time for the same crimes, which has no relevance to the conversation at hand. Why include this? Is this a subtle signal? A tangent slide you fell down? A bad edit of a post?
> I don't know what 'fairness' should actually mean in modern society, but we definitely need to be aware that the US tech industry is hijacking the term to fit their agenda, and need to be checked. And the exact nature of the biases they're injecting should definitely be made transparent and put to public debate.
It's also extremely important to note that cultural elements are using the same processes to normalize acute unfairness in the workplace and in policy, often under the cover of "the machine predicted it."
Making sure that a broader class of people know that all machine learning does is fit models to pre-existing data (and therefore are quite limited in what kind of novel insight they can glean safely, requiring utmost care in the model design and data normalization) is an incredibly good idea, both for the industry itself and for society as a whole. It's in the nearly the exact same vein as, "I wish more people understood the implications of Bayes/Laplace's rule or the law of large numbers." The only downside is that fewer folks will be able to peddle complex-sounding ML methods that boil down to (from a practitioner's standpoint) a few lines of trivial Python as if they are the cutting edge of AI research.
There seems to be big confusion on what is bias and fairness and this course doesn’t seem to help. Fairness is ensuring that peaple are not deliberately punished for things that they have no control over. Typically this translates to making sure that decisions made by governments and employers do not depend on race, color and gender. US constitution adds religion in to the mix and some EU countries may also want to include sexual orientation, disability, age, political orientation but the minimal thing required by US laws in most states are those 4 attributes.
Every ML model can have different bias on different features. That’s what makes model a model. The fairness means you should make sure that your model is not biased towards above specific attributes and explicitly test for it. This is not some liberal propaganda by tech but required by law in many instances even if ML was not involved. This only applies to models used for certain classes of decision making, for example, loan eligibility or hiring or FaceID login. It doesn’t apply to things like breast cancer predictions or shopping recommendation system.
It's more than this, though. For instance, is the presence of the word "captured" (e.g. in a dataset of performance reviews) correlated with gender? This was certainly the case for Amazon: https://www.theguardian.com/technology/2018/oct/10/amazon-hi... . If so, even if you don't include gender as a feature itself, your outputs may end up being biased (in the technical sense) by gender. If you want to correct for this, then you might want to include gender in your analysis in a structured way in order to determine: are there better features that apply relatively equally to indicate a superstar of either gender? Personally, I don't know what the battle-tested approaches in the ML literature are to do this type of thing, so this course would be helpful to people like me!
At the end of the day, it all boils down to this: you can't be "blind" to race, color, religion, and gender if information is leaked to your evaluator (human or otherwise) via a side channel. You might get away with this legally (especially if there's not intent to discriminate), but if you really want to do it right, you have to engineer systems that minimize those side channels.
> If so, even if you don't include gender as a feature itself, your outputs may end up being biased (in the technical sense) by gender.
Part of the problem is people using the same word to mean multiple things. For instance, "bias" has a precise mathematical definition in the context of statistics: https://en.wikipedia.org/wiki/Bias_(statistics) . And this sentence makes no sense with that definition. In fact, with linear models it is mathematically impossible to make a "worse" model (in terms of mean squared error) by including more variables (like gender, age, race, etc...).
> you can't be "blind" to race, color, religion, and gender
Also I am not sure that this train of thought actually leads to where we want to go. A perfect model isn't necessarily blind to these features, a perfect model treats everyone as an individual.
> In fact, with linear models it is mathematically impossible to make a "worse" model (in terms of mean squared error) by including more variables (like gender, age, race, etc...).
That's only true if you mean the mean squared error on the training data, which is not usually a good indicator of model quality. Instead you should use the mean squared error on test data, which gets worse if you add non-predictive variables to the input.
If there are non-predictive variables, the linear model with the lowest expected square error should assign exactly zero weight to them, equivalent to the situation where those variables don't exist. But training on a finite sample, that "exactly zero" outcome is extremely unlikely (as in, the probability is 0) if the non-predictive variables vary at all. That variance allows identifying individual data points, even though the relationship is completely random and doesn't help generalize to unseen data. In other words, the model overfits to noise.
A theoretical ML paper came out recently that directly ties the notion of bias/fairness in machine learning to the tuning of the bias parameter in deep neural network architectures. The authors used stochastic backpropagation to classify an unbalanced dataset (85% class 0, 15% class 1) and show, through a series of topological cross-validation techniques, that very small amounts of perturbation in the bias parameter results in significant increase in the overall bias of the model towards the minority class (class 1).
122 comments
[ 2.9 ms ] story [ 156 ms ] threadIf so, this is not to be encouraged. It would be the equivalent of the sciences rejecting results when they don’t conform to current theory. Go to extra lengths to validate the results, sure, but don’t throw them away out of hand.
What we may be debating is whether gibberish is more or less morally good depending on whether it includes race and gender.
I think this is a myopic view of the current state of ML.
Lets say you are writing ML related to loan applications or deciding if someone gets parole. Do you want to "accurately" look at the historical data? Or should the decision also include the context of that data with an eye towards making a decision that is more socially appropriate than perpetuating the racism of the past?
> THE YEAR WAS 2081, and everybody was finally equal. They weren't only equal before God and the law. They were equal every which way. Nobody was smarter than anybody else. Nobody was better looking than anybody else. Nobody was stronger or quicker than anybody else. All this equality was due to the 211th, 212th, and 213th Amendments to the Constitution, and to the unceasing vigilance of agents of the United States Handicapper General.
Residential segregation is a good example of the difficulty of quantifying the causality of fairness. On one hand, residential segregation is a human universal. From the Papuan jungle to the London asphalt jungle, humans tend to stick with their own kin, for some definition of own kin. People have built simulators demonstrating that even a weak preference for own kin leads to significant residential segregation [1]. On the other hand, there is a history of forced segregation, well documented in many places [2][3] etc.
Furthermore there are more than one forms of segregation, read literally as 'separation from others'. There is the well studied economic and racial residential segregation, the lack of easy access to more economically desirable living quarters. On the flip side, USA is already a very atomised and isolating society. Some would point to the addiction and homelesness crises as direct consequences of social isolation [4]. Living isolated from one's own kin is yet another factor increasing social isolation, and a different kind of segregation.
At what point does enforced fairness morph from an attempt to fix the past ills into the tyranny of today, steering people toward actions that are contrary to their natural preferences and well being? Or is there no such point and we're willing to bludgeon our way into an all out harrisonbergeronian future?
[0] http://tnellen.com/cybereng/harrison.html [1] https://ncase.me/polygons [2] https://en.wikipedia.org/wiki/Residential_segregation_in_the... [3] https://en.wikipedia.org/wiki/Jim_Crow_laws [4] https://www.ted.com/talks/johann_hari_everything_you_think_y...
Is it really imperative to treat reality as if it was uniformly distributed? Why would you want to cast a non-linear environment into a linear shape? Are we trying to get rid of non-uniformity (which is realistically impossible, let's be honest) or are we just correcting towards the falsehoods of current thinking?
I had to take out latitude and longitude out of a model once... The area was not only indicative of income but of race.
These are dialogues we need to have about “why is this occurring”, not talk about bias. You are white washing reality and blaming an algo.
You aren't hiding anything, training wasn't (and isn't ever really) rigorous. You're more than likely discovering issues in your data set. It's like polling. You don't cover up politically inconvenient polls, but there's often a level of post processing you need to do to account for biases in your sample.
Same with ml, except often times the first data set, or three, are useless because no one has ever analyzed them before and doesn't know how to post process correctly.
It's not "fair" to victims of crime if we hamstring investigations just because of political consideration.
I can easily write you a completely unbiased “model” predicting who comitted crimes:
This is 100% unbiased and treats everybody the same. It’s also 100% useless...The more autonomy plays a role in our lives, the more aware we need to be of how internal bias from human source has affected things like predictions.
And please, there is nothing more flippant and off-putting than railing against "SJWs," this is a topic that demands thoughtfulness.
Can you explain what you mean? “Biased” usually means not using just statistics but also extra-statistical personal convictions. What does “unbiased” mean for you?
There are hundreds of questions like that which apply. It's not (necessarily, it certainly can be) the algorithm that carries bias, it's the data.
Sure, which is why the fair machine learning literature has developed more nuanced (and IMO better) definitions of fair predictors.
For example, suppose you have some sensitive attribute A (say race) and want to know some outcome Y. A predictor \hat Y satisfies equalized odds if \hat Y and A are conditionally independent given Y. In binary classification, this means that the classifier has identical false positive rates and false negative rates across races. So e.g. your false negative rate for white people must be the same as your false negative rate for black people. Note that a perfectly accurate classifier satisfies equalized odds -- roughly, its a way of guaranteeing that errors are not concentrated in some group.
I think it's unfair to characterize fair ML research as the work of "SJWs". There's a lot of smart, technical work done by smart, technical people thinking hard about these problems.
Moritz Hardt's course on the topic is still a decent intro: https://fairmlclass.github.io/
Edit: biased not busy.
Such a system would end up requiring black people have higher credit scores than otherwise equal white counterparts, in order to be granted whatever thing it is. That's, as modern feminist/race theory might define it, an implicit bias. Nowhere is anyone explicitly stating "black implies bad", but the way the model is built creates unequal representation.
That part is to be expected, the important thing is that you don't then recorrelate, either explicity or implicitly, race with the lower credit scores, that can essentially double count race (via income and then via the second check) and that's what you want to avoid.
By programming them this way (avoiding programming in human frailties for the purpose of better "fitting in"), I think we can more quickly expose and correct certain areas where us bags of mostly salt water act suboptimally.
The bigger point I'm making is people think AI acts in some form of posthuman reason. At this time no such thing exists: decisions are still bound to human bias, and we should at least consider optimizing with regard to the rules of a modern society.
Humans are constrained by the knowledge that they would dread for the rest of their life the knowledge that they optionally and intentionally took an overt action which resulted in the death of an innocent person (to save five, but they would know they killed one when they could have done nothing and had "clean hands"). Machines don't need to suffer from this.
I would hope that if we program safety-controlling systems that we wouldn't replace the compare and conditional-jump with "halt and catch fire".
In the problem statement, an actor has control over whether this 1 person or those 5 people die. If they choose red, then 1. If they choose blue, then 5. (Failing to make a choice is choosing blue.) If after playing the scenario, they knew someone would use a Men In Black pen on them to completely forget what happened, is there a rational argument against choosing red?
(I realize this might read like trolling, but I genuinely don't find the original problem a difficult one if you eliminate the human guilt aspect.)
Actively choosing an action to deliberately kill someone else could be weighed negatively by the machine, it would depend on the programmer.
I don't think you can apply the word "rational" here, if you don't take a position of utilitarianism, not deliberately killing the person could be axiomatically rational. It's not just about guilt.
The brakes had suddenly failed approaching an intersection and the car had the choice of either running into a group of five pedestrians crossing the street or slamming into the concrete wall to your right, killing only you. Which is the correct answer?
Now suppose it was not you in the car but your wife and two daughters. Would you want your car, the car you bought and paid for, to sacrifice your family to save the lives of five strangers? What if it were only four strangers? Or ten?
There is no right answer. But as humans we're biased to favour, if not our own lives, the lives of our closest family members over strangers. I, personally, would never buy a car if I knew it would make the choice to sacrifice my family to save strangers.
Concretely, the car should take the course of action to minimize the global harm based on the knowledge it has. If that's killing me in the event of a brake failure, so be it. The spectacularly unlikely probability of that is outweighed by the million more times when it and others like it will make a less selfish decision than today's human drivers.
> I, personally, would never buy a car if I knew it would make the choice to sacrifice my family to save strangers.
The point I find interesting here is "this consumer preference, if common, confers a propagation advantage to systems which behave selfishly".
This statement is misleading.
"Instinctual and observational impulses" is poorly defined, but the low latency cognitive systems I think you are describing (such as object categorization) are inputs to higher executive functioning systems which determine behavior.
The fact of and ability to "override" generalizes too much about how these low latency systems are prioritized a-priori. In other words, some of them seem to be able to be able to be re-prioritized fairly easily (positive/negative response to specific olfactory stimuli), while others approach impossible without motor system intervention (Not reacting to severe pain stimuli).
It's categorically misleading to separate these from all other functions of Perceive > Process > Respond as all decisions are all built on the same reinforcement learning foundations.
Searching for razor doesn't give results for occam's razor.
> Regarding the search for [white american inventors], the results would reflect the way the images are labelled. Check the image descriptions and you will find that the image results all contain some of the words you searched for. Maybe the webmasters who label images of inventors have no interest in the colour of the inventor's skin!
https://support.google.com/websearch/forum/AAAAgtjJeM4xP2ts6...
Not even a paragraph in and I see problems with this. Seems somebody bias has already gotten in their own way.
This is the first step - redefine the meanings of words to fit your agenda.
I could be wrong about the intent. But when you can redefine fairness the you can make it fair to take from one person and give to another. Or deny one person and Grant access to another based on things that don't matter.
Also. The attack on human bias is totally silly. It's a key trait that had made us and other species successful. Without these biases we might pick the wrong fruit eat it and die. We might try to cuddel into a cave with bears in the winter.
Our entire existence and reasoning skills is based off bases.
Also. Never in my life have a seen ripe or unripe bannans and described them by their color. Is this a education problem?
But even if I had there is nothing wrong with pointing out when something is not atypical. Besides if I plan on eating bananas in a few days I might want the "green" ones. Or if I was going to make a yummy smoothie the "brown" ones are the ones I want.
Also bananas are a bad analogy for what they were getting at. Every banana has the potential to be green, then yellow, then brown. So none of these states are out of the ordinary.
I don't need or want to be unconsciously retrained by ML. I think we are going to need laws against big cooperations using their power and tech to change socal constructs. It's not their place and we can never be sure they are doing it for good or even the outcome will be good for human kind. It's also a power that can be abused. We already have internal documents comming from Google suggesting swaying public opinion on on political canadates. And this type of research is exactly where you would start.
It mattered less in the past because humans made all the decisions so we could fall back on fuzzy qualitative notions. We can't do that when algorithms make decisions.
Maintaining such constructs is as much a use of power as changing them is. It’s completely impossible to outlaw a corporation’s or any agent’s influence on society. ‘Society’ as we think of it (the nation, a community, bureaucracy, you name it) itself is a massive construct with tons of subjectivity attached.
> I don't need or want to be unconsciously retrained by ML.
If it’s a thing in your world, which it probably already is, it will have an influence on you. This type of research is how we make that visible. IMHO the social sciences are a good thing.
Honestly. We would have to ban all marketing and mass media. The only other solution I can think of is to subject all relevant industries to some democratic process, and I don't think anybody really wants that.
But really, think there is a difference in trying to get me to buy your microwave and a small group of people attempting to retrain the masses on social issues -- according to their views -- "for the good of everybody".
It's really hard for me to understand how anybody can think it would be good for any small group, no matter how much good they think they are doing, or how pure their intent is to have the power to change the masses, little alone be allowed to use such power.
Things are complicated one wrong step and these people can break society.
Ugly short men in a corporate environments have to swim against the tide because everyone instinctively thinks less of them. Tall handsome men get automatic adoration. Prejudice is irrational and harmful. Bill Gates and Bezos are too short to be CEOs. The world is filled with irrational biases that do more harm than good.
If white, then reduce privileges If black, then give reparation as structRep If binary gender, then increase non-binary sexual imagery; equalize all norms If young female wants to be a mother, then show videos of females "succeeding professionally"
Again, at the top of HN with the most political article, but commenters like myself are censored from being able to even give our take on why they're extremely misleading. How is this not exactly like Brave New World?
Ethically bankrupt advertising corporations have no business teaching people about ethics.
This isn't a normative statement, just descriptive. Whether or not google should bias its results is a completely different discussion.
2) I think Google is acutely aware that their results are driven by human behavior and thus are biased. It's the nature of its design
But If you have a better theory of how the 10 of the first 20 "american scientists" are black and 5 are women, I'd be interested to hear it.
Result #6 is the list of African-American inventors and scientists on Wikipedia. Unless Baidu has the same ideological biases as Google (would be strange), the most likely explanation is that it's driven by n-gram frequencies.
Here's the underlying data at duckduckgo: https://duckduckgo.com/?q=american+scientists&t=h_&ia=list
I'm still interested in a possible technique which could lead to this type of bias without it being explicit (or requiring google to have an extremely naive approach).
> I'm still interested in a possible technique which could lead to this type of bias without it being explicit (or requiring google to have an extremely naive approach).
I don't understand. If it's what you expect, then what's left to explain?
I very strongly doubt their approach is based on substring search. They're obviously using a knowledge graph. And if you try a search for "American economists" or "American philosophers" the results look much more expected, either the "American" in this case is not a substring of "African-american" or they simply thought that economy and philosophy aren't as worth of an equality boost as STEM disciplines.
"A black inventor would be seen as out of the ordinary, and so be referred to as an African American inventor, stacking the deck in favour of those results."
So the results look biased. Now, it might just be by chance; however all this talk from Google about fairness and equal representation makes me a bit suspicious. Not certain, just suspicious. I would have preferred a company that just said "look, we're engineers, this is the data, these are the algorithms, we don't care about the outputs, deal with it". But it looks like that stopped happening at Google (I'm not saying in this specific case, I'm saying in general) a long time ago.
You don't think Google search is using 2-grams? Do you think they're conspiring with other search engines? https://www.bing.com/images/search?q=american+scientist
https://www.google.com/search?q=usa+scientists
apparently gives results from the same knowledge graph and displays them under the same heading, but orders them differently from:
https://www.google.com/search?q=american+scientists
So apparently Google, while understanding the search query, still orders the results by the words used to express it- and in the second case clearly privileges African-Americans because of the "American" substring. My bad.
For instance, we would be appalled if a court found that the conviction rate for people like the defendant was extremely low, so the trial was going to start with the presumption of 80% guilt.
What's the problem? It's not one fixed by better modelling or a better bias percentage. The problem is that fairness in many contexts is about processes and institutions. We expect courts to follow principles like due process and equal protection under the law.
Companies like Google can't jump right to social science and social engineering without thinking about fairness and justice in these terms.
Can a user confirm that the data about them is accurate? If not, what can they do about it?
If a business has its account algorithmically cancelled, what is the appeals process? How is it fair?
When companies like Google start getting into social science, the problem is extremely complicated fast. For example, it starts looking like a judicial system in parts, but one that isn't established and maintained by a legitimate government.
If Google wants to be fair, it's not just a matter of data, demographics, and statistics. It needs to think about what philosophers like Kant and Rawls would say about treating even outliers and exceptions equitably.
On the other hand, if you already know at the outset the result that you would like your algorithm to produce, then why bother with machine learning at all? Just hard-code your output.
I'll just call their attention to their own rule on this: "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."
The weak interpretation of these dead comments is that they're just racists who hate dark-skinned people. There are much stronger interpretations and those should be discussed.
Especially in the context of Google, where it's been demonstrated over and over through video leaks, court filings and the Damore firing, that a single and very heavy-handed ideology has power and will not allow discussion. From a centrist position is seems obvious Google's internal thought process will be itself heavily biased, and the possibility of certain factual hypothesis being true about the physical world (not normative statements on what to do) will be denied for ideological reasons. It's like a group of fundamentalist Christian scientists studying geology. This needs to be acknowledged in any discussion of Google, ML, and 'bias'.
To sport the classic over simplification, ML is just glorified curve fitting. But in reality, it's not like we created a special new species with intelligence that we have to careful teach not to be racist. _We_ just have to be careful not to be racist with it.
The problem with them using words like 'fairness' is anyone who disagrees with what they're doing is immediately put on the defensive and labelled as 'unfair'. This is far from the truth though, as what society today is actually doing, is trying to figure out exactly what 'fair' should mean. Google et al just took it upon themselves to start pushing their particular brand of fairness onto society and acting as if it's the most natural widely-accepted common sense.
Google's idea of fairness is to build ML solutions that don't accurately predict the world, but rather inject Google biases into the predictions in order to bias some real-world process, and ultimately change the distribution of some factors in the real world according Google's image of how things should be. Not enough <demographic> represented in the workforce? Bias the candidate selection model to equalize it. Some <demographic> is over-represented in crime statistics? Bias jail sentencing model to be more lenient on them and hope it equalizes their representation in stats over time.
This is not inherently a bad mechanism, but it's an extremely powerful mechanism with far-reaching consequences, and it is currently controlled by a small number of people at the top of the US tech industry, with no checks or oversight. I worry that there's a strong possibility that the long term effects of messing around with these levers have not been considered. We have done serious unintentional damage with far smaller levers like the Australian Rabbit Plague.
I don't know what 'fairness' should actually mean in modern society, but we definitely need to be aware that the US tech industry is hijacking the term to fit their agenda, and need to be checked. And the exact nature of the biases they're injecting should definitely be made transparent and put to public debate.
ML will easily infer gender/color/religion from any other features about an individual. It should not be too surprising that those features play a big role in identity. Consider the recent news about Amazon throwing out a model which kept learning to discriminate on gender no matter how much they tried to blacklist gender-related features. They ended up with a model that could determine the gender of a job applicant just from subtle language cues in how they write their CV - usage of words like 'captured' and 'executed' which were used more frequently by one gender.
ML is worthless if blinded to anything that might give away a sensitive feature about an individual because that's virtually everything. And the current generation of ML is making it it really clear that the world as a whole discriminates on these features all the time, whether directly or indirectly.
These are the facts, the world is not evenly distributed. The question for society now is where do we go from here, and what exactly should 'fair' mean. Do we introduce reverse bias into processes to equalize the output distribution? Do we change the criteria considered by various processes to try to influence the output distribution? Do we look at the underlying social causes of the non-uniform distribution and try to address those? Do we just leave things as they are? Do we legislate to get ML out of critical social processes? Should we maybe first address much more obvious cases of unfairness that don't even require ML to identify, like massive and growing wealth disparity? Google's idea of fairness is only one of these choices, and it does not represent a public consensus on the subject - the public was never consulted. And yet the public is increasingly being impacted by this issue.
https://www.reuters.com/article/us-amazon-com-jobs-automatio...
https://www.cnbc.com/2018/03/02/google-accused-in-lawsuit-of...
Ah yes, subtle clues like the word "women's" and "girl's".
> ML is worthless if blinded to anything that might give away a sensitive feature about an individual because that's virtually everything. And the current generation of ML is making it it really clear that the world as a whole discriminates on these features all the time, whether directly or indirectly.
It's not a problem that the network could identify that fact. It's a problem that the training data for Amazon contained this, because it's pretty clear there is an illegal and unfair bias in Amazon hiring. The data, from the stories we have, didn't include anything but successful hire data. No credible notion of "success" for post-hires has been indicated here.
> Google's idea of fairness is only one of these choices, and it does not represent a public consensus on the subject - the public was never consulted. And yet the public is increasingly being impacted by this issue.
Have you looked at the courseware? The entire courseware is about how these systems optimize to fit data, not to fit truth. They explore numerous ways in which that subtlety creates surprising or undesired outcomes.
It's very difficult to imagine a world where more awareness of the limitations of mathmatical models will lead to an undesirable outcome. Which one of these core tenants from the syllabus are YOU opposed to?
* Engage with a diverse set of users and use-case scenarios, and incorporate feedback before and throughout project development. This will build a rich variety of user perspectives into the project and increase the number of people who benefit from the technology.
* Model potential adverse feedback early in the design process, followed by specific live testing and iteration for a small fraction of traffic before full deployment.
* Consider augmentation and assistance: producing a single answer can be appropriate where there is a high probability that the answer satisfies a diversity of users and use cases. In other cases, it may be optimal for your system to suggest a few options to the user. Technically, it is much more difficult to achieve good precision at one answer (P@1) versus precision at a few answers (e.g., P@3).
* Ensure that your metrics are appropriate for the context and goals of your system, e.g., a fire alarm system should have high recall, even if that means the occasional false alarm.
Most are equally best practice. It's a course designed to supplement the depth and breadth of education in a world increasingly staffed by ML practitioners who learned the entirity of their courseware from 2 free Coursera courses and a bunch of Siraj Raval videos.
You're talking about "fairness" like this course has ML-based policy decisions or even addresses that substantially. This is a category error.
The constitution means (this has been clarified enough) that you get convicted if you steal, irrespective of other factors like skin color or religion.
The modern interpretation is that every race and religion must steal at the same rate, irrespective of the truth in any particular case.
This is the shift people seem to have trouble with.
I'm sorry, but this comment seems reactionary in the extreme. Could you identify aspects of the courseware itself that are "unfair" in the sense you're using? Or give a misleading definition of "fair"?
> Not enough <demographic> represented in the workforce? Bias the candidate selection model to equalize it. Some <demographic> is over-represented in crime statistics? Bias jail sentencing model to be more lenient on them and hope it equalizes their representation in stats over time.
Further, followups like this suggest you're afraid that this is a play by Google to lead to the (presumably undesirable, from your tone) outcome of fairness. You're not-very-subtly suggesting it'd be unfair for black people to do less jail time for the same crimes, which has no relevance to the conversation at hand. Why include this? Is this a subtle signal? A tangent slide you fell down? A bad edit of a post?
> I don't know what 'fairness' should actually mean in modern society, but we definitely need to be aware that the US tech industry is hijacking the term to fit their agenda, and need to be checked. And the exact nature of the biases they're injecting should definitely be made transparent and put to public debate.
It's also extremely important to note that cultural elements are using the same processes to normalize acute unfairness in the workplace and in policy, often under the cover of "the machine predicted it."
Making sure that a broader class of people know that all machine learning does is fit models to pre-existing data (and therefore are quite limited in what kind of novel insight they can glean safely, requiring utmost care in the model design and data normalization) is an incredibly good idea, both for the industry itself and for society as a whole. It's in the nearly the exact same vein as, "I wish more people understood the implications of Bayes/Laplace's rule or the law of large numbers." The only downside is that fewer folks will be able to peddle complex-sounding ML methods that boil down to (from a practitioner's standpoint) a few lines of trivial Python as if they are the cutting edge of AI research.
Every ML model can have different bias on different features. That’s what makes model a model. The fairness means you should make sure that your model is not biased towards above specific attributes and explicitly test for it. This is not some liberal propaganda by tech but required by law in many instances even if ML was not involved. This only applies to models used for certain classes of decision making, for example, loan eligibility or hiring or FaceID login. It doesn’t apply to things like breast cancer predictions or shopping recommendation system.
At the end of the day, it all boils down to this: you can't be "blind" to race, color, religion, and gender if information is leaked to your evaluator (human or otherwise) via a side channel. You might get away with this legally (especially if there's not intent to discriminate), but if you really want to do it right, you have to engineer systems that minimize those side channels.
Part of the problem is people using the same word to mean multiple things. For instance, "bias" has a precise mathematical definition in the context of statistics: https://en.wikipedia.org/wiki/Bias_(statistics) . And this sentence makes no sense with that definition. In fact, with linear models it is mathematically impossible to make a "worse" model (in terms of mean squared error) by including more variables (like gender, age, race, etc...).
> you can't be "blind" to race, color, religion, and gender
Also I am not sure that this train of thought actually leads to where we want to go. A perfect model isn't necessarily blind to these features, a perfect model treats everyone as an individual.
That's only true if you mean the mean squared error on the training data, which is not usually a good indicator of model quality. Instead you should use the mean squared error on test data, which gets worse if you add non-predictive variables to the input.
If there are non-predictive variables, the linear model with the lowest expected square error should assign exactly zero weight to them, equivalent to the situation where those variables don't exist. But training on a finite sample, that "exactly zero" outcome is extremely unlikely (as in, the probability is 0) if the non-predictive variables vary at all. That variance allows identifying individual data points, even though the relationship is completely random and doesn't help generalize to unseen data. In other words, the model overfits to noise.