If the goal is equity of outcome above-all-else, ignoring for any differences derived from the data, then why are we bothered investing so much time, money and effort into this area?
There is, however, some merit to this concept. There is a problem if you are training hidden variables which are dependent on a "race" or similar field in a record. Outside of that, it is arguably unethical to attempt to remove any bias (bias, of one sort or another, being the whole point of deep learning) which does not produce hidden variables based on a protected characteristic.
E.g, in medical diagnostics, with low quality inference inputs, having a "race" field could save lives; in your resumé system, it could unfairly disadvantage (or advantage) somebody based on hidden variables you did not measure which just happen to correlate with race, also leading possibly to worse results.
It's not just about letting the data speak. The data is gathered by someone, containing historical bias we might nowadays disagree with, models are chosen by people, using parameters set by people, using evaluations set by people. It's about making sure models are both predictive, usable, and fair.
I find the premise that different groups should expect the same percentage of interventions highly suspect. Imagine we have a program that distributes seeing eye dogs. This toolkit would discover that sighted persons have a 0% chance of getting a dog, while blind persons have a 50% chance. Oh the injustice!
7 comments
[ 3.0 ms ] story [ 26.7 ms ] threadThere is, however, some merit to this concept. There is a problem if you are training hidden variables which are dependent on a "race" or similar field in a record. Outside of that, it is arguably unethical to attempt to remove any bias (bias, of one sort or another, being the whole point of deep learning) which does not produce hidden variables based on a protected characteristic.
E.g, in medical diagnostics, with low quality inference inputs, having a "race" field could save lives; in your resumé system, it could unfairly disadvantage (or advantage) somebody based on hidden variables you did not measure which just happen to correlate with race, also leading possibly to worse results.
This article: https://www.nature.com/articles/d41586-018-05469-3 also highlights the issues pretty well IMO.
It's not just about letting the data speak. The data is gathered by someone, containing historical bias we might nowadays disagree with, models are chosen by people, using parameters set by people, using evaluations set by people. It's about making sure models are both predictive, usable, and fair.