Gelman's specific critique of Domingos is as follows:
"And here’s the half that I think Domingos gets wrong: He’s too sanguine about existing algorithms being unbiased. I don’t know why he’s so confident that existing algorithms for credit-card scoring, parole consultation, shopping and media recommendations, etc., are unbiased and not capable of outside improvement. I respect his concern about political involvement in these processes—but the existing algorithms are human products and are already the result of political processes. Again, his concern that “progressives will blithely assign prejudices even to algorithms that transparently can’t have any,” is missing the point that the structure and inputs of these algorithms are the result of existing human choices."
There’s no argument here. Domingos stated that there are algorithms that cannot have bias, and Gelman is claiming that there are algorithms that can have bias. These are both objectively true, but Gelman’s twists it into an ad-hominem. This reads like mendacious virtue signaling. Ironic too that he would be the one to levy accusations of unprofessorial behavior.
It would be very hard to convince me that this was not motivated by cheap point-scoring for political reasons, perhaps as defense in anticipation of an attack upon himself.
Domingos’s research is specific to naive algorithms learning from data in experimental realms. Gelman’s work often relates to expert-designed simulations which are actively used in economic policy, banking, insurance, and legal systems. From this perspective, it’s somewhat clear that Domingos’s approach has far less capacity for bias, and thus invites criticism of the older method. Learning from data versus designed simulation. This reads somewhat like a reply suggesting we should trust hand-engineered bias from the right hands rather than seek to eliminate it.
But in this case, the hand’s argument is an ad-hominem, so we see that it’s a religious argument about social values rather than a theoretical argument about statistical bias. Maybe we should have that conversation. Yes, we can use algorithms that have no statistical bias. Yes, we can insert any bias we want into an algorithm. Now that that’s established, what should we do?
I think this comes down to what you think biases are and what you think should be optimised.
Consider the gender imbalance in tech jobs. Let's say that is because more men pursue tech degrees than women and this is because the field is already male dominated and women don't have female role models.
A hiring algorithm will optimise for the best candidate for the job. If we assume men are as good as women, the gender ratio will be skewed simply because the underlying population has a skew. However, we, as a society, may want to overweight women candidates even at a cost of choosing a slightly less fitting candidate because, in the long term, this will provide female role models and convince more women to join tech (there are reasons why this may be desirable, but it does have to be argued for).
Under this view, the algorithm is biased because it's optimising for the wrong thing - it is assigning no extra value to choosing a female candidate even if we, as a society, think there is value.
This is a thorny issue, though, as I've made some assumptions above and plenty of people will disagree with them, on various grounds. Still, I think this is a useful way if thinking about biases.
Also, viewed like a problem of figuring out the correct function to optimise, this suggests fixing this problem via slow and heavy handed law making (e.g. a blanket rule enforcing hiring gender ratios) is probably the wrong way to do this.
By the way, hiring committees can also be thought of as algorithms and at least part of their biases can be explained when viewed as them optimising for the wrong thing.
This was a very clear way of putting it, thank you. Maybe it’s just me, but I find much of the discussion about algorithmic bias a bit fuzzy. Replacing “this algorithm is biased” with “this algorithm isn’t really optimizing what we intended it to” seems like a good step, with the added benefit that it requires people to be explicit about their objective function.
> Replacing “this algorithm is biased” with “this algorithm isn’t really optimizing what we intended it to” seems like a good step,
Why not simply: "we are biased" (that is, busybodies complaining about bias).
From GP:
> This is a thorny issue, though, as I've made some assumptions above and plenty of people will disagree with them, on various grounds.
It's not a thorny issue. No one has to listen to algos. If your algo is buggy and suggests to kill rather than hire a candidate, would you?
Secondly, how is anyone's business what kind of algos I run in course of operating a privately owned company?
The thorny issue is how did we get to this situation where property rights have become so weak that busybodies of all sorts feel entitled enough to order others how they ought to use their property (that is, data and algos).
Seems simple to me - test new algorithms and remove controversial variables. If they are better predictors then use that algorithm. Until then, accurate algorithms should be used - whether they determine that Republicans or race X are more likely to do y.
Don’t get me wrong - You have a duty to get the best models possible, and you should try variables that generalize at higher levels - income instead of race) for example and always be testing.
I heard a story about a particular parole board trying to use an algorithm to predict recidivism rates. It often predicted one race more likely to do so. There is some argument that each case should be looked at individually. I agree. But should we avoid addressing the issue if the algorithm is explainable and a good predictor? Instead we should use it to help lower reoffender rates NOT by denying parole but perhaps assigning additional resources to those more likely to offend once we have already granted parole.
The justice system should proportionally reward good behavior and punish bad behavior. It should neither reward nor punish "neutral" behavior. As parole is a part of the punishment it must be determined also according to these criteria.
Having inmates trying to guess at what some algorithm values so they can game it? Kafkaesque
A bit more leeway can be given to algorithms that companies use but justice system is very sensitive.
yeah, if the algorithm is secret, nobody will trust that it's fair
if the algorithm is public, it can and will be gamed. The solution is to make the algorithm be based on the things that you want people to game, but that becomes not much of an algorithm at some point...
I think it comes down to this question: should people interacting with a model have the right to know how that model behaves in ways that are socially relevant? In the parole example, should parolees have access to the knowledge that the model predicts different recidivism rates for different races and by how much?
In the business world, the answer is reached by answering the question "How would our reputation change if the public had access to this information?" The end result is that AI ethicists are hired for optics and subsequently have no real capacity to publish this information.
Is this similar to how we interact with food? Is the 1990 Nutrition Labeling and Education Act a good or bad analogy to use? How similar is this to a food company hiring a food inspector and then firing them when they discover that peanut might have contaminated some other products? I guess if people don't know they won't complain too much.
Who gets to decide if the result “ax + b” is biased? And by how much?
I believe in the wisdom of the crowds. I like democracy, direct democracy even more so.
We should stop pretending we know better than what our data is telling us, even if it hurts our feelings. Not everything true is going to make us feel good, more often the inverse is true.
10 comments
[ 3.1 ms ] story [ 38.9 ms ] thread"And here’s the half that I think Domingos gets wrong: He’s too sanguine about existing algorithms being unbiased. I don’t know why he’s so confident that existing algorithms for credit-card scoring, parole consultation, shopping and media recommendations, etc., are unbiased and not capable of outside improvement. I respect his concern about political involvement in these processes—but the existing algorithms are human products and are already the result of political processes. Again, his concern that “progressives will blithely assign prejudices even to algorithms that transparently can’t have any,” is missing the point that the structure and inputs of these algorithms are the result of existing human choices."
It would be very hard to convince me that this was not motivated by cheap point-scoring for political reasons, perhaps as defense in anticipation of an attack upon himself.
Domingos’s research is specific to naive algorithms learning from data in experimental realms. Gelman’s work often relates to expert-designed simulations which are actively used in economic policy, banking, insurance, and legal systems. From this perspective, it’s somewhat clear that Domingos’s approach has far less capacity for bias, and thus invites criticism of the older method. Learning from data versus designed simulation. This reads somewhat like a reply suggesting we should trust hand-engineered bias from the right hands rather than seek to eliminate it.
But in this case, the hand’s argument is an ad-hominem, so we see that it’s a religious argument about social values rather than a theoretical argument about statistical bias. Maybe we should have that conversation. Yes, we can use algorithms that have no statistical bias. Yes, we can insert any bias we want into an algorithm. Now that that’s established, what should we do?
Consider the gender imbalance in tech jobs. Let's say that is because more men pursue tech degrees than women and this is because the field is already male dominated and women don't have female role models.
A hiring algorithm will optimise for the best candidate for the job. If we assume men are as good as women, the gender ratio will be skewed simply because the underlying population has a skew. However, we, as a society, may want to overweight women candidates even at a cost of choosing a slightly less fitting candidate because, in the long term, this will provide female role models and convince more women to join tech (there are reasons why this may be desirable, but it does have to be argued for).
Under this view, the algorithm is biased because it's optimising for the wrong thing - it is assigning no extra value to choosing a female candidate even if we, as a society, think there is value.
This is a thorny issue, though, as I've made some assumptions above and plenty of people will disagree with them, on various grounds. Still, I think this is a useful way if thinking about biases.
Also, viewed like a problem of figuring out the correct function to optimise, this suggests fixing this problem via slow and heavy handed law making (e.g. a blanket rule enforcing hiring gender ratios) is probably the wrong way to do this.
By the way, hiring committees can also be thought of as algorithms and at least part of their biases can be explained when viewed as them optimising for the wrong thing.
Why not simply: "we are biased" (that is, busybodies complaining about bias).
From GP:
> This is a thorny issue, though, as I've made some assumptions above and plenty of people will disagree with them, on various grounds.
It's not a thorny issue. No one has to listen to algos. If your algo is buggy and suggests to kill rather than hire a candidate, would you?
Secondly, how is anyone's business what kind of algos I run in course of operating a privately owned company?
The thorny issue is how did we get to this situation where property rights have become so weak that busybodies of all sorts feel entitled enough to order others how they ought to use their property (that is, data and algos).
Don’t get me wrong - You have a duty to get the best models possible, and you should try variables that generalize at higher levels - income instead of race) for example and always be testing.
I heard a story about a particular parole board trying to use an algorithm to predict recidivism rates. It often predicted one race more likely to do so. There is some argument that each case should be looked at individually. I agree. But should we avoid addressing the issue if the algorithm is explainable and a good predictor? Instead we should use it to help lower reoffender rates NOT by denying parole but perhaps assigning additional resources to those more likely to offend once we have already granted parole.
Having inmates trying to guess at what some algorithm values so they can game it? Kafkaesque
A bit more leeway can be given to algorithms that companies use but justice system is very sensitive.
if the algorithm is public, it can and will be gamed. The solution is to make the algorithm be based on the things that you want people to game, but that becomes not much of an algorithm at some point...
In the business world, the answer is reached by answering the question "How would our reputation change if the public had access to this information?" The end result is that AI ethicists are hired for optics and subsequently have no real capacity to publish this information.
Is this similar to how we interact with food? Is the 1990 Nutrition Labeling and Education Act a good or bad analogy to use? How similar is this to a food company hiring a food inspector and then firing them when they discover that peanut might have contaminated some other products? I guess if people don't know they won't complain too much.
I believe in the wisdom of the crowds. I like democracy, direct democracy even more so.
We should stop pretending we know better than what our data is telling us, even if it hurts our feelings. Not everything true is going to make us feel good, more often the inverse is true.