When you put your destination into your phone and press "navigate", do you stop each time to check (or even just wonder) whether the route it gives you is biased in some way? Or do you just trust it to get you to your destination by the optimal route?
I think the important thing in that case is that I can trust it to get me there better than I could get me there.
I mean, yea, I do expect it to do some optimization, in that I expect it to use some path finding algorithm with some collection of weights on the different paths, or something like that, and for the instance of the path finding problem that it produces I expect the route to be optimal, but that the problem it produces may differ in some ways from the true most correct problem to solve,
but, in the end, the solution it gives me is better than the one I would produce, so, why do I care if it isn't quite optimal under the literally perfect criteria? (I mean, if another alternative gave better results that would be better, of course. I just mean that what it is isn't bad.)
I don't think navigation is the best example for this topic though. Navigation seems like a fairly solid/well-defined problem, not like more of the more fuzzy questions, like "does this picture include a face?" .
I rarely use driving directions from an algorithm, but when I do I always assume they are biased. Sometimes I ask them to be biased in a certain way, like stay away from tolls.
I think I agree with everything on that page, but this seems to be incomplete:
> It shouldn't be a mission impossible to find out how and why a decision about you was made.
That does sound attractive. But I don't see how it squares with technology. For certain technologies, the answer will be "the weights in the neural network, when applied to your attributes, yielded a score that fell above a cutoff" or "Your data attributes mapped to a certain location on an eigenvector and it fell above a cutoff defined on that eigenvector".
Is the answer to stop using such algorithms? If not, doesn't this webpage need to give us more of a hint as to how we are to overcome the apparent impasse?
Model explainability is a very active area of research. For the example of an individual neural network prediction you could use SHAP values[0].
Explainability is far from a solved problem but there are certainly tools available to provide at least some transparency. I guess the website could provide references but that doesn't really seem to be the resolution being aimed for.
If you can't guarantee that the algorithm respects protected classes (or other forbidden discriminations) then yes, you can't use it. Compare it to the hypothetical process where "we have Jim decide". You can't open up Jim's head and see why he made choices, but you can run tests against the system (Jim) to probe for forbidden discriminations.
It is very difficult to run test to be sure that Jim is not biased. The advantage of Jim is that you can ask Jim and then take at face value his answer. (And later, if there is a big problem you can just blame Jim and fire him.)
Lack of a (viable) alternative to a solution doesn't negate critiquing an existing solution. Critique can provide insight that leads to generation of new solutions or reforms to the existing solution to fix some of its underlying problems. You always end up with more information than you started, which is a good thing.
Showing in the 1800s that Newtonian Gravity fails to account for the precession of Mercury, with no idea on how to construct the General Theory of Relativity, doesn't make the critique of Newtonian Gravity any less valid.
Not to mention they provide 3 proposals of improving things.
More irritating to me, is most solutions essentially boil down to “give me (the author) all of the power”. For example, this page suggests hiring a “ethics expert” to perform an “algorithm audit”. Unsurprisingly the author of this page has started an ethics consulting company. Because there’s no way that they could be biased or use their power to benefit themselves, right?
It’s funny because it changes based on whoever writes the article. Management thinks that management should get final decision power, academics think that academia should get to audit the code.
Well, the author proposes the algorithm used should be open, i.e. known to the user. But this means the user will try to game it, so the creator would prefer to keep it secret. The big question is: can we make a transparent algorithm that can't be gamed (spam, SEO, etc.)? As cryptography demonstrates, this is definitely possible, but requires a lot of research, smart design and may not be cheap to implement.
Nothing deep here at all. It seems purely for show, little content. At the bottom "site by [...] Technology critic, Privacy designer and Public speaker" so I guess it's just personal puffery on the cheap.
this is a companion marketing piece (to social cooling[0]) for a social trends consultant. like that one, this one makes a loose argument on one topic (algorithmic bias) through sparse statements and vague references, even as the core argument might otherwise be interesting and rich. based on the quality of its points and the focus of the consultancy, it seems to be meant for reference on instagram/twitter, not rigorous debate.
Major eyeroll at this. This site conflates algorithms, data, machine learning, math, and specific applications of any of these.
Math _is_ neutral.
Algorithms _are_ neutral, and are simply instruction sets for operating on data.
Data is certainly be biased, but any researcher, designer, engineer, etc. worth their salt will try to minimise the inherent bias.
Machine learning is basically brute-force statistics on data. Or alternatively, iterated application of an algorithm on data. In and of itself it is neutral. Any apparent bias in the results will almost completely be a result of bias in the data, _not_ the design of the algorithm.
All of these (except data) are tools. What we should be worrying about is the specific applications of these tools. Which, I agree, should be subject to some sort of ethical review before widespread use.
Stop trying to drag math and algorithms into this political quagmire, and instead focus on the actual problem. We don't outlaw knives because they are still incredibly useful tools. We outlaw stabbing people.
27 comments
[ 3.7 ms ] story [ 78.7 ms ] threadIs there?
I mean, yea, I do expect it to do some optimization, in that I expect it to use some path finding algorithm with some collection of weights on the different paths, or something like that, and for the instance of the path finding problem that it produces I expect the route to be optimal, but that the problem it produces may differ in some ways from the true most correct problem to solve,
but, in the end, the solution it gives me is better than the one I would produce, so, why do I care if it isn't quite optimal under the literally perfect criteria? (I mean, if another alternative gave better results that would be better, of course. I just mean that what it is isn't bad.)
I don't think navigation is the best example for this topic though. Navigation seems like a fairly solid/well-defined problem, not like more of the more fuzzy questions, like "does this picture include a face?" .
1. You believe algorithms are biased and must be analyzed for neutrality.
2. You believe algorithms are neutral and there is no need to question them.
3. You don't have the time or the aptitude to pay attention to the algorithms that run our lives. [in effect, you behave like algorithms are neutral]
I think OP's claim is closer to 3 than 2.
> It shouldn't be a mission impossible to find out how and why a decision about you was made.
That does sound attractive. But I don't see how it squares with technology. For certain technologies, the answer will be "the weights in the neural network, when applied to your attributes, yielded a score that fell above a cutoff" or "Your data attributes mapped to a certain location on an eigenvector and it fell above a cutoff defined on that eigenvector".
Is the answer to stop using such algorithms? If not, doesn't this webpage need to give us more of a hint as to how we are to overcome the apparent impasse?
Explainability is far from a solved problem but there are certainly tools available to provide at least some transparency. I guess the website could provide references but that doesn't really seem to be the resolution being aimed for.
[0] https://www.kaggle.com/dansbecker/shap-values.
Not to mention they provide 3 proposals of improving things.
Edit: realised some may not be familiar with the word, so https://en.wikipedia.org/wiki/Puffery
[0]: my previous comment pointing that out: https://news.ycombinator.com/item?id=24629474
Math _is_ neutral.
Algorithms _are_ neutral, and are simply instruction sets for operating on data.
Data is certainly be biased, but any researcher, designer, engineer, etc. worth their salt will try to minimise the inherent bias.
Machine learning is basically brute-force statistics on data. Or alternatively, iterated application of an algorithm on data. In and of itself it is neutral. Any apparent bias in the results will almost completely be a result of bias in the data, _not_ the design of the algorithm.
All of these (except data) are tools. What we should be worrying about is the specific applications of these tools. Which, I agree, should be subject to some sort of ethical review before widespread use.
Stop trying to drag math and algorithms into this political quagmire, and instead focus on the actual problem. We don't outlaw knives because they are still incredibly useful tools. We outlaw stabbing people.