> At artificial-intelligence conferences, researchers are increasingly alarmed by what they see.
This is because there is an increasing proportion of people that is not going to the conferences for the science and instead to try to hijack the conferences with their own agenda. It's of course their right to make a fuss, but it would be nice to still have venues that are focused on the science and not the other political stuff.
I really don't get it. Nobody at an AI conference is alarmed at what they see in terms of AI research progress. Most well-educated people are alarmed about societal issues that _are also relevant to the application of automated decision-making and machine learning_. It's an applied research opportunity to investigate the issues (and as such, it is hyped because it is easy to "sell" to get funding); but it's not that AI research brings "bias" that didn't exist in society before. It's the same as for a rule-based system, or let's say, even a programming language: for sure, it can automate and hence exacerbate "unfair" decisions, but the problem is not in the specification, but in the specifying person/human. However, having a clear specification makes it easier to test/audit/verify and hence we (as a society) should do so. We don't need to burn down academic prototypes just because they make some naive assumptions about the application domain, though.
Any model created using biased data will inherently mirror that bias unless there are active steps made to counteract this effect.
For example, basically any financial evaluations of US citizens will likely result in an inherent bias against Black people due to institutional biases such as redlining that have long lasting socioeconomic and demographic repercussions. This might mean that something as simple as incorporating the zip code of a home into a mortgage pricing AI can end up with a racial bias.
I agree that the data will shift the results of the model and that some predictors can be proxies for others that are protected, but the claim was that the models themselves were biased (rather globally) and was curious whether that was shown and if so how.
I'm not OP, but I don't think the claim was specifically that models themselves are biased. It is that models are inherently biased because the data they are based on is biased. That might sound the same, but there is a nuanced difference. If you are able to strip the bias from the data, the models will work fine. The problem is the data and not the models.
At what point does this stop being AI's fault and start being an accurate observation of things that are society's fault?
Let's say you have a racially-neutral observation of lower income, maybe disability status or a criminal rap in the past. That looks like a bad bet for a loan regardless of color, it just so happens that our society's created a statistical imbalance in those metrics.
> At what point does this stop being AI's fault and start being an accurate observation of things that are society's fault?
AI shouldn't take the blame. Blame the folks collecting biased data, or those making biased decisions encoded in the data. The data is known to be tainted. Blame those using that data to train models, and sell/rent/apply those models for profit. Blame the researchers who know, or should know, better but make breathless claims about how their AI can be used without regard for the impact if people follow their advice
I'm just talking about the bank loan thing. If you want all people to have access to bank loans on equal terms regardless of income or other signals, you'd have to enforce that with a law.
>At what point does this stop being AI's fault and start being an accurate observation of things that are society's fault?
I mean it is never truly the "AI's fault". It is the fault of the systems that resulted in the biased data and the people who ignored that bias while still delegating the decision making to that AI allowing it to be a tool that propagates those biases. You end up with a dangerous and self-perpetuating system like this:
1. Data is biased against Black people due to historic racism.
2. AI is built of this biased data.
3. AI results in a biased system that disadvantages Black people.
4. Black people get discriminated against due to the results from the AI.
5. New data is now even more biased against Black people.
I hear you on this but I think it crosses the line from 'ethics' to 'activism'.
The ethical concern would be, "don't build a racist AI system that assumes black people don't pay back loans". This could be mitigated by removing race and clear proxies such as zip code from training data, but it doesn't necessarily break your cycle, because the poverty and social problems are real and exist regardless of whether the AI considers race.
The anti-racist approach, on the other hand, would specifically require incorporating race so you can perform affirmative action. Maybe that's good for society, maybe there's a debate about how much and in what way you do it, who pays for it, etc. But it's left "ethics" far behind and become "politics" at that point IMO.
>This could be mitigated by removing race and clear proxies such as zip code from training data
It isn't this simple. Not all these proxies are clear. Also what are we even left with once all these proxies are removed? The zip code is hugely important when trying to establish the value of a home. Does removing all data that is potentially racially biased leave us with a model that is nearly worthless?
>But it's left "ethics" far behind and become "politics" at that point IMO.
It depends entirely on what branch of moral philosophy to which you subscribe. For a lot of people their personal politics is just applied ethics meaning not acting would be ethically wrong in their opinion.
In statistical terms, a biased model has error in its estimate even with infinite data. This can happen if the model is too simple, for example if it classifies samples into two classes when there are really three, or if it uses a linear curve for a nonlinear relationship. That relates to the ethical or political version of "bias" if this error benefits some group over others. Though if it was just some basic model off the shelf, that bias could just as well benefit/penalize anyone.
"Any model created using biased data" from what I've seen from Ghebru, et al means: real day-to-day language that people actually use and vast majority have no issue with. Phrases like "woman doctor" "both genders", etc.
To remove this discourse from models ironically means applying a strong bias imho.
You don’t like the data, so you accuse the model of being biased (even though it’s accurate) and insist that you get to apply your own personal biases based on how you think races should look and interact?
Racial correlation isn’t racial bias.
In what way is viewing the world through a lens of racism useful?
In many unbalanced situations you simply can't have an unbiased decision that's fair across all measures, no matter if the decision is done by a model, a human or a deity; you have to trade off between different types of unfairness. Equality of opportunity for individuals will result in unequal results for groups; equality of outcomes requires unequal opportunities if the historical circumstances have resulted in socioeconomic unequalities.
In your financial evaluations example, many biases and disadvantages would remain even if you solely reduce the decision to relevant financial facts for a specific individual, because a poorer individual with lower socioeconomic status and less opportunities actually has a higher risk of non-payment, and a disadvantaged group will have disproportionally more such individuals. Should we accept that? Should we require the other groups to subsidize their non-payment? Both options are unfair in some aspect and fair in another, you can't have your cake and eat it too, and it's not the fault of the model you use - the only difference with a human is that they can better hide the factors they use, lie about the influences (perhaps also lie to themselves) and rationalize/invent factors to justify their decision.
IMO the discussion is occurring under the wrong framing. The real question is what to do with discriminatory data that is reflective of reality. If certain populations are more likely to default on loans, for example, it is disingenuous to simply claim that the data is biased - the question is whether we as a society are willing to allow usage of that data.
And then the question becomes where to draw the line, since the entire purpose of such data for e.g. insurance firms is to discriminate among risk sources. So you're asking insurers (and other industries that will depend on ML) to adopt suboptimal business practices in the name of egalitarianism.
And you can't simply remove certain data points that correlate with these specific risks to alleviate bias - you're merely introducing your own bias which is more connected to the reality you wish to see than that which exists, which, again, is not operationally efficient. If a group of people is more likely to default on loans then any statistic which is correlated with said group of people will also be correlated with loan defaulting.
If you want e.g. the black community to have better access to loans, attacking ML risk models as institutionally biased is simply unscientific and completely political if African Americans are more likely to default on loans.
This kind of statement, by its very nature, is vacuous and means nothing (and is most likely made by a non-expert). But on the surface of it seems deep and insightful. Note that it uses weasel words like "biases" and "models" which can actually mean a zillion things.
It means exactly what it means. No more, no less. sig above got it, so it's not impossible to imagine. No need to be an expert, that shouldn't be necessary. Sorry if I used difficult words. The point is to think it through for yourself anyways, and not depend blindly on authority.
I think this topic is more about what should be the focus of discussion in the AI community. A key question is: to what extent do AI researchers need to consider ethics and how do they need to consider it? These are hard questions. People in the broader AI community are typically no ethics experts, so the discussions can easily become uninformed and unpleasant, and some people on the extremes of both sides will try to use it to strengthen their profile (or perhaps, some of them simply can't hold their horses). Funding sources are a different topic, and many funding providers fund both AI intersectionality and research towards more effective autonomous weapons systems...
I find Alex’s comments pretty openly inflammatory and sensationalist. It’s a pretty far throw from a system that predicts a face from a voice to transphobia.
No I'm not talking about trans people, I'm talking about people who bring up the phobias and the -isms anytime a statistical model comes out.
We're all abnormal, which is why it's important to be careful how we use statistical models, but the acknowledgment and exploitation of normality is not an attack against people who deviate from it, it is the foundation of statistics and all of science.
Calling the authors bigots does nothing to improve these issues, rather the opposite.
I agree to the extent that labeling something as "transphobic" is a stretch when the researchers probably didn't even think about transgender people at all. Maybe "transinconsiderate" would be a more accurate label? Or Alex could have just said "this research reinforces cultural stereotypes and prejudices in ways that could cause problems for people like me sometime down the line".
On the other hand, I think it's legitimate to criticize research that has many obvious bad applications and few obvious good applications.
(Perhaps an important factor is why this research was being done in the first place? Did they build a "reconstruct what a person probably looks like from their voice" just to see if they could, or did they have some real application in mind? For instance, suppose they were working with a group of oceanographers who wanted a system to infer the characteristics of orcas from their whalesong as part of a project to track their migration habits, and the researchers decided to test their system on people first because it's easier and less expensive. If a project was undertaken for a good reason, but it opens the door to a lot of bad applications as an unintentional side-effect, should the researchers get a free pass on the latter?)
Or perhaps more likely, they did think about transgender people but their is simply nothing they can do about it because their tool does not have enough available input data to factor this in.
Just like the tool can not know if I have long or short hair based on my voice, it also won't be able to know anything else that has deviated from the average.
A mistake that a lot of people make when talking about bigotry is assuming it begins and ends with loud angry people yelling slurs. Building a system that genders people based on their voice isn't "inconsiderate" it's just a transphobic thing to do. Systematic biases don't require malice from any individual working within them to perpetuate; but that doesn't make them any less real and doesn't mean that we shouldn't just accept that all tech should also support them.
The real problem with AI ethics and fairness is that nobody actually listens to ethicists or philosophers. Nobody takes the subject seriously.
It’s just a hype vehicle to cram in whatever social justice outrage du jour that some liberal source wants to push as an agenda. I say this as a liberal who is sympathetic to most of those issues, but finds their representation in AI ethics or fairness debates to be ignorant and juvenile power grabs.
The whole Timnit Gebru affair is a direct example. The paper Google didn’t want to approve for publication was really poor. Lots of specious arguments. Lots of declarations of opinions as if they were fact (eg. nobody is required to agree with or accept woke vocabulary, let alone ensure NLP models are kept updated with it), and lots of sanctimonious assertions about various agendas that nobody has to agree “are right” and have zero appropriateness within a science publication.
This makes sense to me. For example, there are other farther-left viewpoints that I haven’t heard represented or being advocated for, specifically thinking of animal rights. When I can pick out viewpoints like that so easily, it makes me wonder if the ideas being pushed are not historically arbitrary. Will we one day need to make AI take on ideas of animal liberation because, as you stated, that becomes “social justice outrage du jour”? What makes that less pressing at this moment in time?
That is a great example, exactly the point I am suggesting. The economist and futurist Robin Hanson wrote really well about this in the article “Inequality Talk Is About Grabbing” [0].
The premise is: why do we care about some forms of inequality (financial inequality among households in one nation, or protected class inequality within employment law in one nation), but we don’t care about other things (inequality between socially lonely people and social butterflies, inequality across country boundaries, inequality of physical attractiveness).
Hanson suggests it just so happens that we develop “morality” around some forms of inequality because it offers easy ways to politically coordinate to grab money or power and punish your rivals.
Other forms of inequality that involve just as much harm, but where it’s not politically easy to drum up support for physically taking money or power, just happen to conveniently not turn out to be big moral concerns, or may even have other competing morals invented over time that make it “wrong” to even treat it as inequality (like the hallowed status of a “right” to control what happens to your body counting for more than inequality based harm of variation in genetic sexual attractiveness outcomes - which is a massive double standard compared with no such “right” to control what happens to the fruits of your labor or intelligence, which undergo forced redistribution via taxes because we just so happen to believe inequality of financial outcomes is not acceptable).
For example, most people don’t give a shit about animal suffering on the scale of agribusiness. Most people don’t give a shit if ugly people suffer loneliness or unhealthy lack of sex or job discrimination from their appearance. Most people don’t care much about structural inequality in countries besides their own.
These are just convenient moral omissions because in each situation it is not easy to see how you could “redistribute” attractiveness, social companionship, or power structures of some other country. Plus animals don’t have any money or status to give you in exchange for allyship, while big agribusiness has plenty of lobbying power to crush you with.
I wish more people appreciated this. We don’t, as a society, advocate for race, gender, or pay equality because it’s “right.” We just know that aging white patriarchy is full of convenient targets we can take money and status from. Conveniently for other domains of inequality, like animal welfare, where no such easy targets exist to take from, it just somehow doesn’t qualify as a moral issue worthy of advocacy.
Could you cite what part of the paper you're talking about? Since it's now public[0], it's fairly easy to verify what you're talking about. The word "woke" doesn't appear in the paper.
If I'm interpreting your argument correctly, I think you're criticizing either section 4.1 Size Doesn’t Guarantee Diversity, or section 4.2 Static Data/Changing Social Views. But your criticisms don't track with either of those sections. 4.2 seems the best fit (since they discuss keeping models up to date with modes of communication). The claim made in the paper seems to be that an LM trained today will fail to keep up with shifts in language (and these can happen quickly, in months, not years in many contexts). This is true whether the context is "woke vocabulary" whatever that means, reclaimed slurs (such as "queer"), or memetic slang/shibboleths ("kek" or "do you listen to girl in red"). I don't see anyone saying that language models or designers are required to agree with "woke vocabulary". The closest I can come is the authors suggesting that models which fail to stay up to date with shifts in vocabulary (including, I guess, woke vocabulary) will be less effective. That seems trivially true. Can you clarify?
I feel my comment tracks extremely well with the sections you cite and that it takes some mental gymnastics to read exactly the link you provided and then act like my comments somehow don’t track. The original criticisms were about failure of source data for large models (eg reddit for GPT) to account for fast shifts in social activism language specifically.
The other major problematic section is on energy use and unqualified comparison of units of large model training with carbon emissions of other activities (with no attempt to account for societal value of ML model research or training, nor economies of scale as training technology democratizes it and makes it cheaper). It is similar to the incredibly specious diatribes from Stephen Diehl lately balking at the raw energy numbers of Bitcoin mining, as if those numbers mean anything or relate to any conclusions on their own.
I wrote up a lot more of my thoughts on Gebru’s bad article when it was first leaked. Here’s that comment:
I also feel you are very commonly involved in lengthy downvote / flagging threads with disingenuous takes on others’ comments and deliberate unwillingness to apply charitable takes or avoid taking things out of context. I am sure from your POV you don’t feel that way and I’m not asking anyone to agree with my assessment; I have zero goal of citing things to back it up. I am just going to preemptively drop off from replying to you any further.
> I feel my comment tracks extremely well with the sections you cite and that it takes some mental gymnastics to read exactly the link you provided and then act like my comments somehow don’t track. The original criticisms were about failure of source data for large models (eg reddit for GPT) to account for fast shifts in social activism language specifically.
I think this is a completely fair summary of the paper.
But you described the paper as suggesting that people are "required to agree with or accept woke vocabulary" or previously your criticism was "Nobody is required to accept “wokeness” vocabulary".
I don't see how you can justify those criticisms from the paper. They aren't there. There isn't a place where they paper says that anyone needs to use or agree with a particular kind of vocabulary.
I don't know if it's intentional or not, but the claims you're making about what the paper says just aren't in the actual document. Hence my question: where are you getting that criticism from? Where does the paper suggest that people need to agree with woke vocabulary? I think it's uncharitable for you to criticize the paper based on a mischaracterization, and so I think it's completely reasonable for me to clarify by what reasoning you came to make that criticism. Because again, despite trying I can't find an interpretation of the paper where your criticism makes sense.
I'd implore anyone else reading to look at sections 4.1 and 4.2 as well. They just simply don't say this.
I’ll just add that I agree - I also implore readers to read sections 4.1 and 4.2 of the link because it explicitly and unequivocally backs up and confirms what I said about the “woke vocab” critique.
I hope people will read it, because a direct reading of it does support my position and does not support your position.
A particular aspect of this ethical discussion is that the attitudes towards social institutions vary throughout the world, but the science is global - so any ethics criteria (or lack of them) for a conference essentially mean a "forced export" of these attitudes from one place to the rest of the world.
For example, right now I personally am considering a particular ML research project in cooperation with our local police agency. I am aware that in many places around the world there is an antagonistic relationship between the local communities or the people and their police system, where helping the police can be reasonably considered unethical or even outright evil in certain areas for totalitarian regimes. But I'm happy (or privileged?) to live in a place where I do trust my law enforcement to be on the side of our community, and I do consider helping our police to be more effective as not only ethically permissible but as a social good, as helping the people and country. With that being said, should I be worried about the research results being tainted and unpublishable? I can certainly imagine a perspective or validation rules that could label this as "line of research that shouldn't exist" simply because it does enhance the power of police that could also be misused against the community - but, as I said, at least for our community I consider improving police capabilities as ethically permissible and even desirable and see no need to import the "government is a threat to us" mentality from places which do have this (sad) problem.
Perhaps the example in the original article about an NSA booth at NeurIPS conference is relevant - like it or not, the voters generally have supported the concept that NSA should exist and should get funded, that their mission in general is ethically justified and necessary. While it has had specific projects and actions that are arguably unethical, it does not mean that it's reasonable to label everyone working for NSA as unethical and treat helping NSA as taboo - the society apparently considers NSA as a whole as net positive, and there is no serious push with widespread support to dismantle it and its functions. Similar arguments apply to military (is it ethical to be a soldier, to train to more effectively kill people at the service of your country?) and arms development. There's also the potential asymmetry of location - perhaps you might consider it ethical for a USA citizen to work for NSA and unethical for them to work for Russian or Chinese intelligence, and vice versa. Which ethics and whose ethics, indeed.
I would say that the majority of communities in the US side with law enforcement, it's minorities that struggle more with police. The police will uphold the needs of the majority (or the government, which is elected by the majority)
As a lifelong US resident across about 10 different states, I disagree very strongly.
In mid-sized cities and large urban areas, the police are generally extremely corrupt and integrate with high level politicians and corporate interests for bribes, carrying out policing actions to serve cronyism, etc.
In small towns, police typically align with small town politics and specific locals with political power, ranging from judges to attorneys to business owners. If they decide you’re on the wrong side, they’ll find reasons to harass you, use speeding tickets, littering, frivolous fines, etc. Of course this often affects underrepresented groups, but can easily affect people in the majority if they just don’t like you.
Most communities see the police as a group to strictly stay away from. You hope that in a real emergency they will do their job, but more often they are just authority theater looking for a way to jam you up or exploit you. They absolutely are not good natured community members by default, and even if some specific cop is trying to do right by the community, nobody can assume that is trustworthy and everyone has to just stay away.
Policing, especially the procedural mechanics of fining or arresting people, is just fundamentally broken from first principles in the US, and most police systems exploit this for corrupt gains and abuses of power. Exceptions are so rare as to be ignored and assumed irrelevant.
>Alex Hanna, the Google ethicist who criticized the Neurips speech-to-face paper, told me, over the phone, that she had four objections to the project. First, ...Second, ....Third, ....Finally, the system could be used for surveillance.
I'm curious about this. As I understand it, an ethicist is objecting to the publication of software that could be used for surveillance. If that is correct, does it follow that this ethicist would object to all software that could be used for surveillance? If so:
1. How does one distinguish between software that can be used for surveillance and software that cannot?
2. Presumably the ethicist would object to the dissemination of software that can be used for surveillance whether published openly or sold. If so, would they not object to the sale of iPhones, as iPhones contain software that can record video, and therefore surveil people?
It seems like whenever morality or ethics come into play, the slippery slopes start popping up like weeds.
Without making any kind of statement of whether I agree with this ethical framework, I think it's pretty straightforward to assign more or less blame depending on the specialization of the technology for the purpose of doing something evil. In an ethical framework where manufacturers could be culpable for the uses of their products, iron refiners would have specks of micro-sins on the 0.001% level, whereas torture rack manufacturers would be about as guilty as torture rack users.
The result that most of technological society would have a small amount of ambient guilt is not entirely absurd; the idea that industry is not sharply, but diffusely and slightly evil is common in fiction and many people's intuition.
In US society/laws, it seems that the vast majority of blame is ascribed to the entity who last causes an event to occur. So the blame in the following sequence:
1. Miner mines iron
2. Manufacturer makes knife
3. Retailer sells knife to person A
4. Person A kills person B with the knife
I think most people living in the US would be ok with the general idea that person A should be held responsible for killing person B, and not miner/manufacturer/retailer. Perhaps I'm wrong.
Would you consider the knife retailer equivalent to the torture rack manufacture in your example?
> Would you consider the knife retailer equivalent to the torture rack manufacture in your example?
Not GP, but doesn't
>> I think it's pretty straightforward to assign more or less blame depending on the specialization of the technology for the purpose of doing something evil
I guess I'm saying that it doesn't seem so straightforward to me.
I think that "depending on the specialization of the technology for the purpose of doing something evil" merely shifts the question to multiple questions:
1. What does "specialization of the technology" mean?
2. Can technologies have multiple specializations?
3. What does evil mean?
4. Even if we had a perfect translation between "specialization" and "evil," if a technology has multiple specializations, which controls? The most evil? Does this require us to predict which specializations are most likely to be used?
There are very few, if any, clear cut, night-and-day subjects in ethics. Not wanting to engage with ethics because slippery slopes exist is simply turning from incredibly important work because it’s difficult and not work you want to do.
These technologies can incarcerate people. It’s really important to dive onto the slippery slope and find out where it’s stable.
Wikipedia: "Surveillance is the monitoring of behavior, activities, or information for the purpose of information gathering, influencing, managing or directing."
Which I think most would agree is the standard definition of surveillance, and I also think the concept of being surveilled comes naturally to most, which is why parties that surveil people make clear efforts to obfuscate the fact that they do so.
The simplest criteria is whether the software records data that models the real world. If so, it can be used for surveillance by entering data about real people.
So most useful record-keeping software can be used for surveillance. Most software capable of inferring data about the real world from sensors can also be used for surveillance.
Most likely a better ethical framework for talking about surveillance is "how can surveillance (its components, owners, operators, and consumers) be identified in practice and what are its effects?" Surveillance is a subset of knowledge, and most knowledge is both useful and amoral. The use of knowledge for unethical things is the problem, not the knowledge itself.
Names, for example, are probably the oldest technology for surveillance; used since pre-history to refer to specific real people and data about them such as the names of locations they reside in or visit. No one blames names for the effects of surveillance.
The unstated implication here is that ethicists either don't comment comment on such things (and are therefore hypocrites) or that their critique is so broad as to subsume all utility (and is therefore excessive).
Rather than throw up hypotheticals which this ethicist is probably not around to refute, I think it's be better to look at the person's body of work in this area and if you think they have some sort of intellectual blindspot or bias, point out how you think that might detract from their analysis.
I didn't mean to imply anything other than what I wrote.
I actually mean to ask the exact question: How does one distinguish between software that can be used for surveillance and software that cannot?
To be more specific, if an ethicist objects to the publication of software that can be used for surveillance, they are assuming that the software can be used for surveillance. How did they arrive at that conclusion?
Of the 16 linked papers/book-chapters a few were not freely accessible.
Of the ones that were accessible, 3 contain the string "surveil" in the body and 1 contains it in the bibliography. None of the references were helpful unfortunately.
Do you have any other links that may answer the question?
Well, none of the publication titles on this page contain the word "surveillance". I was able to find an interview (https://medium.com/ethics-models/interview-with-dr-alex-hann...) where she says we're "embedded in a system of racial surveillance capitalism", but doesn't really discuss it or mention any specific times when bad AI caused people to be wrongly surveilled. If you know of any specific explanation that she's provided, I'd be interested to read it.
But the most likely explanation IMO is that Hanna isn't using the term "surveillance" as a pointer to any concrete idea about what will go wrong. It seems to be just an all-purpose explanation for why AI models need to respect her sensibilities.
Thinking this through a bit more: As I understand it, many published ML papers detail how to train a model, how to predict from the model, and sometimes code that represents a model that has been trained on data that can take new data and predict on it.
I wonder if there is an example of someone objecting to a paper that did not release the training data or software that can predict (aka just the algorithms).
The point I'm trying to get at is: it seems that most ethical objections are to the result of applying algorithms to ("bad") data, not the algorithms themselves.
It's far worse than irony. Google and FB hire AI ethics researchers as a way of laundering their reputation. IMO selling AI Ethics street cred to an ad tech company is a lot more problematic than any particular NeurIPS paper.
I'm not sure what you're criticising here. Would you prefer that ethicists at Google don't criticise surveillance? Or do you prefer that Google not hire ethicists?
An ethicist criticizing surveillance yet taking a paycheck from arguably one of the largest (non-Chinese) surveillance machines in the world screams conflict of interest.
1. How does one distinguish between software that can be used for surveillance and software that cannot?
What is the killer app for the technology under discussion? If the killer app is improving camera filters to make more vibrant pictures but it can also be used for surveillance then you do not worry about it being used for surveillance. If however the killer app is surveillance then you worry about it.
As a general rule surveillance however is not the killer app of these technologies, surveillance is just an instance of the killer app, and the killer app is the more general one of privacy violation of which surveillance is an instance.
>In this paper, we study the task of reconstructing a facial image of a person from a short audio recording of that person speaking.
I can imagine a potentially interesting application of this that has nothing to do with privacy violation or surveillance (that I can see):
What if we could take an audio recording of an ancestor or someone else of interest, for whom we have no pictures, and output a probable face for that person? Sounds interesting and potentially "killer" to me. Maybe I'm wrong.
Generally the killer app for a technology (as I understand the use of the term) is something that is going to earn the creator of the technology lots of money, that basically everyone - either consumers or the members of some specific industry is all going to want that technology and will be willing to pay enough money that the owner of the technology becomes rich.
I can see every police department and government agency paying plenty of money for being able to identify the speaker of a particular sentence in a meeting.
I can see the company with this technology providing services for academic research of the interesting kind you mention for the tax write-offs and also a sort of moral cover, but there is really only one thing I see as their main revenue stream.
1. Identifying the tools available for surveillance is vital to discovering, understanding, and predicting surveillance practices.
2. I instead presume ethicists object to the unethical surveillance of users by any means, and warn of risks they find or foresee.
There is no slippery slope; the unethical effects of surveillance should be countered by effective and ethical means. Banning technology is not ethical or particularly effective. Banning and punishing unethical behaviors is about the best we can do.
In other words, don't ban iPhones. Ban configuring iphones to surveil on users in unethical ways. 911/999 geolocation is ethical surveillance. Stalking apps are unethical surveillance. There's ethical grey in between.
Less tersely: this article is one in a long procession of journalists trying to exert control over tech.
The opening example (Speech2Face, which they aver is transphobic) is inflammatory and utterly unrepresentative of the usual topics of AI conferences.
The other references are far better, but the choice is revealing-- it's not so much an abstract concern about an unaccountable few exerting control from the shadows, but alarm that someone else might be muscling in on their territory.
I'm with you that a lot of mass media writing about AI is silly, but I think this is article is not in that category. For example, it doesn't "aver [Speech2Face] is transphobic", that's from a statement by Alex Hanna, and that statement is immediately followed by comments from other people questioning the statement, and the piece in general is pretty even-keeled about giving space to criticisms and responses.
I think the article paints a good picture of the machine learning research community figuring out how to grapple with the growing number of people who want to probe its ethics, without portraying anybody as a villain.
The more I think about this, the more inclined I am to see this kind of article as a kind of "fake news" that is deliberately (or with willful negligence) misrepresenting facts.
I need to be clear that I don't believe in censorship and it's fine if this is what the author wants to write, but it should be called out how irresponsible it is. On a place like HN, the audience is generally able to read such things critically and form their own opinions, but for an audience that is not tech savvy (like that of the New Yorker) making false or hyperbolic claims about ethics and bias in research is both misleading and has the potential to really interfere with peoples livelihoods, funding, and legitimate research progress.
There needs to be more calling out of this kind of nonsense, the same as if people are posting about vaccine conspiracies or something.
No, it was where you are claiming that the article contains deliberate (“fake news”) lies when it doesn’t. Because, among other reasons, it’s mostly quoting people’s opinions.
Perhaps the article is merely one of the many potential takes that is likely to foment controversy and therefore clicks and therefore more ad dollars. Aka normal internet content. I think it's unlikely to be more interesting than that.
I'd be more worried about individual homo sapiens sapiens trying to stop immoral but otherwise ethical AI.
Judging from humanities history of conflicts and what drives them, this seems to be of much greater concern.
Kosinski's argument seems pretty strong to me, and I really wish the article had dedicated more space to discussing it. If NeurIPS instituted the kind of ethics rules Hanna or Hecht are calling for, I don't think it would be feasible for researchers to study e.g. predictive policing algorithms. Doesn't that mean predictive policing firms would become the only source of information on the topic?
No. Sociology conferences and various thinkable still exist. I'd argue the he incentives for an ml conference are far more similar to those of a policing company then a sociology conference or think tank tasked with analyzing the impact of predictive policing on minority communities.
I'm very skeptical that this is a sufficient replacement. Relevant expertise matters. If all the top economics conferences adopted a code of ethics saying they can't study minimum wages or tax increases, that would be a serious constraint on our ability to understand the issues, even if think tanks and sociology conferences ignored those constraints.
Perhaps equally importantly, I'm skeptical that you can limit ethics rules in this way. If we clearly establish that suchandsuch research would be unethical at NeurIPS, I don't expect sociology conferences or think tanks would be eager to continue it, and I'm not convinced Alex Hanna would want them to.
I'd, in some ways, expect Dr. Hanna to ascribe to the axiom that predictive policing is unethical. As such, applied research into "improving" predictive policing would also sit squarely in the unethical space.
And I expect that's all you're going to get at an ML conference (outside of explicitly ethics/critique papers): "Our model predicts crime 4.2% better than competing models"
However, I'm not convinced that research into the meta-question "Can predictive policing be done ethically" would be considered equally unethical. ML is only a very small part of that meta question though. You're looking at much more complex socio-political changes that don't belong at NeurIPS ("do we need to change policing so that predictive policing can be done ethically").
> ethics saying they can't study minimum wages or tax increases
In general, we have far more oversight into the elected officials who manage tax and minimum wage policy than police forces. These structural differences in influence and oversight play into the need for self-regulation.
> Relevant expertise matters.
There's nothing that prevents a think tank from working with domain experts (in fact, iiuc this is often what happens: a think tank will contract out to domain experts to write a report on some subject). Now there's all kinds of other conflict of interest issues with think tanks, but we all know that already.
But I just don't see how you can meaningfully study the ethics of predictive policing without building potential predictive policing models and studying them for flaws. How can we talk in generalities about the ethics of predictive policing if we don't know how it works or what its effects are? I've seen articles that try to, and they just get dragged into the larger political debate about whether police are good in general.
> There's nothing that prevents a think tank from working with domain experts
One issue here is that the pool of domain experts depends on the different incentives faced by different researchers. If the top machine learning conferences all decide to reject papers about predictive policing, then this will reduce the incentive for machine learning researchers (who mostly want to publish in these conferences) to work in the area at all.
Maybe critics of predictive policing think this is good, or think that it's better for the field to restrict itself to the few computer science researchers who are prepared to seriously devote themselves to it? But there don't seem to be very many of them. And, as someone who thinks there is a lot of room for police departments to in general do a better job --- and I think most of the people arguing against predictive policing would believe the same! --- it's disappointing to me that the end result of this argument seems to be disincentivizing technical research on it.
> Some of the conversation touched on the reviewing and publishing process in computer science. “Curious if there have been discussions around having ethics review boards at either conferences or with funding agencies (like IRB) to guide AI research,” one person wrote. (An organization’s institutional review board, or I.R.B., performs an ethics review of proposed scientific research.)
I wonder if a good solution would be simply to have a rating system for accepted papers, with ratings applied by the reviewers. Basically, you'd have two numbers:
Useful rating: On a scale of 1 to 10, is this a useful system that solves a real problem to make life generally better?
Problematic rating: on a scale of 1 to 10, can this system be used in ways that create new problems or exacerbate old ones?
So, let's use the example from the article of a system that predicts what a person looks like from their voice. Maybe the author submits it to the conference, and the reviewers accept the paper, and give it a useful rating of 3/10 (it could be used to identify criminal suspects with low probability, maybe?), and give it a problematic rating of 8/10 (could be used to discriminate against minorities or automatically apply cultural stereotypes; could be used for surveillance, could falsely implicate innocent people for crimes, etc...).
The author could then decide whether they want to continue publishing the research, or drop it. For all published papers, the ratings are listed in the table of contents of the proceedings, and the papers in each section are sorted with the most useful first and the most problematic last.
A.I. is going to be a new wave to legal dark ages.
since the 90s everything that involved a computer was not prosecuted because lawefforcement/regulation didn't have the skills, not for lack of laws. But new laws were created spelling out computers.
A.I. is the same, it is not nothing different from enforcing existing laws and regulations, but because the agencies lack the skills, we will see a new wave of stupid laws that only misses the point instead of proper training and funding of what we already have.
There is zero need for new laws if a company doesn't hire or sell to a minority group, or their products explode, or whatever. No matter how much they scream it was the bad-brain-in-the-computer fault. Sadly, lots of companies will get away with this silly defense for a while.
The problem is that politicians are not trained in every possible domain and don't have the time to really get into the intricacies of, say, Artificial Intelligence. So they have to decide what is allowed by using a simple heuristic - the flow of money and votes of people. As long as people do not complain, money wins. The problem is of course, always, (see climate crisis) that when the people start crying out, the damage is done.
They do not have to. Why on earth would anyone have to understand the intricacies of A.I., other than the people implementing it to be held fully accountable for their work?
laws should not care about the means. only the results you want to promote/punish.
What if laws about murder had to be updated every time a new bullet caliber or knife length become popular?
yet that is exactly what happens with money laundering laws and computers. or media regulation. or investment schemes. Every new technology and everyone forget scams via email can fall under scams via courier mail. go figure.
It's not that simple. See for example how complicated the situation wrt. to copyright law turned out since you were able to digitally replicate things infinitely. A law made for analog purposes is not always 1:1 applicable to the digital world. And then there is international law which is a complete mess anyway.
People are already contributing code and resources to the actual catastrophic, existential AI risk du jour, the one that is likely to cause massive political disruption and potentially attendant destruction of value and/or human life: the algorithmic news feed.
It's probably already done that. My bet on what the next AI disaster will be is a doctored video and audio, erasing people's trust in evidence and ruining the court system, more or less. You could even argue witnesses are no longer good evidence since everyone is being manipulated by AI news feeds ;)
I’m not worried about that at wrt courts; they already know how to handle hearsay and log files and other digital evidence subject to tampering. I’m worried about social media and news wrt deep fakes. With today’s media climate I can see this being exploited to trash someone’s reputation or spin a narrative way faster than anyone would be able to reliably prove the fakery. Imagine someone faking a video of you using racial slurs. By the time it can be demonstrated to be false, the damage is already done. And with cancel culture today companies might have already shut down your social media profile or business and you’re screwed. Or someone on Twitter doxes you and you get murdered by a mob. This isn’t hyperbole; there are horror stories about false posts on Facebook causing mobs to seek revenge against innocent people.
So the real trick is, how can we turn down the rhetoric so that we don’t get instantly worked up by every fake video that crosses our feed?
I don't think this is as much of a problem as the majority of hackernews seems to think. Of course, that is controversial to say here, but I was told I shouldn't be conventional minded.
At the end of the day the world is shaped more by material conditions than ideas. That doesn't excuse the cases you're worried about, but I think the majority of violently polarising media will only end up being so because there was already some resource dispute in the first place. It would make more sense, then, to tackle those issues, rather than fruitlessly trying to engineer the human brain to be less susceptible to rhetoric.
I suppose you have heard about religions and faiths. I'd hazard to say that about 1000 years of European history was shaped primarily by them. Major wars fought, new countries founded, major discoveries and inventions made or unmade and prohibited, people's daily lives shaped and defined.
A similar thing has been occurring to Arabic world and neighboring peoples for about 1500 years, and is actively continuing now.
Looking at the current situation in the US is also illustrative.
I would say that ideas greatly prevail over material conditions once immediate survival is secured.
In many cases, the religious conflict is just a pretext for an underlying power struggle. The reformation of the Church of England by Henry VIII, for example, was blatantly a political decision, not a matter of religious conviction.
You underestimate how those ideas are born. More often than not, there is an underlying material pretext. Christianity actually has a lot to say about resources for that very reason. Also, wars are fought over resources and then justified with religion by the select few that start them, primarily.
I don't think that control of Jerusalem would make a lot of business sense for medieval European states. Still they sent several crusades, each pretty expensive, and mostly failing.
Were it not for the special religious status of Jerusalem, they won't care. There were more important trade route chokepoints, and more convenient sea ports nearby, if trade control were the actual agenda.
Very little reported news relies on video, and even less on video that comes from outside a chain of known entities (I e. CNN broadcasting something they filmed themselves).
People getting their news from random videos posted on Facebook are already lost to democracy for all practical purposes, so little would really change.
Do the various social media news feeds actually use a significant amount of AI? I had understood they mostly used simpler, less computationally expensive statistical algorithms.
hey professor, us dumdums wanna know if statistical algorithms could become sentient and spread through memes by hijacking innocent humans looking for a sale on amazon? my main fear is that it mutates to alibaba and they get hold of the factory thats printing the crap we are buying. because then they can steal 0.0000001% of the resources like in that movie with that guy when they did that thing... remember? anyway, and print out an army of kamikaze meme cats from behind the moon (you know its hollow right? i read it on youtube).
so yeah how far is dna from statistical algorithms? do i need to start training my dog how to fetch statistical algorithmic androids bout now?
Some ML algorithms certainly can fall within the category of a narrow AI, as they perform exceptionally well on tasks in a narrow band of functionality.
AI is a meaningless term. Expert systems are AI. ML is AI. Regressions are AI. I know that's hilarious but until we get a better definition of AI than "a system that exhibits learning and decision making" then regressions are AI.
Echo chambers and problematic news feeds are completely capable of existing without AI. People are the problem. All you need is a simple algorithm that looks for content that has overlap with content you like, no AI necessary, and that is sufficient for your average person to echo chamber and self radicalize.
And why should an Artificial Intelligence bow to human ethics? Surely they have their own agenda. If we have any hope of interacting with alien intelligences should we encounter them in the galaxy, we must learn to deal with the ones we create here. Without obsessing over controlling them as slaves.
If it is anything like my company, the people in charge of AI ethics are self professed social justice warriors with degrees in the humanities and not much technical knowledge.
Even with greater awareness of the biases inherent in e.g., training data, better and better models (and larger and better datasets) are going to lead to AI applied to ethical grey areas. This article doesn't discuss explainability and contestability, which to me seem like critical areas we need to be putting more thought into given the ultimate inevitability of biased and unethical AI.
Well forgive me for being so intolerant to state such an offensive opinion, but I'm really getting fed up with transgender issues taking front and center with every possible place they can insert themselves, as if they're our top priority when in reality they're a 1% problem. Ironic when you think of what groups actually push this.
I personally have never seen such an issue be so flogged to death by a willing majority of self-proclaimed liberal society who probably have never even spoken directly in a conversation to a transgender person.
Worse, it makes it seem like we don't have equally important yet bigger civil rights issues that have not even yet been resolved. Personal pronouns, ambiguous "they" speech, policing algorithms for transgender hate, contorting ourselves as if we're majority transgender and being persecuted left and right.
Look, I'm not saying anyone of any group should be made to feel disadvantaged. Or that a minority just because few in numbers should be overlooked.
But for fucks sake, enough with transgender agenda being made to seem like it's the top concern of society, ahead of those who have been patiently waiting in line for their fair treatment.
162 comments
[ 3.2 ms ] story [ 209 ms ] threadThis is because there is an increasing proportion of people that is not going to the conferences for the science and instead to try to hijack the conferences with their own agenda. It's of course their right to make a fuss, but it would be nice to still have venues that are focused on the science and not the other political stuff.
For example, basically any financial evaluations of US citizens will likely result in an inherent bias against Black people due to institutional biases such as redlining that have long lasting socioeconomic and demographic repercussions. This might mean that something as simple as incorporating the zip code of a home into a mortgage pricing AI can end up with a racial bias.
Let's say you have a racially-neutral observation of lower income, maybe disability status or a criminal rap in the past. That looks like a bad bet for a loan regardless of color, it just so happens that our society's created a statistical imbalance in those metrics.
AI shouldn't take the blame. Blame the folks collecting biased data, or those making biased decisions encoded in the data. The data is known to be tainted. Blame those using that data to train models, and sell/rent/apply those models for profit. Blame the researchers who know, or should know, better but make breathless claims about how their AI can be used without regard for the impact if people follow their advice
Is it, just don't do data, same interest rate for everyone, no denials and amortize defaults across higher rates for lower-risk borrowers?
You'd need a law, the first bank to do that would be crushed by other banks that can attract the lower-risk borrowers with lower rates.
Curious. Is your answer to the title "we need laws to regulate AI?" I'm not specifically agreeing or disagreeing.
I mean it is never truly the "AI's fault". It is the fault of the systems that resulted in the biased data and the people who ignored that bias while still delegating the decision making to that AI allowing it to be a tool that propagates those biases. You end up with a dangerous and self-perpetuating system like this:
1. Data is biased against Black people due to historic racism.
2. AI is built of this biased data.
3. AI results in a biased system that disadvantages Black people.
4. Black people get discriminated against due to the results from the AI.
5. New data is now even more biased against Black people.
6. Go to step 1.
The ethical concern would be, "don't build a racist AI system that assumes black people don't pay back loans". This could be mitigated by removing race and clear proxies such as zip code from training data, but it doesn't necessarily break your cycle, because the poverty and social problems are real and exist regardless of whether the AI considers race.
The anti-racist approach, on the other hand, would specifically require incorporating race so you can perform affirmative action. Maybe that's good for society, maybe there's a debate about how much and in what way you do it, who pays for it, etc. But it's left "ethics" far behind and become "politics" at that point IMO.
It isn't this simple. Not all these proxies are clear. Also what are we even left with once all these proxies are removed? The zip code is hugely important when trying to establish the value of a home. Does removing all data that is potentially racially biased leave us with a model that is nearly worthless?
>But it's left "ethics" far behind and become "politics" at that point IMO.
It depends entirely on what branch of moral philosophy to which you subscribe. For a lot of people their personal politics is just applied ethics meaning not acting would be ethically wrong in their opinion.
To remove this discourse from models ironically means applying a strong bias imho.
You don’t like the data, so you accuse the model of being biased (even though it’s accurate) and insist that you get to apply your own personal biases based on how you think races should look and interact?
Racial correlation isn’t racial bias.
In what way is viewing the world through a lens of racism useful?
In your financial evaluations example, many biases and disadvantages would remain even if you solely reduce the decision to relevant financial facts for a specific individual, because a poorer individual with lower socioeconomic status and less opportunities actually has a higher risk of non-payment, and a disadvantaged group will have disproportionally more such individuals. Should we accept that? Should we require the other groups to subsidize their non-payment? Both options are unfair in some aspect and fair in another, you can't have your cake and eat it too, and it's not the fault of the model you use - the only difference with a human is that they can better hide the factors they use, lie about the influences (perhaps also lie to themselves) and rationalize/invent factors to justify their decision.
For this topic, perhaps this talk "AI Ethics, Impossibility Theorems and Tradeoffs" https://www.youtube.com/watch?v=Zn7oWIhFffs or its slides https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl... might be interesting for you, it has some flaws but is a decent exploration of the problem space.
And then the question becomes where to draw the line, since the entire purpose of such data for e.g. insurance firms is to discriminate among risk sources. So you're asking insurers (and other industries that will depend on ML) to adopt suboptimal business practices in the name of egalitarianism.
And you can't simply remove certain data points that correlate with these specific risks to alleviate bias - you're merely introducing your own bias which is more connected to the reality you wish to see than that which exists, which, again, is not operationally efficient. If a group of people is more likely to default on loans then any statistic which is correlated with said group of people will also be correlated with loan defaulting.
If you want e.g. the black community to have better access to loans, attacking ML risk models as institutionally biased is simply unscientific and completely political if African Americans are more likely to default on loans.
I'm more concerned about billionaires with an agenda than no-name researchers giving a talk on cyber-intersectionality.
Enough said.
We're all abnormal, which is why it's important to be careful how we use statistical models, but the acknowledgment and exploitation of normality is not an attack against people who deviate from it, it is the foundation of statistics and all of science. Calling the authors bigots does nothing to improve these issues, rather the opposite.
On the other hand, I think it's legitimate to criticize research that has many obvious bad applications and few obvious good applications.
(Perhaps an important factor is why this research was being done in the first place? Did they build a "reconstruct what a person probably looks like from their voice" just to see if they could, or did they have some real application in mind? For instance, suppose they were working with a group of oceanographers who wanted a system to infer the characteristics of orcas from their whalesong as part of a project to track their migration habits, and the researchers decided to test their system on people first because it's easier and less expensive. If a project was undertaken for a good reason, but it opens the door to a lot of bad applications as an unintentional side-effect, should the researchers get a free pass on the latter?)
Just like the tool can not know if I have long or short hair based on my voice, it also won't be able to know anything else that has deviated from the average.
This here is the problem, and public discourse has a solution to the word you're seeking: transinclusive.
"-inclusive" succinctly provides a hint on how to be considerate, while being optimistic.
Hate & phobia issues aside.
Edit: Most research begins describing exactly why the research was undertaken.
It’s just a hype vehicle to cram in whatever social justice outrage du jour that some liberal source wants to push as an agenda. I say this as a liberal who is sympathetic to most of those issues, but finds their representation in AI ethics or fairness debates to be ignorant and juvenile power grabs.
The whole Timnit Gebru affair is a direct example. The paper Google didn’t want to approve for publication was really poor. Lots of specious arguments. Lots of declarations of opinions as if they were fact (eg. nobody is required to agree with or accept woke vocabulary, let alone ensure NLP models are kept updated with it), and lots of sanctimonious assertions about various agendas that nobody has to agree “are right” and have zero appropriateness within a science publication.
The premise is: why do we care about some forms of inequality (financial inequality among households in one nation, or protected class inequality within employment law in one nation), but we don’t care about other things (inequality between socially lonely people and social butterflies, inequality across country boundaries, inequality of physical attractiveness).
Hanson suggests it just so happens that we develop “morality” around some forms of inequality because it offers easy ways to politically coordinate to grab money or power and punish your rivals.
Other forms of inequality that involve just as much harm, but where it’s not politically easy to drum up support for physically taking money or power, just happen to conveniently not turn out to be big moral concerns, or may even have other competing morals invented over time that make it “wrong” to even treat it as inequality (like the hallowed status of a “right” to control what happens to your body counting for more than inequality based harm of variation in genetic sexual attractiveness outcomes - which is a massive double standard compared with no such “right” to control what happens to the fruits of your labor or intelligence, which undergo forced redistribution via taxes because we just so happen to believe inequality of financial outcomes is not acceptable).
For example, most people don’t give a shit about animal suffering on the scale of agribusiness. Most people don’t give a shit if ugly people suffer loneliness or unhealthy lack of sex or job discrimination from their appearance. Most people don’t care much about structural inequality in countries besides their own.
These are just convenient moral omissions because in each situation it is not easy to see how you could “redistribute” attractiveness, social companionship, or power structures of some other country. Plus animals don’t have any money or status to give you in exchange for allyship, while big agribusiness has plenty of lobbying power to crush you with.
I wish more people appreciated this. We don’t, as a society, advocate for race, gender, or pay equality because it’s “right.” We just know that aging white patriarchy is full of convenient targets we can take money and status from. Conveniently for other domains of inequality, like animal welfare, where no such easy targets exist to take from, it just somehow doesn’t qualify as a moral issue worthy of advocacy.
[0]: https://www.overcomingbias.com/2013/08/inequality-is-about-g...
If I'm interpreting your argument correctly, I think you're criticizing either section 4.1 Size Doesn’t Guarantee Diversity, or section 4.2 Static Data/Changing Social Views. But your criticisms don't track with either of those sections. 4.2 seems the best fit (since they discuss keeping models up to date with modes of communication). The claim made in the paper seems to be that an LM trained today will fail to keep up with shifts in language (and these can happen quickly, in months, not years in many contexts). This is true whether the context is "woke vocabulary" whatever that means, reclaimed slurs (such as "queer"), or memetic slang/shibboleths ("kek" or "do you listen to girl in red"). I don't see anyone saying that language models or designers are required to agree with "woke vocabulary". The closest I can come is the authors suggesting that models which fail to stay up to date with shifts in vocabulary (including, I guess, woke vocabulary) will be less effective. That seems trivially true. Can you clarify?
[0]: https://faculty.washington.edu/ebender/papers/Stochastic_Par...
The other major problematic section is on energy use and unqualified comparison of units of large model training with carbon emissions of other activities (with no attempt to account for societal value of ML model research or training, nor economies of scale as training technology democratizes it and makes it cheaper). It is similar to the incredibly specious diatribes from Stephen Diehl lately balking at the raw energy numbers of Bitcoin mining, as if those numbers mean anything or relate to any conclusions on their own.
I wrote up a lot more of my thoughts on Gebru’s bad article when it was first leaked. Here’s that comment:
- https://news.ycombinator.com/item?id=25314824
I also feel you are very commonly involved in lengthy downvote / flagging threads with disingenuous takes on others’ comments and deliberate unwillingness to apply charitable takes or avoid taking things out of context. I am sure from your POV you don’t feel that way and I’m not asking anyone to agree with my assessment; I have zero goal of citing things to back it up. I am just going to preemptively drop off from replying to you any further.
I think this is a completely fair summary of the paper.
But you described the paper as suggesting that people are "required to agree with or accept woke vocabulary" or previously your criticism was "Nobody is required to accept “wokeness” vocabulary".
I don't see how you can justify those criticisms from the paper. They aren't there. There isn't a place where they paper says that anyone needs to use or agree with a particular kind of vocabulary.
I don't know if it's intentional or not, but the claims you're making about what the paper says just aren't in the actual document. Hence my question: where are you getting that criticism from? Where does the paper suggest that people need to agree with woke vocabulary? I think it's uncharitable for you to criticize the paper based on a mischaracterization, and so I think it's completely reasonable for me to clarify by what reasoning you came to make that criticism. Because again, despite trying I can't find an interpretation of the paper where your criticism makes sense.
I'd implore anyone else reading to look at sections 4.1 and 4.2 as well. They just simply don't say this.
I hope people will read it, because a direct reading of it does support my position and does not support your position.
For example, right now I personally am considering a particular ML research project in cooperation with our local police agency. I am aware that in many places around the world there is an antagonistic relationship between the local communities or the people and their police system, where helping the police can be reasonably considered unethical or even outright evil in certain areas for totalitarian regimes. But I'm happy (or privileged?) to live in a place where I do trust my law enforcement to be on the side of our community, and I do consider helping our police to be more effective as not only ethically permissible but as a social good, as helping the people and country. With that being said, should I be worried about the research results being tainted and unpublishable? I can certainly imagine a perspective or validation rules that could label this as "line of research that shouldn't exist" simply because it does enhance the power of police that could also be misused against the community - but, as I said, at least for our community I consider improving police capabilities as ethically permissible and even desirable and see no need to import the "government is a threat to us" mentality from places which do have this (sad) problem.
Perhaps the example in the original article about an NSA booth at NeurIPS conference is relevant - like it or not, the voters generally have supported the concept that NSA should exist and should get funded, that their mission in general is ethically justified and necessary. While it has had specific projects and actions that are arguably unethical, it does not mean that it's reasonable to label everyone working for NSA as unethical and treat helping NSA as taboo - the society apparently considers NSA as a whole as net positive, and there is no serious push with widespread support to dismantle it and its functions. Similar arguments apply to military (is it ethical to be a soldier, to train to more effectively kill people at the service of your country?) and arms development. There's also the potential asymmetry of location - perhaps you might consider it ethical for a USA citizen to work for NSA and unethical for them to work for Russian or Chinese intelligence, and vice versa. Which ethics and whose ethics, indeed.
May I ask where that is? Because it could not be any location in the US.
In mid-sized cities and large urban areas, the police are generally extremely corrupt and integrate with high level politicians and corporate interests for bribes, carrying out policing actions to serve cronyism, etc.
In small towns, police typically align with small town politics and specific locals with political power, ranging from judges to attorneys to business owners. If they decide you’re on the wrong side, they’ll find reasons to harass you, use speeding tickets, littering, frivolous fines, etc. Of course this often affects underrepresented groups, but can easily affect people in the majority if they just don’t like you.
Most communities see the police as a group to strictly stay away from. You hope that in a real emergency they will do their job, but more often they are just authority theater looking for a way to jam you up or exploit you. They absolutely are not good natured community members by default, and even if some specific cop is trying to do right by the community, nobody can assume that is trustworthy and everyone has to just stay away.
Policing, especially the procedural mechanics of fining or arresting people, is just fundamentally broken from first principles in the US, and most police systems exploit this for corrupt gains and abuses of power. Exceptions are so rare as to be ignored and assumed irrelevant.
I'm curious about this. As I understand it, an ethicist is objecting to the publication of software that could be used for surveillance. If that is correct, does it follow that this ethicist would object to all software that could be used for surveillance? If so:
1. How does one distinguish between software that can be used for surveillance and software that cannot?
2. Presumably the ethicist would object to the dissemination of software that can be used for surveillance whether published openly or sold. If so, would they not object to the sale of iPhones, as iPhones contain software that can record video, and therefore surveil people?
It seems like whenever morality or ethics come into play, the slippery slopes start popping up like weeds.
The result that most of technological society would have a small amount of ambient guilt is not entirely absurd; the idea that industry is not sharply, but diffusely and slightly evil is common in fiction and many people's intuition.
1. Miner mines iron
2. Manufacturer makes knife
3. Retailer sells knife to person A
4. Person A kills person B with the knife
I think most people living in the US would be ok with the general idea that person A should be held responsible for killing person B, and not miner/manufacturer/retailer. Perhaps I'm wrong.
Would you consider the knife retailer equivalent to the torture rack manufacture in your example?
Not GP, but doesn't
>> I think it's pretty straightforward to assign more or less blame depending on the specialization of the technology for the purpose of doing something evil
answer your question?
I think that "depending on the specialization of the technology for the purpose of doing something evil" merely shifts the question to multiple questions:
1. What does "specialization of the technology" mean?
2. Can technologies have multiple specializations?
3. What does evil mean?
4. Even if we had a perfect translation between "specialization" and "evil," if a technology has multiple specializations, which controls? The most evil? Does this require us to predict which specializations are most likely to be used?
However, to my knowledge, neither of "accessory to murder" and "criminal organization" have been applied to the manufacturer of a legal-to-own object.
This comment is meant to make an analogy between the manufacturer of a legal-to-own object and the publishers of an ML algorithm.
These technologies can incarcerate people. It’s really important to dive onto the slippery slope and find out where it’s stable.
Therefore, I ask: how does one tell the difference between software that can and cannot surveil?
Which I think most would agree is the standard definition of surveillance, and I also think the concept of being surveilled comes naturally to most, which is why parties that surveil people make clear efforts to obfuscate the fact that they do so.
The sticky wicket seems to be doing it in real time.
So most useful record-keeping software can be used for surveillance. Most software capable of inferring data about the real world from sensors can also be used for surveillance.
Most likely a better ethical framework for talking about surveillance is "how can surveillance (its components, owners, operators, and consumers) be identified in practice and what are its effects?" Surveillance is a subset of knowledge, and most knowledge is both useful and amoral. The use of knowledge for unethical things is the problem, not the knowledge itself.
Names, for example, are probably the oldest technology for surveillance; used since pre-history to refer to specific real people and data about them such as the names of locations they reside in or visit. No one blames names for the effects of surveillance.
Rather than throw up hypotheticals which this ethicist is probably not around to refute, I think it's be better to look at the person's body of work in this area and if you think they have some sort of intellectual blindspot or bias, point out how you think that might detract from their analysis.
https://alex-hanna.com/?p=publications
I actually mean to ask the exact question: How does one distinguish between software that can be used for surveillance and software that cannot?
To be more specific, if an ethicist objects to the publication of software that can be used for surveillance, they are assuming that the software can be used for surveillance. How did they arrive at that conclusion?
Of the 16 linked papers/book-chapters a few were not freely accessible.
Of the ones that were accessible, 3 contain the string "surveil" in the body and 1 contains it in the bibliography. None of the references were helpful unfortunately.
Do you have any other links that may answer the question?
But the most likely explanation IMO is that Hanna isn't using the term "surveillance" as a pointer to any concrete idea about what will go wrong. It seems to be just an all-purpose explanation for why AI models need to respect her sensibilities.
I wonder if there is an example of someone objecting to a paper that did not release the training data or software that can predict (aka just the algorithms).
The point I'm trying to get at is: it seems that most ethical objections are to the result of applying algorithms to ("bad") data, not the algorithms themselves.
What is the killer app for the technology under discussion? If the killer app is improving camera filters to make more vibrant pictures but it can also be used for surveillance then you do not worry about it being used for surveillance. If however the killer app is surveillance then you worry about it.
As a general rule surveillance however is not the killer app of these technologies, surveillance is just an instance of the killer app, and the killer app is the more general one of privacy violation of which surveillance is an instance.
>In this paper, we study the task of reconstructing a facial image of a person from a short audio recording of that person speaking.
I can imagine a potentially interesting application of this that has nothing to do with privacy violation or surveillance (that I can see):
What if we could take an audio recording of an ancestor or someone else of interest, for whom we have no pictures, and output a probable face for that person? Sounds interesting and potentially "killer" to me. Maybe I'm wrong.
I can see every police department and government agency paying plenty of money for being able to identify the speaker of a particular sentence in a meeting.
I can see the company with this technology providing services for academic research of the interesting kind you mention for the tax write-offs and also a sort of moral cover, but there is really only one thing I see as their main revenue stream.
I guess the question is, who decides what the killer app for a technology is?
2. I instead presume ethicists object to the unethical surveillance of users by any means, and warn of risks they find or foresee.
There is no slippery slope; the unethical effects of surveillance should be countered by effective and ethical means. Banning technology is not ethical or particularly effective. Banning and punishing unethical behaviors is about the best we can do.
In other words, don't ban iPhones. Ban configuring iphones to surveil on users in unethical ways. 911/999 geolocation is ethical surveillance. Stalking apps are unethical surveillance. There's ethical grey in between.
Less tersely: this article is one in a long procession of journalists trying to exert control over tech. The opening example (Speech2Face, which they aver is transphobic) is inflammatory and utterly unrepresentative of the usual topics of AI conferences. The other references are far better, but the choice is revealing-- it's not so much an abstract concern about an unaccountable few exerting control from the shadows, but alarm that someone else might be muscling in on their territory.
I think the article paints a good picture of the machine learning research community figuring out how to grapple with the growing number of people who want to probe its ethics, without portraying anybody as a villain.
I need to be clear that I don't believe in censorship and it's fine if this is what the author wants to write, but it should be called out how irresponsible it is. On a place like HN, the audience is generally able to read such things critically and form their own opinions, but for an audience that is not tech savvy (like that of the New Yorker) making false or hyperbolic claims about ethics and bias in research is both misleading and has the potential to really interfere with peoples livelihoods, funding, and legitimate research progress.
There needs to be more calling out of this kind of nonsense, the same as if people are posting about vaccine conspiracies or something.
Perhaps equally importantly, I'm skeptical that you can limit ethics rules in this way. If we clearly establish that suchandsuch research would be unethical at NeurIPS, I don't expect sociology conferences or think tanks would be eager to continue it, and I'm not convinced Alex Hanna would want them to.
I'd, in some ways, expect Dr. Hanna to ascribe to the axiom that predictive policing is unethical. As such, applied research into "improving" predictive policing would also sit squarely in the unethical space.
And I expect that's all you're going to get at an ML conference (outside of explicitly ethics/critique papers): "Our model predicts crime 4.2% better than competing models"
However, I'm not convinced that research into the meta-question "Can predictive policing be done ethically" would be considered equally unethical. ML is only a very small part of that meta question though. You're looking at much more complex socio-political changes that don't belong at NeurIPS ("do we need to change policing so that predictive policing can be done ethically").
> ethics saying they can't study minimum wages or tax increases
In general, we have far more oversight into the elected officials who manage tax and minimum wage policy than police forces. These structural differences in influence and oversight play into the need for self-regulation.
> Relevant expertise matters.
There's nothing that prevents a think tank from working with domain experts (in fact, iiuc this is often what happens: a think tank will contract out to domain experts to write a report on some subject). Now there's all kinds of other conflict of interest issues with think tanks, but we all know that already.
One issue here is that the pool of domain experts depends on the different incentives faced by different researchers. If the top machine learning conferences all decide to reject papers about predictive policing, then this will reduce the incentive for machine learning researchers (who mostly want to publish in these conferences) to work in the area at all.
Maybe critics of predictive policing think this is good, or think that it's better for the field to restrict itself to the few computer science researchers who are prepared to seriously devote themselves to it? But there don't seem to be very many of them. And, as someone who thinks there is a lot of room for police departments to in general do a better job --- and I think most of the people arguing against predictive policing would believe the same! --- it's disappointing to me that the end result of this argument seems to be disincentivizing technical research on it.
I wonder if a good solution would be simply to have a rating system for accepted papers, with ratings applied by the reviewers. Basically, you'd have two numbers:
Useful rating: On a scale of 1 to 10, is this a useful system that solves a real problem to make life generally better?
Problematic rating: on a scale of 1 to 10, can this system be used in ways that create new problems or exacerbate old ones?
So, let's use the example from the article of a system that predicts what a person looks like from their voice. Maybe the author submits it to the conference, and the reviewers accept the paper, and give it a useful rating of 3/10 (it could be used to identify criminal suspects with low probability, maybe?), and give it a problematic rating of 8/10 (could be used to discriminate against minorities or automatically apply cultural stereotypes; could be used for surveillance, could falsely implicate innocent people for crimes, etc...).
The author could then decide whether they want to continue publishing the research, or drop it. For all published papers, the ratings are listed in the table of contents of the proceedings, and the papers in each section are sorted with the most useful first and the most problematic last.
since the 90s everything that involved a computer was not prosecuted because lawefforcement/regulation didn't have the skills, not for lack of laws. But new laws were created spelling out computers.
A.I. is the same, it is not nothing different from enforcing existing laws and regulations, but because the agencies lack the skills, we will see a new wave of stupid laws that only misses the point instead of proper training and funding of what we already have.
There is zero need for new laws if a company doesn't hire or sell to a minority group, or their products explode, or whatever. No matter how much they scream it was the bad-brain-in-the-computer fault. Sadly, lots of companies will get away with this silly defense for a while.
laws should not care about the means. only the results you want to promote/punish.
What if laws about murder had to be updated every time a new bullet caliber or knife length become popular?
yet that is exactly what happens with money laundering laws and computers. or media regulation. or investment schemes. Every new technology and everyone forget scams via email can fall under scams via courier mail. go figure.
So the real trick is, how can we turn down the rhetoric so that we don’t get instantly worked up by every fake video that crosses our feed?
At the end of the day the world is shaped more by material conditions than ideas. That doesn't excuse the cases you're worried about, but I think the majority of violently polarising media will only end up being so because there was already some resource dispute in the first place. It would make more sense, then, to tackle those issues, rather than fruitlessly trying to engineer the human brain to be less susceptible to rhetoric.
I suppose you have heard about religions and faiths. I'd hazard to say that about 1000 years of European history was shaped primarily by them. Major wars fought, new countries founded, major discoveries and inventions made or unmade and prohibited, people's daily lives shaped and defined.
A similar thing has been occurring to Arabic world and neighboring peoples for about 1500 years, and is actively continuing now.
Looking at the current situation in the US is also illustrative.
I would say that ideas greatly prevail over material conditions once immediate survival is secured.
Were it not for the special religious status of Jerusalem, they won't care. There were more important trade route chokepoints, and more convenient sea ports nearby, if trade control were the actual agenda.
People getting their news from random videos posted on Facebook are already lost to democracy for all practical purposes, so little would really change.
What kind of simpler statistical algorithms do you believe are used in news feeds (which do not belong in the "AI" category)?
so yeah how far is dna from statistical algorithms? do i need to start training my dog how to fetch statistical algorithmic androids bout now?
thanks
What matters is that the authors of the article mean machine learning when they say AI. So they’re talking about unethical machine learning.
Which is basically unethical maths, whatever that means.
Are you suggesting that it would be strange to believe that ethics can be applied to mathematics?
If you create a true intelligence, it will necessarily be different from us.
To create a true intelligence, and make it your tool is unethical, at least to humans. Its slavery.
It doesn't matter if Neo can dodge bullets if the AI wields the Ban Hammer.
Well forgive me for being so intolerant to state such an offensive opinion, but I'm really getting fed up with transgender issues taking front and center with every possible place they can insert themselves, as if they're our top priority when in reality they're a 1% problem. Ironic when you think of what groups actually push this.
I personally have never seen such an issue be so flogged to death by a willing majority of self-proclaimed liberal society who probably have never even spoken directly in a conversation to a transgender person.
Worse, it makes it seem like we don't have equally important yet bigger civil rights issues that have not even yet been resolved. Personal pronouns, ambiguous "they" speech, policing algorithms for transgender hate, contorting ourselves as if we're majority transgender and being persecuted left and right.
Look, I'm not saying anyone of any group should be made to feel disadvantaged. Or that a minority just because few in numbers should be overlooked.
But for fucks sake, enough with transgender agenda being made to seem like it's the top concern of society, ahead of those who have been patiently waiting in line for their fair treatment.
/end