It's nice to finally get to see some of the content of the paper and it's awesome that Bender was willing to step up and give some context to world about what the hell it was about.
> A version of Google’s language model, BERT, which underpins the company’s search engine, produced 1,438 pounds of CO2 equivalent in Strubell’s estimate
So... they're saying it used about $100 worth of electricity.
The paper should have been probably focused on the title only, because the CO2 usage looks like a publicity stunt. She also didn't subtract the amount of CO2 saved by improved productivity by getting better search results.
It is like someone complaining that google map uses x, amount of energy, yet the old alternative (printing maps, and getting lost) was much worse CO2 wise.
Does OpenAI? Does the rest of the industry? Everybody is training larger and larger models, with research groups competing on who will be the first to scale up to some number of parameters.
It seems obvious that this would be something an ethicist would be interested in researching to figure out what the impact will be and start discussions about how to offset it.
Google isn't the only player in the AI field. The paper is about the entire field in general, not just what Google is doing. And plenty of the other companies in the field have not made similar pledges.
The total amount of electricity being used so far for training models does not yet move the needle in a significant way, but the point of the paper is that it's currently growing in an unbounded exponential fashion. And you know how exponential functions work.
There are models much larger than BERT with a much larger footprint, GPT-3 being the most well kniwn example.
Models like BERT aren't just trained once either when they are developed, but trained again with different domains, different parameters, different tasks in some cases. There is also fine-tuning (more frequent, less carbon intendive), so these are real environmental problems, and others have pointed them out.
How much more of a problem are a billion cars and 40% of our electricity being generated from coal?
We’ve squandered decades ignoring the big problems and now people want to run into the weeds with a thousand little problems, that individually don’t amount to much.
I find these CO2 arguments from model training extremely misleading. If this is what you care about, why don't you actually try to make electricity cleaner. Realistic, that's a much more effective way to address this tiny issue, together with the grand total of 50% of humanity's CO2 emissions on top. To me it always seems like virtue signalling, showing that you "care", while not having to actually make any realistic progress. Go build nuclear power plants, or at least prevent them from shutting down, if you want you're electricity to be at 12 g CO2 / kWh rather than 500 from natural gas or 800 from oil (you can also move your servers to France or Ontario). Do not focus on tiny, 10% efficiency gains in NLP. That's a marginal gain within a miniature cause to start with.
1438 pounds of CO2 is 250 liters of gasoline, or thereabouts, that's pumping gas five times. How long does a tank of gas last the average Google employee? Less than a week, considering what East Bay traffic looks like. People get paid to publish that stuff?
The issue isn't how much BERT uses, the issue is the trend over time in how much models use, with BERT being a recent data point.
The whole point of having AI ethicists is to identify current indicators of potential future ethical problems so that they can be considered in guiding the direction of development, so that you minimize acute ethical crisis.
> You do realize just how small this co2 output is compared to the human co2 output it replaces.
Sure, the issue is that the scale of models is increasing by orders of magnitude in a fairly short span of years, as well as the range of applications rapidly expanding. It doesn't take long for that kind of growth to go from a trivial issue to catastrophic one, and it's the exact kind of risk you have ethicists in a field to call out while it still is trivial so that some of the energy of people doing technical development gets directed to mitigate the risk of it ever reaching the catastrophic stage.
Analogously, I used to pooh-pooh the concerns about the total electrical usage of Bitcoin years ago, but it's since risen to an actually meaningful level. There's easily more money in AI in total than in crypto, and thus, you could see the total energy usage ending up a lot higher than crypto. That's not an insignificant amount of energy we're talking about here.
The number is probably quite a bit lower. The Strubell paper uses a PUE coefficient of 1.58, while Google datacenters are at 1.1. The figures are for GPUs, but Google uses TPUs, whose power characteristics were not public. Price is a proximate for power usage, though. TPUs might have been half as expensive in that experiment. Let's say that a further half of those gains are money that goes to Nvidia and not really related to power. So, making numbers up, training with TPUs might be another 25% more efficient. That's probably conservative, as Google claims that the TPUs are tens of times less power-hungry, due to their simplicity, but on the other hand, you also other fixed costs like racks, fabric and the CPUs to feed the chips.
In an ideal world, nobody would get hung up on details and everybody would understand that there is a lot of nuance when comparing things. In practice, if Google published a paper which quoted the Strubell paper without the caveats, I can see headlines about how inefficient and bad for the environment Google Translate is. PR would get busy and obtain corrections or follow-up articles to clarify things, but those rarely get the same attention. And it's still extra work that could have been avoided, which in itself is bad optics ("do you folks even review the stuff you send out for publication?").
I'm all for reducing emissions and improving efficiency, but I find the premise a bit of a stretch.
Yes, large and wealthy organizations have a big advantage, but that applies to pretty much anything they do, not just language models. Inefficiencies are bad, but if you ask a number of people why fix them, I think they'd mention financial cost and wasted time well before looking at it as an issue of ethics and fairness.
Reducing costs for language models is already a great idea across all fronts. So is reducing inequalities. It's linking the two that sounds like a strained argument to me. Suppose someone makes models ten times smaller next week. Will marginalized communities' lives improve soon? There must be more. I haven't seen the paper, so I'm curious how it is all framed.
> The whole point of having AI ethicists is to identify current indicators of potential future ethical problems so that they can be considered in guiding the direction of development, so that you minimize acute ethical crisis.
An alternative motivation could be to launder accountability so that when the acute ethical crisis /does/ come, you can throw your hands in the air and say "See? Look at how much resources we poured into this and it still happened! At least we tried!"
You have 1438 pounds of CO2, roughly 1/3 of that mass is carbon, so that's about 500 pounds or 250 kilograms of carbon.
Gasoline is a hydrocarbon, but the mass of hydrogen is neglegible compared to that of carbon, so it's not too wrong to say that this mass of carbon came from the same mass of gasoline, 250 kg. The density of gasoline is around 1 kg/l, so we are talking about 250 liters of fuel. The typical tank of a compact car holds about 50 l.
Is this paper on arxiv? This overview doesn't answer any critical questions. For example, it's easy to fill up 128 references and a reader shouldn't blindly trust a claim that, "The version of the paper we saw does also nod to several research efforts on reducing the size and computational costs of large language models, and on measuring the embedded bias of models."
If a key part of Google's claim is that the paper omits relevant research, an author should have simply posted their 128 references and openly asked what work was missing. This whole saga could be easily solved instead of being dragged out for clicks.
> [...] Though Bender asked us not to publish the paper itself because the authors didn’t want such an early draft circulating online, it gives some insight into the questions Gebru and her colleagues were raising about AI that might be causing Google concern.
> If a key part of Google's claim is that the paper omits relevant research, an author should have simply posted their 128 references and openly asked what work was missing. This whole saga could be easily solved instead of being dragged out for clicks.
So, we have on one hand, a researcher who got perilously close to litigation with her employer in the past (IMO because of missteps on both sides).
On the other hand, we have an employer that then was skittish about telling her that they didn't want the paper published (to protect the employer's business interests, mostly, it seems, while maintaining a veneer of open research organizations). And resorted to small statements through HR and intermediaries demanding retraction.
This relationship has broken down; there's no ready process to tidy up the misunderstandings.
I will say that Google's claims to be fostering an open discussion of AI ethics and confronting potentially uncomfortable truths on this path are looking a bit more dubious, though Ms. Gebru doesn't look so particularly easy to work with, either.
Are these numbers the energy to train a model? The whole point of these new NLP models is transfer learning, meaning you train the big model once and fine-tune it for each use case with a lot less training data.
5 cars worth of carbon emissions is not a lot given that it is a fixed cost. Very few are retraining BERT from scratch.
EDIT:
The other two points are also disingenuous.
* "[AI models] will also fail to capture the language and the norms of countries and peoples that have less access to the internet and thus a smaller linguistic footprint online. "
NLP in "low resource" languages is a major area of research, especially because that's where the "next billion users" are for Big Tech. Facebook especially is financially motivated to solve machine translation to/from such languages. https://ai.facebook.com/blog/recent-advances-in-low-resource...
* "Not as much effort goes into working on AI models that might achieve understanding, or that achieve good results with smaller, more carefully curated datasets (and thus also use less energy)."
This is also a major area of research. Achieving understanding falls under the purview of AGI, which itself carries ethical and safety concerns. There are certainly research groups working toward this. And reducing parameter sizes of big networks like GPT-3 is the next big race. See https://news.ycombinator.com/item?id=24704952
Carbon emissions arguments tend to ignore the value of what's being done as well. BERT and other transformers were meaningful experiments that were valuable in furthering a major research direction and enabling more effective consumer and business applications. In that sense, it's like any other company doing R&D - of course energy will be used and of course there will be some inefficiencies.
I think it's quite misleading to compare the energy usage of an industry-wide research effort to individual consumption. The graphs look bad - "wow, 626,000 lbs! that's 284 metric tons of CO2! a plane flight is way less!" - but there's a fundamental difference between "progress on a problem being worked on by thousands of highly-paid researchers" and "I bought a car".
Meanwhile, the worst power plants are generating on the order of 10+ million tons of CO2 every year. There are at least a dozen of these in the US alone. Car factories are emitting hundreds of thousands of tons of CO2 (Tesla is somewhere around 150,000 tons a year, apparently, and it's designed to be efficient). Perhaps activism around CO2 emissions in ML training might be better focused on improving the efficiency of those things instead, seeing as a 1% improvement would outweigh the entirety of the NLP model training industry. It's certainly good to keep in mind the energy costs of training in case things balloon out of control, but right now the costs relative to the results seem small and not worth highlighting as some forgotten sin.
Still though, it’s good that these numbers are brought into light, along with other “hidden” costs in the IT industry. Otherwise we’ll just spiral into whataboutism and “my own carbon footprint is totally fine, because somewhere out there a model is doing worse things”. It also goes to show the sheer scale of this research field (arguably in a double-edges sword way) if the general public was still thinking “nerds in a basement recognizing cats”.
I think we all know that there is a carbon footprint cost to what we do. Its the reason google has been working on renewable energy datacenters for so long:
This is exactly the problem I have with naive environmentalism.
The most recent data I could find for the United States total carbon footprint was 5 billion (metric) tons in 2016.
Total energy consumption of all computers, mobile phones, datacenters, servers etc, combined, isn't even a percentage point of that.
Yes, CO2 emissions are a problem. You are not going to solve that problem by targeting sectors which entire footprint is not even a significant digit.
I think the main was sidetracked. If we collectively emit as much CO₂ as 0.1 or 100 persons lifetime for a single model, every year .. who cares? I don't.
But, what if Amazon wants it's own model with its own curation? Maybe we need different languages, maybe countries would like to have their own model with a different world-view. Why shouldn't a researcher train their own model, maybe experiment with different versions? Why should consumers be relegated to pre-trained model with inscrutable preconceptions?
> 5 cars worth of carbon emissions is not a lot given that it is a fixed cost. Very few are retraining BERT from scratch.
It's not a fixed cost though, it's just how much was spent on this year's iteration of the model. The overall point being made is that model training costs are growing unbounded. Next year it could be 30 cars' worth or whatever for BERT-2, then 600 cars' worth for BERT-3 the year after. That's what it's warning against. At some point it isn't worth it.
This is a very valid argument but it's hard to know what scaling a transformer will really do without trying (looking at you GPT-3). This is probably an issue for ML in general at this point.
I think a more nuanced conversation around these topics will look at exactly what you bring up, how do we properly trade the potential knowledge benefit against the costs?
It pains me that entirely valid avenues of research like this get covered up in nonsense and drama and their message seemingly lost in the midst of it.
Yes, an in-depth nuanced conversation is absolutely merited here, and Gebru and her colleagues were having it in the most rigorous way possible -- in peer-reviewed papers at AI conferences. I won't pretend to remotely be contributing as much to the discussion as they were.
The Strubell paper which is the origin of this "5 cars" number isn't even in the right ballpark for this stuff. What they did was take desktop GPU power consumption running the model in fp32, extrapolate to a 240x GPU (P100) setup that would run for a year straight at 100% power consumption.
Yes if you do run 240x p100s at literally 100% 24/7 for a year you get the power consumption of 5 cars. This run never happened though, this all ran on TPUs at lower precision, lower power consumption and much lower time to converge.
If anything this tells you that electronics are ridiculously green even when operating at 100%. I've never profiled world-wide carbon production but something tells me if you wanted to carbon optimise you'd be better served trying to take cars off the road and planes out of the sky.
> I've never profiled world-wide carbon production but something tells me if you wanted to carbon optimise you'd be better served trying to take cars off the road and planes out of the sky.
We're getting a bit off-topic here, but the #1 target by far in reducing greenhouse emissions is power generation. In transportation it's significantly trickier to replace petrol-based fuels (especially for airplanes), but it's straightforward enough in power plants. And crucially, you can convert all the petrol-powered vehicles to EVs that you want, but if the electricity they're getting from the wall is still provided by burning petroleum then you haven't actually done that much.
Luckily, computation can for the most part be located anywhere (exactly the opposite of transportation), and thus you have a lot of data centers near hydro and other renewable sources so that they can use the cheapest green power available.
> We're getting a bit off-topic here, but the #1 target by far in reducing greenhouse emissions is power generation.
I readily admit I don't know any of the numbers associated with carbon production and my comment was solely based on the one GPU vs car figure presented in the aforementioned paper.
As I already mentioned, the paper also uses an industry average PUE factor of 1.58, when Google's is 1.1. Other large tech companies can't be too far behind.
Which makes me wonder how far removed AI researchers are from actual production environments. I'm not faulting them, because there's only so much time in your life; the more realistic problem is when someone else takes a paper as gospel and runs headlines with it. Kinda like the trolley problem. Imagine the absurd extreme in this case of governments wanting to regulate large language models because of pollution or to level the competition playing field.
I'm more interested in the related nugget, which is the carbon footprint of a Google Search. Some estimates from a decade back put it at 7g (boil a cup of water), but since then it's probably only gotten larger. However if Google's dstacenters truly carbon neutral, does it even matter?
Yes it's something I often see ignored as "common knowledge" dictates that in ML inference is way cheaper than training. But if you're running a model in production at google with loads of google searches hitting it every second. At what point does the inference costs start to outweigh the training costs?
I simply have no idea where the hinge point is. This could inform other questions like, could it be worth to scale up to get a more accurate model (pay up-front in training) to avoid further searches (inference)?
Green energy is often used in addition to the carbon-based energy. The total amount of used energy increases. Green energy should replace carbon-base energy.
Wind and solar are finite too. The places where they can be harvested are scarce. So if Google using up green energy for bells and whistles on the search page, less homes can use green energy for heating and transport.
This doesn't strike me as anything that needs to be buried. The energy argument is tenuous and at this point models use relatively little compute power. The language bias is a better tack but I don't think this is earth shattering to anyone. Most of the internet is from Western sources and some fraction of that is racist/prejudiced. I think this is obvious to anyone who has ever used the internet.
It sounds like it should be buried because it's clickbait headline generating fluff that only would get attention because of where the authors come from. And I say that after reading a summary that sounds like the authors had a positive opinion of the paper.
Google (probably) wanted to bury it because many of those clickbait headlines would have been negative to google.
A few decades ago? What is a fire-and-forget guided missile but an autonomous suicide drone? Some missiles, particularly anti-ship missiles, even use computer vision in the air to identify objects and pick targets. Mark 60 CAPTOR naval mines sit underwater listening to nearby ships. When a noise is classified as an enemy ship, the mine releases a guided torpedo to kill it. Of course autonomy in naval mines is nothing new; the most basic sort 'decide' to kill any object that bumps them too hard. The innovation is making them discriminate between enemy ships and everybody else.
The point is not that autonomous weapons systems are possible, but that their existence does not presage SkyNet, and the whole discussion distracts from the real societal impact, like some people being denied shelter/finance/employment/social connection/etc based on opaque ML boxes that intrinsically discriminate but in a way that can't be easily interrogated it controlled with legislation or court action.
> Most of the internet is from Western sources and some fraction of that is racist/prejudiced. I think this is obvious to anyone who has ever used the internet.
The impact of that on AI and the difficulties in counteracting it are not obvious to anyone who has ever used the internet or even—from, among other bits of evidence, public clashes Gebru has had with people who work in AI outside of ethics—not even to everyone building and training AI models on public data that is impacted.
While this is a lot of electricity and CO2 to consider, is it fair to compare them to cars? I don't think this so much means that one should get rid of training these big models but rather do their best to ensure that the data centers that these models are trained on are getting their electricity from mostly 0 emission sources. This actually seems like a doable task since you don't need to be physically near your compute cluster. I think Google and Facebook could also turn this into a PR move by putting some research money into green tech (ambiguous).
I was a little disappointed in the quality of this research. I can see where Google is coming from. Some of these topics seem regurgitated (e.g. bias in online text) and some are just irrelevant (CO2 emissions). Training language models does not contribute in any meaningful way to global CO2 emissions. Overall not a very strong paper in contrast to the other reactions I've been seeing online.
We're only reading a summary of the paper because apparently the authors aren't confident enough in its quality to release it publicly.
You can't claim that Google dismissed this paper out-of-hand while simultaneously saying "oh, but it's too much of a draft to release publicly". Uh, if it was too much of a draft for the public why shouldn't it be too drafty for Google? Are we really pretending that Google has lower standards of quality than the general public?
If a company is paying a researcher a lofty salary (I am guessing ballpark mid six figures), and provides that researcher virtually unlimited tools and resources, shouldn't that company have a say in what material gets published by the researcher? Why shouldn't it have the ability to restrict research of dubious quality that makes borderline incorrect conclusions about the company?
Training language models does not contribute in any meaningful way to global CO2 emissions.
Google consumes a vast amount of energy: "10.6 terawatt hours in 2018, up from 2.86 terawatt hours in 2011."[1]
If training and retraining models is a significant and inefficient part of google's energy consumption, the point doesn't seem insignificant(edit: The most advanced AI model involve as much as computing and energy any programs ever created [2], btw). I'm biased by the impression Google's actual search results haven't improved very much but I don't think I'm alone in that impression.
I obviously don't have the data, but I would argue that a tiny tiny fraction of Google's energy usage is dedicated to training language models.
I'm guessing the vast majority of the energy usage is for serving billions of requests for various products, such as search, youtube, maps, gmail, photos, etc.., and the cpu, network, and storage requirements for those requests.
Training and retraining ML models is definitely not on the hot path.
I’ve recently noticed that this commonly occurs ie multiple relevant threads that make up the top 50 of HN over say a 72 hour period, all related or rifts on a similar discussion.
Wondering if there is a way to group them as part of the same topic/submission? (thus saving you the manual work of posts like this). I appreciate this would (i) require a code change and (ii) would shift the HN model from individual threads based on 1 URL submission, but just thought I’d suggest it to the HN brains trust.
In other news: Keep up the good work that you do on HN to give this online community ‘structure’.
One potential implementation: at the top of each thread, after the metadata for the thread itself and before the comment box, are a list of ‘Related discussions’ with a bulletpoint list of related HN threads.
Unfortunately, PG is of the opinion that he "finished" HN over a decade ago, which is why we don't get any iteration on the feature set here, no matter how useful it might be. I agree that this would be a good addition but I don't see it happening with this mindset.
As someone who's worked on HN code since then, I can tell you there's no such "mindset", nor is it true that "we don't get any iteration on the feature set here" (11 days ago: https://news.ycombinator.com/item?id=25197418). It is true, though, that most of the changes are subtle enough not to be so visible, including the ones I'm working on this evening. Most of the effort goes into attempting to improve, or at least preserve, the quality of submissions and comments.
> There have been a few high-profile cases, such as the college student who churned out AI-generated self-help and productivity advice on a blog, which went viral.
This is how much of the internet has felt for a long time. After this nugget, I now wonder if Vox is just a model trained on Piketty and Tumblr.
edit: Also not sold on the CO2 argument. Too many variables! Nerds will calculate and re-calculate such things, with the result jumping all over the place, swearing that they've gotten it right--this time! No humility, in spite of the odds.
Medical ethics is a serious subject, and then there was Asilomar on the ethics of genetic engineering. Just because this AI ethicist isn't one, it doesn't follow the field is worthless.
The Asilomar Conference on Recombinant DNA was a high-caliber event. The paper here, done at Google, a leading AI company isn't, and the ethics-washing on view here is deeply concerning.
That’s the crux of the problem. Top tier academia and ethics both attract the absolute best and the absolute worst. So I suppose we need both the critical cynicism and the optimists in threads like these.
The thing is, that's about all an AI ethicist can say. The problem is there isn't that much to modern "AI"/machine learning. It's just brute force approximation/simulation/curve-fitting. There's a lot of tweaking, a lot of data and a lot of processing power. It does a lot but fails unpredictably and has unpredictable (or predictable) biases and leaves you at the mercy of a black box.
There, I've repeated the paper without reading it. But Google did hire her to be an AI ethicist so what else would they expect?
The significance of their job is imagined at best, borderline academic welfare. Guess that is why Google can make the choice to remove her this quickly.
It’s academic welfare but also woke political welfare / a PR exercise. It shouldn’t be legitimized because it is just inviting unnecessary political controversy into the workplace.
If a person of color (I am myself non-white) is not performing, what does it take to fire them without the entire world playing the race card on you?
Are we creating a society that makes it impossible to fire a person of color? You know there are bad apples in every race, right? How do we handle such scenarios? Seems unfair to me, myself being a person of color - I don't want the world to treat me like some kind of a hero for being non-white / minority. I want fairness and it is frankly offensive.
I am not pleased with the way we're treating each other. It's supposed to be equal opportunity.
I also want us to have scientific discussion about gender differences (backed by research) and other difficult conversations. Nature doesn't give a fuck about any of this - if our goal is to uncover the way mother nature works, we're going to have to meet difficult truths and not be afraid of it.
You’re going to be downvoted, not because of your questions, but because you decided to put “please bring the downvotes” at the end of your comment. Why would you undermine yourself by sticking such an antagonistic comment at the end of your post?
Considering a Pennsylvania law maker made fake Twitter accounts to write "as a gay person", when you start saying you're a non PoC and conflating what happened with Timnit to "firing badly performing PoC"s you do not at all sound genuine.
There is no way to prove you are actually PoC. And even if you did, it's incredibly unrelated to the matter at hand.
No one has even claimed she was fired for bad performance and dozens of Google employees have said in no part of their process was academic rigor taken into account.
What you're doing is the literal definition of concern trolling.
That's an insane accusation. You can check my comment history perhaps that will reveal some aspects of it.
Also, I am kind of shocked you would accuse of me something like this. WTF. I am not trolling. I am asking a difficult question that needs to be discussed because no one is discussing it.
She was fired after sending a mass email, to hundreds of colleagues, criticizing her employer in very strong terms. What company wouldn’t fire an exec who sends a “f* this place” email to the company mailing list?
You seem to have rigid ideas of what it's possible to experience, believe and be genuine about on the basis of the color of one's skins. Maybe that should be a signal to step back and rethink.
You also illustrate beautifully the vapidness of this fad notion of "concern trolling". Because they have not conformed with your view of the issues and used your preferred language, they can only be conceived of as trolling.
The other poster seems to be making a genuine effort to express their thoughts and edited their post to be less combative. Yet you can ONLY see them as being dishonest and disingenuous.
Downvote goading, things like 'the entire world playing the race card on you' are basically trolling, they don't sound like asking a question in good faith. This particular case is also not about someone being fired for 'not performing' - nobody has claimed that. So if trolling is not your intent, you should edit it down to whatever it is you are actually asking.
That's why I removed it. I am asking the questions in good faith. I want to know how the society will avoid having chilling effect of not being able to fire underperformers and making difficult choices when they need to be made.
If we cannot sit down and have a peaceful conversation, calling trolls and other non-sense, please don't divulge in this thread.
'playing the race card' and asking about firing for non-performance in the context of a story that does not involve someone being fired for that are not invitations to 'peaceful conversation'. They come off as deliberate provocation. Again, if that's not what you have in mind, just edit it out and focus on whatever it is you want to ask.
No, I'm sorry, the high horse you're trying to ride here just isn't going to hold the weight of the ideological argument you tried to make. Timnit Gebru was severed from Google because of a conflict between her and the research practices at Google. Moreover, the story begins with her offering an ultimatum and Google preemptively accepting her resignation. No part of this story reasonably involves the "gender differences" and "difficult truths" you're invoking.
Dropping boilerplate ideological provocations onto unrelated threads isn't good-faith conversation. Whether you mean it to be or not, it has the effect of trolling, and on HN, trolling is a strict-liability offense; your mens rea matters less than the outcome.
Probably fairer to say she was severed because the fruits of her research are irreconcilable with the company's peaceful enjoyment of its core business model. The sudden commitment to research practices (as documented elsewhere in this thread) being a fig leaf for that less comfortable truth.
Sorry, I know this is somewhat off-topic, but I feel like it is important to engage on a human level for a minute.
I’m genuinely sorry to hear that you feel uncomfortable voicing your opinions. I know from personal experience how hard it is to feel like you have to keep yourself closed off to the world. As a species generally, and as technologists specifically, we have some way to go to create non-toxic spaces for people to share ideas.
Still, you are engaging in fortune telling[0]—you really can’t know that your comments will be downvoted before you make them. Feeling compelled to add notes to the end of your posts encouraging others to downvote is your brain tricking you into tilting the scales to “confirm” what you “knew” to be true. It may feel like a helpful strategy to blunt the emotional pain of discovering that people don’t agree with you all the time, but from what little you’ve said, it sure seems like it’s just reinforcing your negative outlook. I don’t want you to feel bad all the time, and I suspect it is not actually true that most people here are going to disagree and downvote you to oblivion all the time so long as you avoid self-sabotaging.
I know it can be incredibly hard not to take downvotes personally, and, I hope you are able to try to reframe them as what they are: some random people, some of whom are thoughtful and some of whom are not, pushing a button. It’s not a personal attack, even though our brains can make it feel very much like it is. If you truly are getting downvoted a lot, it may be a signal that some of your opinions aren’t fully thought through and need to be re-evaluated, or perhaps that you just didn’t present your ideas well. On the other hand, your brain can and will exaggerate the negative experiences, make them seem like they are happening a lot more than they are, and make you feel bad even though you’re actually doing just fine.
Anyway, while I’m sure it happens (I don’t think there’s any space that is totally immune to bandwagons), I don’t get the sense that genuine and thoughtful comments regularly get downvoted to oblivion here. It’s trickier than ever these days since there is a lot of bad-faith argumentation going on everywhere online under the guise of innocently “just asking questions”[1], and I think it’s fair to say that there is an growing immune-like reaction which is sometimes attacking genuine posters because it’s just impossible to tell who’s being honest and who’s being a shitty troll.
So just keep doing your best, anonymous internet commenter. :-) If you feel like you can’t, I hope you can find a counsellor or friend who will listen and help you into a more positive head space. At the least, your post has generated some reasonable and civil discussion, and that’s what we’re here for, right?
Thanks, I read your comment in its entirety. I think we need more people like you. Disagreements about issues and these things happen all the time. My fear was founded in the fact that people perceive me as either 1) Faking as a non-white person (see some accusations here) 2) On top of that, accusing me of being racist. The truth cannot be more further from that. 3) Downvoting because its a difficult conversation. 4) Perhaps tracking me down and finding my identitity and may be someone crazy can go nuts and cause harm to my persons or my career.
I think it is causing a reverse effect. It's going to create more racist behavior from employers - "If we cannot fire a PoC or some minority, let's not even bother hiring them in the first place."
The problem with sarcasm on this issue (beyond the inherent ambiguity of plain text sarcasm) is that this is something you can see people saying literally (or, at least, as what they see as only a slightly hyperbolic description of a real trend that they are complaining about.)
When they want to publicly claim credit for being a certain way but their actions point to another, it's up to them to clarify their position, and use whatever evidence to support their claim.
This seems an extreme solution to media investigation of a small number of people losing their jobs
Perhaps instead, in these situations, employers could provide, at the discretion of the dismissed, information gathered while performing due diligence leading up to the dismissal
I personally think that just the way the employer is required to document the reasons for firing, the person going off on media must also provide proof that they were being treated unfairly.
Most people who get fired don't the option of going to media. The way "star" researchers and the way average employees get treated is going to be different whether we're talking white or non-white.
That's a good point. May be we need ways to have a common person without a huge media presence have a way to voice their concerns of racism.
Since media cannot cover thousands of individual cases, there should be legal avenues without deep pockets for lawyer fees to sue companies for racist behavior.
The court should look at this situation objectively and factually.
It doesn’t matter because aggrieved employees with a compelling story will always have going public with real or imagined grievances as an option. The persons protected class is just another attribute — this is a story because the subjects are notable.
If a company is responsible in how they manage, follows policies, etc they are fine. If executives or others are allowed to misbehave and the company is too cheap to buy silence, things may not be fine.
What’s the real story here? I don’t see evidence of incompetence. But you can be fired for any legal reason in absence of a contract. Maybe there’s some unknown political or other issue. Maybe some conduct crossed a line. Who knows.
This is just your perception based on media speculation, not something actually backed up by numbers. Just because people on the internet are talking about race doesn't mean a firing has anything to do with race.
The common "nice" way to remove non-performers is a "Reduction in force", also known as "layoff" where they let multiple people go ritualistically, though I've seen it for as few as two people. Then they can argue it was about finances, not personal. They offer a decent severance that requires the employee to sign an agreement that requires, in part, they don't sue.
The word "fired" often is reserved for terminating someone for cause, where you really broke the rules and get no severance, not even the two weeks minimum that is customary. Non-performers usually aren't treated this way. I'm not sure this is really what happened in this story.
Employers need solid documented reasons for firing someone. Fired person needs proper recourse for appealing it if they felt it was unfair. Going to NYTimes is OK if there is solid proof of racism, the company needs to face the consequences - both legal and public image / media perception. If the fired person goes out to media and brings down the company without proof - we need to have mechanisms for that as well - I don't see us addressing that.
Here we're talking about something different: suppressing the speech of a person of color raising some well-thought out arguments about racism that are problematic to the employer's core business.
Google has set up a research organization that ostensibly is empowered to ask and ultimately work to openly resolve difficult questions like these... but when the rubber hits the road, they instead throw the researcher under the bus.
What are the arguments about racism that were suppressed? The paper that didn't pass internal review was about the external costs of training large models (correction: the paper also discussed language models not adhering to anti-racist language standards).
> If a person of color (I am myself non-white) is not performing, what does it take to fire them without the entire world playing the race card on you?
This line of reasoning means, you can't be pro diversity and fire someone from the underrepresented groups at the same time for their behaviour.
People are conveniently choosing to forget this is the same company which not long ago fired a person when he complained about the company being too pro diversity in their hiring.
Unless I missed something (and I just rechecked) there's nothing in the guidelines about comments having to be on the topic of the submission.
IMO it enriches HN comment sections to allow for people to bring up adjacent topics, things that came to mind or funky little "this reminds me of..." anecdotes.
If a person of color (I am myself non-white) is not performing, what does it take to fire them without the entire world playing the race card on you?
For them to be one of 99.9% of the world who can't mobilize a following to create a complaint about this?
Are we creating a society that makes it impossible to fire a person of color
We so far from such a situation like that that your complaint is absurd. A few places with a history of discrimination may have trouble firing the few people of color they might hire. That's about it. In the real world, incompetent people get fired and often competent people as well. A few people may make a career of playing the race card but that's a limited number of people.
Both racism and opportunists "playing the race card" can be real at the same time.
> I have a genuine question:
If a person of color (I am myself non-white) is not performing, what does it take to fire them without the entire world playing the race card on you?
Great question. Part of the reason why the black unemployment rate is so low is that it's virtually impossible to fire them.
> An AI model taught to view racist language as normal is obviously bad. The researchers, though, point out a couple of more subtle problems. One is that shifts in language play an important role in social change; the MeToo and Black Lives Matter movements, for example, have tried to establish a new anti-sexist and anti-racist vocabulary. An AI model trained on vast swaths of the internet won’t be attuned to the nuances of this vocabulary and won’t produce or interpret language in line with these new cultural norms.
This is a pretty superficial take on what is an extremely interesting sociological topic. (To be clear, I’m referring to the article, not the underlying paper which we don’t have.) Obviously just because social movements “have tried to establish ... vocabulary” doesn’t meant that vocabulary has become a “new cultural norm.” Plenty of such efforts end up being cultural dead-ends.
Take for example a term like “LatinX.” This term has been proposed and is used by certain people, but is extremely unfamiliar and often alienating to Latinos themselves: https://www.vox.com/2020/11/5/21548677/trump-hispanic-vote-l... (“[O]nly 3 percent of US Hispanics actually use it themselves.... The message of the term, however, is that the entire grammatical system of the Spanish language is problematic, which in any other context progressives would recognize as an alienating and insensitive message.”).
The article hand-waves away a deeply interesting question: What should an AI do here? Should AI reflect society, or be a vehicle for accelerating change? It seems at least reasonable to say that the AI should reflect what people actually say, in which case a big training dataset is appropriate, instead of what some experts decide that people should say. In some contexts, for example with “LatinX,” researchers seeking to enhance inclusivity could instead end up imposing a kind of racist elitism. (People without college educations—which disproportionately comprises immigrants and people of color—tend to be less knowledgeable about and slower to adopt these changes in vocabulary.)
The paper seems to imply that AIs should not reflect “social norms” but that training data should be selected to accentuate “attempt[ed]” shifts in such norms. Maybe that’s true, but it doesn’t seem obviously true. To return to the example above, is some Google AI generating the phrase “LatinX” (which 3/4 of Latinos have never even heard of: https://www.pewresearch.org/hispanic/2020/08/11/about-one-in...) in preference to “Latino” or “Hispanic” actually the desired result?
"What people say" is itself quite nuanced. Words that two black might say to one another may be casual but appropriate and yet extremely offensive if said by a white person, or, presumably, an AI, in a different context.
What the AI should say is hard given that there is no one right answer to the question of what any individual should say. Different contexts change the equation completely. Seems like a nightmare to define the behavior or test it.
The right solution may involve training a model on everything we have access to and finetuning it based on the context you want to use the model in and the historical examples we will build up of mistakes previous models have made.
Sorry to hijack this comment but are you aware stumblingon.com is down?
Error message: Not found - Request ID: 78591950-5c3e-4c76-9499-03931fab131e-53382846
I was actually going to turn the website off (I believe it hasn't been working for a couple weeks now) as I didn't think anybody was using it. Glad to see someone is though - it's back on!
Really? That's a shame. I use it whenever I want to experience the indie web. I shared it with a friend and he's also a fan.
I have even submitted a couple of websites.
If you do turn it off, would you mind sharing your list of websites?
I'm learning django right now and for my first project (after the tutorial project) I will recreate stumblingon. When I do I'll add a logging/metrics system of some kind. That way, I won't make premature decisions about turning it off in future.
I'll also commit to replacing the website with a list of the index for a month or so before (if) I turn it off for good.
Now that I'm thinking about remaking it, is there anything you think is missing from the website?
> Take for example a term like “LatinX.” This term has been proposed and is used by certain people, but is extremely unfamiliar and often alienating to Latinos themselves: https://www.vox.com/2020/11/5/21548677/trump-hispanic-vote-l... (“[O]nly 3 percent of US Hispanics actually use it themselves.... The message of the term, however, is that the entire grammatical system of the Spanish language is problematic, which in any other context progressives would recognize as an alienating and insensitive message.”).
what’s the energy expense of training GPT-3? It should be very high. Eager to know the upside of such models. Environmental and social impacts of tech are becoming more direct and mainstream in today’s world!
How exactly is the environmental impact of tech becoming more "direct and mainstream" [citation needed]?
Data centers around the world account only for a tiny fraction of electricity consumption or carbon emissions. Do you even account for the reduced car and air travel due to remote working and online shopping?
How are advanced few-shot learners like GPT-3 even remotely a problem, training less models is somehow worse? Do they even know what they are taking about?
Its all very confusing, but a lot of the work done by grievance studies becomes immediately easier to understand once you realize they are arguing in bad faith.
What is the energy expended on training GPT-3? Eager to know the upsides of such models. The environmental and social impacts of tech are more direct and mainstream in today’s world.
Dr Gebru's research AFAICT shines a valuable light on some interesting ethical problems. Yet the response so far from Google and even HN has been one of censorship and suppression of Dr Gebru's free speech. How can we reconcile free speech and Google's right of review to her research?
Not sure the details of her employment arrangement, but usually work you do for your company belongs to the company and you don't have any right to freely publish it. Seems she was able to publish it anyway, so that's pretty good, isn't it? She's got more than the usual helping of free speech. Where's the censorship or suppression?
I did not read the paper (just like most people here), but by the title — “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” — it does not look like the CO2 emissions thing is the main topic of this research.
BTW, "Stochastic Parrots" is a very descriptive name for the problem
> Moreover, because the training datasets are so large, it’s hard to audit them to check for these embedded biases. “A methodology that relies on datasets too large to document is therefore inherently risky,” the researchers conclude. “While documentation allows for potential accountability, [...] undocumented training data perpetuates harm without recourse.”
Since these models are being applied in a lot of fields that directly affects the life of millions of people, this is a very important and underdiscussed problem.
> Since these models are being applied in a lot of fields that directly affects the life of millions of people
In particular, it is being applied right now to rank Google search results, and probably responsible for lots and lots of Google's profit. You should be skeptical of Google's appraisal of the paper that is material to Google's profit.
> BTW, "Stochastic Parrots" is a very descriptive name for the problem
Meh, both parrots and language models are inferior to humans in producing language, but one is more useful than the other. And real parrots are also stochastic, like all living things.
Why? It sounds like ideologically driven circular reasoning. If you train an AI on the largest dataset it's possible to obtain then you have, almost by definition, done the most you can to avoid bias of any sort: the model will learn the most accurate representation of reality it can given the data available.
Gebru is the type of person who defines "bias" as anything that isn't sufficiently positive towards people who look like herself, not the usual definition of a deviation from reality as exists. Having encountered AI "fairness" and "bias" papers (words quoted because the words aren't used with their dictionary definitions), it's not even clear to me they should count as research at all, let alone be worth reading. They take as the starting premise that anything a model learns about the world that is politically incorrect is a bug, and go downhill from there.
If you train an AI on the largest dataset it's possible to obtain then you have, almost by definition, done the most you can to avoid bias of any sort
All politics aside, this is not even true for toy ML problems. If I’m trying to do digit recognition and “all the data I can find” is a billion hand-written 0’s and a million hand-written 1’s through 9’s, naively training on that data will yield a model that’s pretty close to guessing 0 every time.
We're talking about tech firms that have access to the entire internet and use it. Hypothetical examples involving imaginary datasets that nobody would use don't prove anything relevant. And note that my argument is not about whether you actually avoid bias, it's about whether you've done the most you can do to avoid it. If you used all the data you've got, then you've done the most you can, even if for some reason the data you've got isn't any good.
> We're talking about tech firms that have access to the entire internet and use it.
I think you're trying to say that a large enough dataset will be free of bias. I don't see how that follows. If I train a model on home mortgage decisions, I will replicate the bias on that currently exists on that dataset - https://news.northwestern.edu/stories/2020/01/racial-discrim... - unless there are conscientious choices to reduce that bias.
Researchers in ethics in ML are specifically trying to enable tech companies to do a better job of not replicating bias and justifiably point out where that is occurring.
Third, I would argue that applying an ML model to do something faster if it replicates the bias of a previously human decision is even worse. The bias has taken the human element completely out and systematized the bias and made it possible with even less friction.
Given the current public discourse climate in academia, where 99% of the researchers are politically left or extreme left, I recommend a grain of salt for any research that makes claims racial discrimination. The report you quote has at least a couple of strange holes. The paper itself is behind a paywall, thus beyond mere mortals reach.
> For example, in about 10% audits in which a white and an African-American auditor were sent to apply for the same unit after 2005, the white auditor was recommended more units than the African-American auditor. These trends hold in both the large HUD (Housing and Urban Development)-sponsored housing audits, which others have examined with similar findings to us, and in smaller correspondence studies
They fail to mention how large is the gap. Is the white auditor recommended 102 vs the black auditor 97, or 150 vs 50, or 200 vs 3? Without such critical information it is hard to form an opinion, unless one already has a large bias in accepting discrimination narratives uncritically.
> In the mortgage market the researchers found that racial gaps in loan denial have declined only slightly, and racial gaps in mortgage cost have not declined at all, suggesting persistent racial discrimination. Black and Hispanic borrowers are more likely to be rejected when they apply for a loan and are more likely to receive a high-cost mortgage.
They fail to mention the magic words 'when controlled for income'. America has a huge income disparity problem, which is conveniently forgotten behind the ongoing race (and gender) hucksterism. Assuming we'd wave a magic wand and fix all disparities across visible populations tomorrow, it will still not fix the fact that huge income disparities exist between individuals. Google engineers and researchers get paid 5 times the median national income or more, and (senior) Google management in the 10x to 1000x range. The vast majority of the population is stuck in dead end precarious jobs, with little social mobility, one medical emergency from bankruptcy.
In causal modelling thou shallst not control by consequences of the causal treatment (the causal treatment here is being born black). Read your Rubin&Rosenbaum.
>> Gebru is the type of person who defines "bias" as anything that isn't sufficiently positive towards people who look like herself, not the usual definition of a deviation from reality as exists.
I haven't seen this "usual definition" of bias as a "deviation from reality" that you give anywhere before. Can you say where it is coming from?
The greatest struggle facing the tech elite today is how to use the proletariat's data to train an artificial intelligence that doesn't inherit the proletariat's beliefs.
If your data set is biased for the purpose of answering your question, then having more data doesn't make things better. In this case 'big data' just becomes 'big bias'. Go to wikipedia and read on Simpsons Paradox.
The first page was leaked. The environmental angle was a significant part of it, particularly the claim that environmental and financial costs ‘doubly punishes marginalized communities’.
What was this article from technology review - yes it gave insight into the contents of the paper but then barely made any conclusion or added anything about what that meant to the situation other than the very end saying the obvious maybe this cuts into Google's 'cash cow'?
If the authors aren't confident enough in their paper to release it publicly, why would I want to read someone else's (presumably inferior) summary of it?
This is a good point that I did not think of. Google maintains the paper was not up to their standard. The authors submitted the paper to a conference. And now, one of the author asks MIT Tech Review to not publish it because it is an early draft. Gives some credibility to Google's claim IMO.
> It will also fail to capture the language and the norms of countries and peoples that have less access to the internet and thus a smaller linguistic footprint online. The result is that AI-generated language will be homogenized, reflecting the practices of the richest countries and communities.
It seems amusing to read this when the same person and her followers hounded Yann LeCun for pointing out the same thing with image models.
Anyway, this seems interesting. But I am not sure how you solve this. Do we take representative dataset according to population of a place? Also, assuming this is limited to a single language. How can AI generated language account for nuances from regional differences at the same time in a common model. Isn't what the author asking for here is kind of train, en_US, en_GB, en_IN separately here. For things like completion don't language models already account for this?
That's a problem for lots of technologies. The native language in my country hardly has any literature published in it because it's so obscure. How should the printing press solve this unfairness? The internet in general also doesn't use it much at all, neither do all the popular movies and TV series. They don't even get subtitled. Apple Maps can't pronounce the street names correctly for navigation either. It's pretty hopeless but does it really matter?
Maybe if you have an unpopular language, that's just unfortunate and please encourage your kids to learn English (which they probably do at school just about everywhere anyway) so you don't perpetuate the same problem to the next generation.
Honestly question to fellow HN commenters: what is the popular take on energy efficiency against entertainment industry? Where do people draw the line, regarding what and what not should be an ethnical use case for energy?
See the other thread were one mentions that with reCaptcha Google learns that all cabs are yellow (and traffic lights hang over streets - they don't here). But I guess everyone only sees the blind spots of other people.
You're assuming that reCaptcha is training some Google algorithm. But in fact it has trained you -- to recognise US style dangly traffic lights, yellow school busses and cabs, pedestrian cross walks -- and this makes me wonder if reCaptcha does use the same image set for all different locations. Are people in Lagos, Pune and Tashkent routinely failing reCaptcha at higher rates because they don't watch US television? How fast do they learn what a school bus looks like?
I knew those already from being flooded for 40+ years with Hollywood films (all of them) and sitcoms, I know Cosby, Malcolm in the Middle, Beverly Hills 90210, Miami Vice, the Fall Guy, Seinfeld, Friends, Lou Grant, Mary Tyler Moore Show, Diff'rent Strokes, Cheers, Golden Girls, MAS*H (the best), Married... with Children, Three’s Company, King of Queens, Family Matters and on and on and on.
Basically I know the US better - or the image it projects - than my own country.
Funnily it was a huge letdown when I visited for the first time decades ago, as it felt just like a sitcom and there was nothing new really.
Those images being exported to the planet are not representative of the USA. Hollywood is mostly Jewish and Homosexual, those are the products of these minds instead. I doubt actual American programming would resemble any of these sitcoms.
Even in Europe I sometimes find reCAPTCHA more difficult for being "too American". Maybe we're categorised as "close enough" and there are south American or African datasets but I suspect not.
Well, she did Research while Google expected a PR piece. And it's obvious she couldn't fulfill Google's expectations, as the whole point of an ethics committee is to be critical, not just doing rubber-stamping.
Can I postulate that the problem with these kind of positions goes even deeper? Let's say we had a wonderful company which would create an unimpeachable ethics board and which enthusiastically endorsed the findings of said board. This is good for today but what about tomorrow? What about when the company comes up with a promising product which violates one of those ethics? What happens when the companies stock stagnates? What happens when the company is in financial trouble? What happens when the company is coming up with promising new products every day for years that could get it out of that financial trouble? What happens when ethnically minded managers have to argue day in and day out that we should let the company fail rather than dain to implement one of these profitable products which violates just a few of these ethics? Eventually, some combination of desperation and profit motive will break the damn.
And often it is a damn that need only break once. Corporate ethics as a whole seems contradictory. The only ethical rules which a company may obey in perpetuity are those that do not for long conflict with profit and those which are backed by effective laws.
Even when done with a positive intent, I fear creating positions such as this is begging for eventual corruption.
She didn’t “refuse to fix it when asked”. She agreed with the proviso that she could have a meeting to figure out what she was and was not permitted to publish.
The response was to decline to meet and fire her.
It’s entirely plausible that the paper is bunk; however, when someone is willing to come down that hard to prevent an idea being published, I tend to err on the side of “worth finding out what”.
> She didn’t “refuse to fix it when asked”. She agreed with the proviso that she could have a meeting to figure out what she was and was not permitted to publish.
I think that is in dispute. Jeff Dean claims she did refuse to fix it.
> It’s entirely plausible that the paper is bunk; however, when someone is willing to come down that hard to prevent an idea being published, I tend to err on the side of “worth finding out what”
But they weren't trying to prevent the idea from being published. They certainly knew by firing her they'd give it far more attention than it would otherwise get. What they wanted was not to put their name on this idea.
> She didn’t “refuse to fix it when asked”. She agreed with the proviso that she ...
Are you sure she agreed to fix it? According to her tweets she agreed to take her name off the paper, not to fix it, and only if they agreed to her conditions which included releasing the names of all the reviewers involved. From Jeff Dean's email:
> ... including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback
If I was in their position, I would be extremely uncomfortable releasing the names of the reviewers with the likely result that they would be doxxed by Gebru on Twitter
Ethics is inherently "PR, ideology, and politics" since the ethicist has to select a set of foundational moral principles to work off of but that set won't be universally shared by everyone.
To her defense, she was working on ethics in AI. What ethics to adopt inherently comes down to ideology.
While I don't really understand the emissions argument, the rest strikes me as very defensible. If the best language models need giant datasets to excel, it is very difficult to ensure your AI is trained on reasonable data.
I wouldn't want an AI therapist to be trained on 4chan. Now obviously nobody would be that stupid, but unfortunately it seems we are not far from it.
Unless we build models that rely on less data, it will be difficult to prevent problematic biases in the AI we put into production.
If that is the argument Timnit makes, I think that's the exact type of work I would expect from an AI ethics department. And good work at that.
You train your LM on web crawl data, but also train a 4chan classifier, then you condition your LM not to generate in 4chan style. GPT-3 got a similar chaperone classifier for offensive speech. It's like knowing swear words but choosing not to use them. You could also condition a general LM to bias and debias its outputs as you like.
> That's just outright false. She was terminated, effective immediately.
Without taking sides (I don't know the full story to understand who is objectively in the wrong here), that's not completely true.
She basically gave an ultimatum - meet my demands or I can work on a last date.[1] It is quite common at big tech that once you resign - officially or unofficially - you could be asked to leave effective immediately. Usually happens when you work in critical areas or moving to a competitor or a disgruntled employee.
Could this have been better handled? Maybe, but no matter who was in the right or wrong, she wasn't technically fired.
Firstly, there is ample evidence of her toxic behaviour online. (See the exchange with lecun on twitter, absolutely horrible). So its clear she has issues with how to collaborate and communicate.
She then goes on to threaten the company she works for. Like wtf? Irrespective of what you want to do in response, act like a professional. If you were a CEO/Manager etc and had someone with toxic behavior come in and threaten you if you didnt comply to their demands, wouldnt you go 'ok, see ya'.
I certainly would. Everyone is replaceable. Especially if youre toxic when it comes to dealing with situations you dont agree with
> If you were a CEO/Manager etc and had someone with toxic behavior come in and threaten you if you didnt comply to their demands, wouldnt you go 'ok, see ya'. I certainly would.
Which is fine, but that's firing them for acting up. It doesn't really matter what the demands are.
Ok, so basically she is a bullshit social engineer, masquerading as an 'AI ethics researcher'
"Gebru’s draft paper points out that the sheer resources required to build and sustain such large AI models means they tend to benefit wealthy organizations, while climate change hits marginalized communities hardest. “It is past time for researchers to prioritize energy efficiency and cost to reduce negative environmental impact and inequitable access to resources,” they write."
-- Has she calculated the opposite alternatives? What were the costs of the previous models? How about the alternative of not-having ai-backed search at all. Would worse results be actually worse energy wise? If you are not including the substitues/alternative costs it is just sloppy research, or just done purposely on bad faith.
"One is that shifts in language play an important role in social change; the MeToo and Black Lives Matter movements, for example, have tried to establish a new anti-sexist and anti-racist vocabulary. An AI model trained on vast swaths of the internet won’t be attuned to the nuances of this vocabulary and won’t produce or interpret language in line with these new cultural norms."
-- Ok, basically she is pissed as AI is picking normal language, that people use, and not using the vocabulary that is in vogue on certain political circles. Basically censure, and forced speech.
Why does google even bother at this point? It seems that certain circles just don't want to have positive constructive way, but it is becoming: "my way, or you are X", with "X" being whatever prerogative they will through at you.
I think social engineering b.s. should be kept as far away as possible from science. This is turning true ai research into a masquerade to push certain political agendas.
I feel google was being open minded, but the other party was not, and actually being malicious...
basically: instead of reform, and fixing whats broken; to tear it down, as it is just another means of oppression.... (yes, that's the mindset of the movement).
Her research, looks to be mean spirited, and made to discredit google, so it can be blackmailed into whatever the ideology wants.
It is not meant to fix/improve the situation, but used as political amo. Hence it is just a political paper/propaganda, and not proper ai research.
Hiring an activist and then firing them before a paper is published can also be viewed as expecting them to be your activist. Maybe Google wanted to be part of the anti-establishment movement as long as its part focused on competitors.
I find the sequence of decisions made by Google problematic.
Until they describe in intimate detail what their management has been planning to filter they are effectively retracting all research from their institution as having this unknown bias. Like a tabaco lab that blocks all results against their products. That this was shoddy means nothing unless someone would have caught a Google positive piece as shoddy and actually stopped it at this process. I would guess no.
The right thing to do was let it be published, get external critique and end the relationship if most people with no money in the game thought it was bad science.
Was this outcome that difficult to predict based on the hiring interviews, and based on prior articles and public statements? In other words, would this come as a surprise to a half decent hiring manager and hiring committee?
Yeah it seems like her paper was light in technical substance and high in social justice activism. Pointing out that google emits carbon is worth consideration in the appropriate venues but it doesn’t really advance the state of the art.
Yes I agree. A substantive analysis of carbon emission should look at multiple countries, multiple industries, multiple companies, and all the outputs from those companies to give a good picture of what types of outputs are the most inefficient w.r.t. carbon emission.
We haven't read her paper, just an article about it. We can't read it, because google won't let us. That you don't see that as a problem is a little concerning.
The article we are using as reference is a summary of her paper and is literally titled “we read [her] paper” from MIT Technology Review.
What do you suggest might the article be getting wrong about her paper? Or are you suggesting that the information contained about the paper in the article is inherently invalid and should be ignored altogether?
I'm saying that saying her paper was "light in technical substance" when we haven't read it is problematic. Any article such as this is going to be light on technical substance in comparison to the paper itself.
The article states her paper focused on the dangers of large language models: “Environmental and financial costs,” “Massive data, inscrutable models,” and “Research opportunity costs.”
The dangers of large language models is an interesting topic but it’s not AI research and it doesn’t advance the state of the art of AI.
When it comes to technical substance in the field of AI, her paper is indeed lacking in that unless the article completely failed to mention something that would directly qualify as original AI research.
Additionally, according to TFA, it's not Google who don't want the article published. "Bender asked us not to publish the paper itself because the authors didn’t want such an early draft circulating online". Though they were happy to present it at a conference just one day later.
hmm... not sure what you are talking about, but you are an anonymous person, with no history... retorting to bullying and name calling... not sure how you are contributing to the conversation.
What do you mean by pretty serious? I looked at three, two are about better documenting ML models/data, and one is about inferring the political leanings of an area by what type of cars are parked in it. None seem particularly groundbreaking to me...
The paper referenced ("Gender shades: Intersectional accuracy disparities in commercial gender classification") has been cited 1000+ times per her Google Scholar page (https://scholar.google.com/citations?user=lemnAcwAAAAJ). For a 2-year old paper, this is easily a top 1% most cited paper.
You can disagree with how she and/or Google has handled this whole situation, but please do not denigrate work that has been cited (https://scholar.google.com/scholar?oi=bibs&hl=en&cites=14954...) by papers accepted at the most competitive/prestigious ML conferences.
EDIT: I also do not see how in good faith you can say that VentureBeat, a company who makes the bulk of its revenue from running conferences catering to C-suite execs who can shell out thousands of dollars for a ticket, is "leftist".
We see that on average, tech founders are less likely to support vs. even Democrats generally (not just progressives):
* Banning the Keystone XL pipeline (60% vs 78%)
* The individual healthcare mandate (59% vs 70%)
* Labor unions being good (29% vs 73%)
This is to say, the average Silicon Valley type, particularly the C-suite exec or founder, tends not to be on the left wing of the Democratic party.
During the 2020 Democratic primary, even the Silicon Valley billionaires who are openly Democratic-leaning donated to candidates who were not to the left of the field (i.e. Elizabeth Warren and Bernie Sanders) (https://www.cnbc.com/2019/08/13/2020-democratic-presidential...):
* Eric Schmidt -> Cory Booker and Joe Biden
* Reed Hastings -> Pete Buttigieg
* Marc Benioff -> Cory Booker, Kamala Harris, and Jay Inslee
I'm engaging with you in good faith, and because I was intrigued that in a previous comment you mentioned that you live in Spain (though who's to say you're not a US ex-pat). But calling US tech companies "leftist" is a stretch at best.
"Left" and "right" carry little meaning when it comes to analyzing the divide between people. The "with/without money" bit is of a much higher order than the global "left/right" bit to me.
I don’t really care about the paper author’s fate, but it seems unsettling to me that she is discussed more than the paper itself.
Honestly? I don't have much experience with ML and in particular issues of bias in ML (though I have a general understanding of a lot of the former), so I don't really feel qualified to evaluate her more recent work, but her older stuff definitely appeared like what I'd expect from a machine learning practitioner, so I don't think attacking her as someone that only understands social sciences (which I perceived the parent as doing) to be fair.
Your argument sounds like a red herring. Qualified for what? I didn't noticed the OP making any remark on the author's qualifications.
And by the way, you should think things through before pulling appeals to authority to try to defend the credibility of researchers. See James Watson, a nobel laureate and a founder of modern genetics, and his comments on race and gender.
If you take a step back and read the snippet you cherry-picked, I believe you'll be able to understand that it referred to how the author is trying to pass opinions on social engineering under the guise of research on AI ethics.
That does not mean at all that the author lacks qualifications.
> When you accuse someone of pretending to be an "AI ethics researcher" (...)
You would do well to work on your literacy and reading comprehension skills.
To start off, be aware that "a bullshit social engineer masquerading as a AI research" does not, by any means, mean the author is "pretending to be an AI researcher".
> (...) if it's not meant to imply "pretending to be"?
It means "social engineering pretending to be AI research", doesn't it?
Think about it.
Look at the context. Look at the people involved. Look at the topic of the discussion.
And let's be honest for a second: do you personally believe that anyone would accuse an AI researcher who worked at Google of not being an AI researcher?
Literacy matters, and so does honest and well meaning interpretations of what someone actually did said.
"an action or appearance that is mere disguise or show"
As in social engineering masquerading as AI research?
Honestly, is there any room for doubt at this stage?
I don't expect honest and well-meaning people to continue insisting on this at this point. It feels that these attempts to distort what was said are simply attempts to attack the OP based on misreprentations of what he did said.
> If you take a step back and read the snippet you cherry-picked
Cherry-picked? It's literally the first sentence, the central thesis of OP's entire "argument". If they can't get that down on paper correctly, they're not worth taking seriously.
> You would do well to work on your literacy and reading comprehension skills.
Ascribing unsubstantiated sentiments to someone's post in lieu of what they actually said doesn't fall under literacy or reading comprehension, sorry. It's actually the polar opposite of that.
Why are you claiming to know OP's intentions (despite their own words) better than everyone else here? Is OP your alt account?
> I don't expect honest and well-meaning people to continue insisting on this at this point. It feels that these attempts to distort what was said are simply attempts to attack the OP based on misreprentations of what he did said.
It's extremely bizarre that you're questioning the literacy of people when you're projecting your own unsupported sentiment to OP's post while simultaneously claiming that people who are directly quoting the OP's post verbatim are "misrepresenting" what OP said.
You're being very disingenuous here (as well as borderline manipulative) by trying to shut down any dissent against your flawed position by questioning the good faith of the people doing nothing but directly quoting the OP's words.
>If they can't get that down on paper correctly, they're not worth taking seriously.
Amusingly, the OP got shirty at someone* further down this thread for calling them names, but with an unfortunate textual blunder which exposed the hypocrisy of their own name calling (not to mention total lack of self awareness).
I pointed this out but got downvoted to oblivion of course. Classic goose / gander politics.
* (and rightly so btw, that kind of conduct has no place on HN)
I sincerely think that your CEOs of large companies are much less powerful than one would naturally think. They have money and power, but much of it is based on a group of people sympathetic to some pretty extreme views approving of you.
We call it “FU money,” but these guys aren’t satisfied with being able to say “FU” and still live comfortably; they want to be in charge of things that are seen as significant.
I believe it will always take more than money to get thousands of largely independent adults to do what you want.
I think you’re making a weaker version of this argument than you need to, because you’re framing it in terms that oppose the politics of half of the public, when you really only need to argue against a much more specific set of ideas.
For example:
> Ok, basically she is pissed as AI is picking normal language, that people use, and not using the vocabulary that is in vogue on certain political circles. Basically censure, and forced speech.
I think a stronger version of this argument, which can appeal more broadly, would be: an automated system that is constantly being trained on data from social media can more quickly pick up new trends in human speech than a manually-programmed expert system. Furthermore, even if humans are constantly updating an expert system, the biases inherent in the composition of the company-hired set of experts will limit the representation of minority trends in the data. On the other hand, a model examining the entire data set can pick these up automatically. For example, I can easily get AI Dungeon (powered by GPT-3) to communicate using language and concepts that are only present in a minority community that I’m part of that I’ve rarely seen represented in literature.
When arguing against Timnit Gebru or their ilk, you don’t need to just give them the support of liberals. We actually make up the majority of techies, so it’s a bad position to put yourself in if you want your cause to succeed.
> Furthermore, even if humans are constantly updating an expert system, the biases inherent in the composition of the company-hired set of experts will limit the representation of minority trends in the data.
Exactly this. Framing it in terms of alternate options makes AI the more reasonable choice.
Google AI appears to have drawn unnecessary attention to themselves for relatively benign paper. Most of the arguments in this paper have already been discussed elsewhere and could be refuted in simple terms.
That really depends on the context you think about this paper in. If you see it as Critical race theory activism (which I do) then it's not benign at all on the contrary it's deeply damaging that Google have well paid people who think they should push their own political agendas through an organization that underlines almost everyone in the wests daily lives.
If you see AI as a technological amplifier, it could backfire the other way around. I.e. that the status quo is kept longer than necessary because it is so baked into the technology itself, not only the people who wield it.
I think it is precisely for these reasons this debate is so important, because it is truly not a clear cut improvement in my opinion.
They didn't say they shouldn't make them, they said "prioritize energy efficiency".
normal language, that people use
As they said, language changes. Do we want a model that acts like the average person of the last 20 years? Or do we want a model that acts like a person who has just been through 2020?
> Do we want a model that acts like the average person of the last 20 years? Or do we want a model that acts like a person who has just been through 2020?
You're free to use the model you believe is best for your goals.
I you use real world data from the last 20 years to generate the model then your model will reflect an amalgamation of the real world data collected in the last 20 years.
If you prefer to focus on last month then go ahead. You'll have less real world data and seasonal patterns will have a deeper impact and the model will be less robust, but go right ahead.
If, instead, you want a model that is not generated from real world data and instead just uses your own artificial data that is blatantly and openly manipulated to reflect your personal biases and political preferences then just call the spade a spade.
>-- Has she calculated the oposite alternatives? People driving to libraries, to search for something?
Driving to the library, that is the alternative to training these models, really?
> -- Ok, basically she is pissed as AI is picking normal language, that people use, and not using the vocabulary that is in vogue on certain political circles. Basically censure, and forced speech.
If you had a model trained on a large corpus of data from the pre civil war southern American states, it would have been deeply racist, and would even view black people as possible property. If you had one that was trained on data from the 1950 it would be less racist but still problematic viewed by people from today.
Is there really something special with today, that removes these kind of concerns with a model trained with current data?
> I think social engineering b.s. should be kept as far away as possible from science. This is turning true ai research into a masquerade to push certain political agendas.
It seems to me that it would be impossible to do any social science research, and specifically any research on racism. With this kind of attitude.
Some of the concerns brought up in the paper seems less consequential than others. Especially the pollution one seems weak to me, that doesn't make it false, and fair enough that is was brought up. I find the racism issue a lot more problematic, and is something I've run up to working on deep learning solutions myself. Even if it worked fine for my group, and most of our customers, that is definitely not fun, and something practitioners should consider when building these things.
> If you had a model trained on a large corpus of data from the pre civil war southern American states, it would have been deeply racist, and would even view black people as possible property. If you had one that was trained on data from the 1950 it would be less racist but still problematic viewed by people from today. Is there really something special with today, that removes these kind of concerns with a model trained with current data?
I think this argument applies not just to machine learning, but to learning in general. Any kind of knowledge-acquisition process is going to be biased by the environment in which it occurs. That goes not just for digital neural networks, but also those in our human brains, operating on the same racist data the ML models are. If that means we shouldn’t do machine learning, it also means we shouldn’t do human learning either.
Of course, the preceding is absurd. A more reasonable take is that we should adjust the objective function of our learning processes to try to account for the effects of biases. We try to do that subjectively as any decent person operating in a biased society should, but our ML models can do it more accurately. In fact, I’d argue that such techniques are necessary to more carefully analyze and build evidence describing the effects of those biases. They can provide insights that will even improve our ability to correct for biases in the real world.
> It seems to me that it would be impossible to do any social science research, and specifically any research on racism. With this kind of attitude.
I think this sort of veiled personal attack resorting to baseless extrapolation is not a productive line of public discourse.
You should take a step back and think about a) the field of study, b) how the main argument completely ignores the basis of said technical field and what it studies, c) how the argument being created lies on the idea that a self-annointed elite should have the right to manipulate the public to force it to fall in place with it's goals and desires.
> I think this sort of veiled personal attack resorting to baseless extrapolation is not a productive line of public discourse.
I thought that my comment was fitting to the tone of the text I replied too, but fair enough, maybe I shouldn't have included that paragraph.
> You should take a step back and think about a) the field of study, b) how the main argument completely ignores the basis of said technical field and what it studies, c) how the argument being created lies on the idea that a self-annointed elite should have the right to manipulate the public to force it to fall in place with it's goals and desires.
It is unclear to me what this means, is it arguing that studying bias in AI and specifically deep learning is not germane?
Is it arguing that it is not acceptable to institute policies and action to create a level playing field for minorities and repressed group because that would be social engineering, and forcing the common man to do the elites bidding?
If that is what it means, I have to disagree, I find both of those things worthwhile.
> I thought that my comment was fitting to the tone of the text (...)
Yes, that was one of the problems.
> It is unclear to me what this means, is it arguing that studying bias in AI and specifically deep learning is not germane?
Let me make it clear for you so that a) we are able to talk about things objectively, b) your options to continue using veiled personal attacks is curtailed.
Either your goal is to model reality and real life, or your goal is to model your idealization of what you feel real life should be.
If you pick option #2 then your model does not reflect real life.
Bias is by definition the way the model returns results that don't match the real world and real life, in frequency and in proportion.
If your goal is to use your model to manipulate and control society based on your own personal criteria, by manipulating it to return results that distort the real world and real life, then call it something else, because bias is not it.
I'm not OP, but what is "real life"? In cases where some communities/countries have (to quote from the article) a "smaller linguistic footprint online", is trying to control for this a form of manipulation of society or a way to fix a bias? Can you actually choose a sampling measure without making some sort of value judgement?
In the context of creating models, it's representative data collected from statistically significant observations of a population, for starters.
> In cases where some communities/countries have (to quote from the article) a "smaller linguistic footprint online", is trying to control for this a form of manipulation of society or a way to fix a bias?
In modelling there is no such thing as control. There's the input data and there's the model generated from input data.
If you are looking for a model that is expected to represent a property intrinsic to a specific community then you use data collected from that community to generate that model. That's it.
Models generalize, and reflect the norm. They are like that by design. That's their point.
If your plan is to have a model that does not reflect the input data but instead forces your biases regardless of the input data then your goal is not to model reality but to distort it to comply with your personal goals.
> Either your goal is to model reality and real life, or your goal is to model your idealization of what you feel real life should be.
> If you pick option #2 then your model does not reflect real life.
These deep learning models built by corporations are not scientific models, they are engineering solutions, built to solve problems. Reflecting the real world is only useful if it furthers what the company wants to solve. If they for example remove swear words from their training set, that will make them a less accurate model of the world, but make them more useful for building solutions. But it is probably a trade of they would be happy with. We've also seen example of risk scoring application for felons that seem to end up doing racial profiling, because that is what the data seem to indicate makes sense. But that's deeply problematic and runs counter to laws in some places and seem ethically problematic (https://www.theverge.com/2020/6/24/21301465/ai-machine-learn...).
> Bias is by definition the way the model returns results that don't match the real world and real life, in frequency and in proportion.
Getting a good data set without bias is hard, even if you crawl the whole internet like Google does. Not everything is on the internet and there are systematic drivers that make some part of the human condition over represented (English, science, the views of the affluent and educated), and some under presented (small languages, the discourse of people behind the Chinese great firewall, the poor). So just getting a ginormous data set does not fix bias.
> If your goal is to use your model to manipulate and control society based on your own personal criteria, by manipulating it to return results that distort the real world and real life, then call it something else, because bias is not it.
Positive bias is absolutely something that we use, and while it might seem sinister it does not have to be. The example I'm most familiar with a facial recognition technology. Most groups building that ends up with a model that is better at some groups than others. Asian research groups often end up with models that does well with asians and worse with whites, while European groups usually end up with the reverse. In some sense these results do reflect the reality of these groups, most people in Europe is white and most people Asian is asians, so that you training sets ends up like that is not surprising. But no one is happy with these kind of results, and everyone wants to fix that.
To bring it back to speech and text models, let's say you are building a customer service solutions incorporating a deep learning model, the reality might be that you current customer service representatives treat blacks (or people who use "black" dialects), worse than people who sound white. An accurate model built on this data set will then also do that. But is that acceptable? I hope most companies would want to fix that, and be fine with adding some positive bias in their solution.
The social engineering and woke culture stuff seems like a moot point honestly.
If the large language model analyzes everything, it will veer towards normal. It doesn’t seem so far fetched to think they can build something which also tracks and understands language trends and slang, and how to appropriately use this “special vocabulary”.
I don’t know much about how these models work, but I would hope something trained on the entire internet is smart enough to know to speak to me in modern English, not Civil War English or Shakespeare English.
> "One is that shifts in language play an important role in social change; the MeToo and Black Lives Matter movements, for example, have tried to establish a new anti-sexist and anti-racist vocabulary. An AI model trained on vast swaths of the internet won’t be attuned to the nuances of this vocabulary and won’t produce or interpret language in line with these new cultural norms."
This was just glossed over and maybe slightly off-topic, but what kind of shifts in language did these two movements in particular affect?
Stuff like #MeToo was about raising awareness about sexual abuse, particularly in the entertainment industry, can't really think of how it forced people to change words that they use.
The best example I can think of is people opting to not use "master/slave"... Which is much more debatable and far off from "core tenets" of the BLM movement like police reform and dealing with systemic racism.
I guess what I am trying to say is that when it comes to language, things are a lot less black/white (heh) and we should be careful about people that are setting themselves up as arbiters. Sort of a who guards, the guardians issue I suppose.
> Ok, basically she is pissed as AI is picking normal language, that people use, and not using the vocabulary that is in vogue on certain political circles. Basically censure, and forced speech.
The thing is: The language that people use is often enough offensive: racial slurs, sexism and other forms of discrimination, plus seemingly innocent "code words" like "globalist".
AI developers should know about these problems and keep them in mind while developing training data for AI models. The consequences of ignoring such bias is continuing discrimination - with the most obvious and low-tech reminder being the "racist soap dispenser" which had only been trained on white people's skin and failed to dispense soap for people of color.
The danger in such discriminatory models is that while it's an annoyance that a soap dispenser does not dispense soap for a person of color, with algorithms running more and more of our daily lives, the impact is way higher: whether being refused credit or affordable insurance rates because an AI "thinks" the applicant is black and living in a black, poor neighborhood is grounds enough for that, or "predictive policing" AIs sending in massive police responses to everyday calls, this directly hits on people's survival without human oversight.
That is why ethics in AI are so important and why "social justice activism" is so desperately needed!
I think your “social justice” cause(s) would be much better served if you didn’t make up problems where there are none (what’s the issue with “globalist”??) and didn’t project ill intent where there is none (the soap dispenser wasn’t “racist”, it was just “bad”).
It’s just basic intellectual honesty, sorely missing in the SJW circles.
> the soap dispenser wasn’t “racist”, it was just “bad”
For a person of color who wants to wash their hand and sees that it dispenses soap for white people but not for them it is yet another piece of everyday racism.
Globalist probably can be used as a dog-whistle, but we do need a word for people who support free trade, free migration, borderless finance etc. That’s a quite common position. Indeed, the Haaretz article you link to is considerably more ambiguous than just saying “globalism is a bad word and must be banned “.
I think that “dog-whistle” is a dog-whistle for wokeist people who prefer to insult others as racists instead of making actual rational arguments.
As for “globalist”, I don’t see an issue with criticising globalism and those who promote it, be they Jews or others. Clearly a lot of Jews (in Israel) are very “localist”, and I suppose there are some that are “globalist” as well. So?
Just because a view is also supported by some or other “vile” people (that are almost always a tiny minority, if disproportionately loud and/or salient), doesn’t make that view invalid or immoral.
The soap dispenser clearly didn’t have racist thoughts. But it effectively worked worse for black people, so it has racially discriminatory effects. I don’t think there’s much to argue about here.
Also, I believe it’s best not to accuse your interlocutor of lacking intellectual honesty unless the evidence is very clear. Perhaps they just see the world differently to you.
algorithm is biased => we should fix it is reasonable, but also a misrepresentation of the argument that this whole thread is about, which is more along the lines of algorithm is biased => researchers are racist => the company that employs them is racist => the whole country is racist which definitely is driven more by ideology than reason.
I’m happy about people viewing the world differently and having an honest, good-faith discussion about the issues. But I’ll criticise intellectually dishonest arguments, like the moat-and-bailey you just pulled here.
I never argued for that. My point is that researchers (as well as everyone!) have biases, which may manifest in the end as racism or other forms of discriminations. To prevent these biases from manifesting, (AI) ethics experts are needed as oversight.
> the company that employs them is racist
To take the example of the "racist soap dispenser" again: while the individual people who have developed it may not have been racist, they have failed to think outside their biases and ask themselves "can people with non-white skin use the product?". Also, management has failed here, because more diverse staff would have resulted in at least one internal test candidate of color noticing "hey, I can't get soap!".
The result of this lack of diversity and out-of-the-ordinary thinking was that a person of color was rightfully offended at not being dispensed soap. Therefore, the company acted racist even if it never intended to do so.
> the whole country is racist
Just short of half of the country recently elected a President who openly spouted white-supremacist conspiracy theories. For half the people in the US racism is not a dealbreaker for the highest office it has to offer!
> Just short of half of the country recently elected a President who openly spouted white-supremacist conspiracy theories. For half the people in the US racism is not a dealbreaker for the highest office it has to offer!
... which is a bit different from saying that half the country is racist. Perhaps they thought the alternative was worse.
> Just short of half of the country recently elected a President who openly spouted white-supremacist conspiracy theories.
He was elected by the votes of ~27% of eligible voters, and about ~20% of the population, not “just short of half the country”.
You could say it was a little less than half the people who felt it is worthwhile to vote in a system where votes matter so little that getting the most of them doesn't mean you win, but then the story is about alienation from the electoral system more than support or even indifference to racism.
> The language that people use is often enough offensive
Please define "often". And "offensive" (as in to whom). And of course, you also need to factor in whether being offended is justified.
As an extreme counterexample, a deeply racist person would be offended by a mixed couple. And racists on both sides currently are. I see no reason whatsoever to pander to either side on this, so just there being someone who is or claims to be offended is no guide.
If you find "globalist" offensive, then you are the problem. And, the vast majority of Americans are find this type of PC culture and social justice activism offensive, including the vast majority of "marginalized" groups.
I am honestly disappointed to see this kind of response at the top of comments here. Every time we have some sort of social issue come up people here come in to pick apart the person involved, and I think it’s one of the bigger reasons why some maintain a generally negative view of Hacker News. Can you imagine being Timnit and stopping by this thread, where the top comment, in colorful language, reduces you to a social activist fraud? A comment coming from someone who hasn’t even read the full paper? One that ignores the job you were hiring to do (which seems precisely to draw out the social issues of AI) and lambasts it?
At you the commenter yourself: you seem to be exceptionally harsh on her based on the most minimal information that you’ve purposes to fit your claim of this being “useless things that we shouldn’t care about”. Do you know how much energy goes into training a model? I certain didn’t. Do you think that showing that AI is picking up discriminatory language is something worth looking into? I certainly think having someone look into it would be useful, yes. And Google apparently thought so too. Claiming that these things are idiotic and attacking the author of this paper for bad faith and censure is…very extreme.
Have you read her exchange with Lecun and the impact it had?
Edit - Also, if her behavior online is like that, imagine what its like inside Google? Have you thought about that? Clearly her email to the brain group was an issue here, i wonder how it compared to her exchange with lecun?
One final edit, even her co-authors were not ready for publishing the paper (per article), so why is the reason of censorship, supression etc even being discussed. This whole thing has been blown out of proportion and Google likely did the right thing, purely based on her behavior
Given that she seems relatively popular within Brain - more importantly her collaborators and direct manager, why should we assume she fostered a toxic work environment with those around her? More like, it appears the people who wanted her out were leadership and a set of anonymous PR/IP reviewers she does not know.
Her behavior is clear evidence of how she interacts with people. You can be a brilliant jerk and still get along with some people. In a team environment however, there is no place for this. Her exchange with Lecun, her threatening words and many of the way she puts forward her arguments. Even if she was in the right here, she can easily deal with it without doing what she did here.
So yes, there is more evidence for her toxic behavior than there is for her ability deal with disagreements professionally and work in a team environment.
TBH, I think this whole review things is just a reason for google letting her go. They were probably looking for a reason and she using her threatening (aka toxic) nature was just what google needed.
Again, I repeat, if her team had issues with her, her direct report would know and her team wouldn't be harping constant praise towards her.
The people who removed her are executives several skips above her. That is unusual.
You keep bringing up LeCun, but LeCun did not get harmed as you imagine - he only conceded to her point because she was frankly the expert in the topic of discussion and he finally acknowledged that.
Because thats not how I have seen it. She threatened to leave. If any of my team had an issue, and their response 'Do X, Y, X or ill leave', even if they are in the right, I would not align with them purely out of the fact that that is not the right behavior. You can still leave any company with out threats.
Also we have not seen the email to the brain group. I wonder how that compared to her exchange with Lecun and her threatening nature.
Lecun was not harmed? How did she contribute to a healthy debate there? Please also link me. Because all I saw were aggressive tones towards someone she disagreed with (even if she was in the right). Again, he behavior (not if she was right or not) is the problem here. You can be right, but still be an asshole.
Also, how do you know some people in her team didnt have problems with her? Do you have here perf reports? I believe her direct manager is not the only person that can fire her. She can be ok good terms with her direct manager and still behave unacceptably towards higher ups
2. How was LeCun harmed? As far as I'm concerned, he accepted the feedback. And just so we're clear on what happened then - Gebru was only one of many with qualifications who were aggressively critical because of his abdication of research responsibilities. I don't know why you're so focused on Gebru in particular.
3. Yes, if this was about performance, her direct manager who oversees 300 people would know.
4. So yes, glad to see we agree higher ups, who she likely has little to no interaction with to supposedly be exhausted of her behaviors, stepped in to get rid of her.
1. Again, Managers are not the only person nor do they have to wait for someone’s direct manager to fire someone. If someone’s behavior is not acceptable, anyone authorized person in the company can fire them. Your link is irrelevant to facts.
2. Do you really think lecun wouldn’t have left Twitter if he didn’t find that conversation enjoyable? Do you discount the many other people who found her interactions with lecun horrible? Did any of the other people behave as arrogantly towards lecun as she did? Again, no point made.
3. See 1
4. See 1
So, you can either stop ignoring the fact that her own behavior led to this, or you can continue to think many other respected people in the industry who act way more professionally than her are just against her because they don’t support her views or think google is racist.
That’s not the point. Being removed by executives you rarely interact with because they want you to retract a paper is extremely suspicious. And the community has responded with disappointment.
You are the only person postulating the intention of her firing. I am merely calling out your assumptions because the consensus from the entire community involved has declared otherwise.
Most other people who found her refutations horrible are outsiders who don’t fully grasp the topic LeCun and Gebru and everyone else was alluding to.
Who are the respected in the industry that are against her? Not even LeCun, who has said he thinks her work is important. Nothing in your post has facts at all.
Again, you are assuming that this incident was the only thing that led to her firing. If she behaves like this in public, how does she behave and interact with individuals inside the company? It doesnt matter if she interacts with senior management. If I saw someone i never interacted with harass someone, as a manager, should I just let her direct manager deal with it? Absolutely not. HR would fire that person if it was deemed against expected management behaviour. Google even explicitly stated that she was fired for unexpected management behavior.
You dont have to grasp a topic to know if someone is debating respectfully. I dont have to be a doctor to know that the person is an asshole if they act like an asshole towards me.
I think lets just agree to disagree. Focus on the behavior. Not what she is writing about in her papers. And imaging someone acted like that towards you in a workplace. Imagine you are a CEO or a manager and someone comes to you and says 'Do x y or z otherwise im leaving'. Id be like wtf, sure, if thats how you deal with disagreements, cya'. Everyone is replaceable especially toxic people.
I am only working with what has been corroborated by Google, her teammates, and Dean. There has been no indication whatsoever, thus far, by any account - anonymous or not - of her interactions OR behaviors being problematic in any sense with any single individual. Now, you have gone as far as making up imaginary hypotheticals and blatant accusations - that not a single person has made - of harassment. You are not a person who argues facts - only personal biases and feelings about how you perceive a twitter conversation went down - one that peacefully discontinued.
How can you objectively comment on her behavior if you have never interacted with the person ever?
This has been discussed in prior threads. A manager is well within his rights to act upon his decisions emotionally. Employees are well within their rights to express concern. If I work in a workplace with severe safety issues, I would most definitely demand an ultimatum for my personal occupational security. Managers DO NOT impulsively fire individuals unless they are mentally unable to navigate through a conflict. It's far more nuanced than that.
"Following my posts of the last week, I'd like to ask everyone to please stop attacking each other via Twitter or other means.
In particular, I'd like everyone to please stop attacking @timnitGebru and everyone who has been critical of my posts."
Was I pointing out what Lecun said or her specific way of handling that interaction? Its the latter thats toxic. If people were attacking Timnit for that interaction why is that? Clearly she did not handle that very well.
I think if she is to be judged on whether she was being toxic to Lecun or not, Lecun's opinion is relevant. Why do you feel it's not? And if his opinion isn't relevant, why should we care about yours?
I also find your question about why people attacked her, as if people in the internet always have good reasons to attack others, is an incredibly silly attempt to appeal to the Bandwagon fallacy.
Lecun is way more of a professional than her. He handled it the way it should be, do you think lecun is going to come out and say something other than what he did as a respected individual in the industry?
Now go and look at how she has handled it and how she argued with Lecun. Then she also thratens people if she does not get her way. There is ample evidence for her behavior. Does she behave like this all the time? Of course probably not. But the fact that she is openly behaving like this is a clear data point. If you can not see this behavior as not suitable in a work environment, then lets hope we never work together.
They were continuing to work on the paper for publication, which, if you've never written an academic paper, is a lengthy process. There's a heck of a difference between not being done and not wanting a rough draft circulating vs being asked to withdraw the paper entirely.
Based on previous discussions, she bullied facebook's AI chief, Yann LeCun, who is apparently a respected expert in the field, off of twitter in an attempted cancellation.
Lecun is way more an adult and professional than she is. I dont expect anything other response from Lecun. Now, lets turn the tables, have you ever seen a response like that from the person in question here?
If you’re referring to her spat on Twitter with Yann LeCun, I am aware of it although not particularly well versed in its details. Certainly not well enough that I could claim it was “toxic”, although maybe that would be an accurate classification of her behavior. Regardless, her toxic behavior elsewhere is not an excuse for toxic behavior here.
It was my impression that Twitter was mostly about spreading false negativity. Anyway, I would prefer we discuss the content of the paper and what it implies for large models.
I’m not asking you to spread false positivity, I’m asking you to not read quotes of a rereading of a paper by an author who is not the person we’re discussing and then using it to complain that a researcher is useless and that her entire field is useless.
A comment that started off with “it seems like Timnit Gebru mat have overestimated the impact…” is not false positivity.
Sometimes researchers investigate boring questions like "what is the carbon emissions impact of doing this thing?" Just because you don't think the question is interesting doesn't mean the answers aren't relevant to somebody, or that finding some answers isn't increasing our understanding of the field in some way.
There are all kinds of research papers; they aren't always describing some new clever trick no one thought of before. Sometimes it's just "hey, we had this question we wanted to know the answer to and so we did some investigation and some math and figured it out." As long as they're questions that some other people somewhere are asking, there's no reason why a respectable conference or journal wouldn't accept such a paper. It's up to them and the reviewers to decide whether the content is a good fit for that venue. If it isn't, it'll get rejected; no reason for Google to involve themselves in the decision.
How rigorously the authors treated these questions in the paper is something we don't know without being able to see the paper itself.
> This is turning true ai research into a masquerade to push certain political agendas.
Spot on. Naive scientism is used as cloak to advance political agendas, instead of searching for truth:
"When a domain becomes respected source of Truth, there is incentive for hijackers. There is a huge window of opportunity (from decades to centuries) until the public opinion figures it out. Many (not all) still trust Harvard credentials, as Cardinal credentials in the past." [0]
This is the trend for "Believe in Science", a magical catchphrase evoked to put trust in frauds like Steven Pinker and Paul Krugman:
Calling Stephen Pinker a fraud is way over-the-top. He’s published a large number of extremely serious papers, including the paper introducing the ngrams dataset you just linked to.
He got challenged on his statistics [0], and didn't engage, correct or retract. This is antithetical to scholarly behaviour. Pinker is an ideologue and absolutely "the Mozart of bullshit vendors".
While I understand what are you trying to say (the manuscript in question exhibits low empiricism), I don't think you fully understand different kinds of research that exist (you don't have to agree with it, but more helpful would be to take it appropriately according to how serious the study is).
Please take a look at (Wieringa, R., Maiden, N., Mead, N., & Rolland, C. (2005). Requirements engineering paper classification and evaluation criteria: A proposal and a discussion. Requirements Engineering, 11(1), 102–107) available here: http://www.cse.chalmers.se/~feldt/advice/wieringa_2006_re_pa... Specifically read the section 3 (which is on p. 4). There you will find that research in software engineering can be broadly grouped into the following categories: Evaluation research, Proposal of solution, Validation research, Philosophical papers, Opinion papers, Personal experience papers.
While I agree that the usual way to deal with the criticism of the work is to redo the work with better evaluation (and thus, more strongly supported arguments), I think that in any ethics research you'd need more than just publications that fall under Evaluation research (which is what you are likely referring to as "true AI research").
This is an shameful comment to read on this matter regarding a real, well-known, and serious researcher. It’s the worst kind of tech-bro middlebrow dismissal that makes no interesting comment on the content in question beyond some ill-informed shallow analysis, and assumes that years of making half-baked shitty web-apps have somehow gifted you insight into AI ethics that’s enough to decide what research is “real” and what’s not - such that anyone who isn’t aligned with your high-school level of meta-philosophy is best ignored.
Your goal here wasn’t to make any useful point about how it was actually a reasonable decision for her to resign/be fired from Google. It wasn’t to comment on the process or difficulty of having these discussions at scale or in public, or even respond to the specific points raised in the article. These would all be valid discussions to have. It was specifically a personal attack based on a paper you haven’t actually read intended to demonstrate how much better you are than everyone else.
682 comments
[ 3.1 ms ] story [ 340 ms ] threadSo... they're saying it used about $100 worth of electricity.
[ https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 ]
[ https://www.statista.com/statistics/190680/us-industrial-con... ]
https://sustainability.google/commitments/
It is like someone complaining that google map uses x, amount of energy, yet the old alternative (printing maps, and getting lost) was much worse CO2 wise.
It seems obvious that this would be something an ethicist would be interested in researching to figure out what the impact will be and start discussions about how to offset it.
The total amount of electricity being used so far for training models does not yet move the needle in a significant way, but the point of the paper is that it's currently growing in an unbounded exponential fashion. And you know how exponential functions work.
Models like BERT aren't just trained once either when they are developed, but trained again with different domains, different parameters, different tasks in some cases. There is also fine-tuning (more frequent, less carbon intendive), so these are real environmental problems, and others have pointed them out.
Now about the billion cars on the road and the 40% of the world’s electricity being generated from coal.
How much more of a problem are a billion cars and 40% of our electricity being generated from coal?
We’ve squandered decades ignoring the big problems and now people want to run into the weeds with a thousand little problems, that individually don’t amount to much.
Wasn't Google prioritizing green energy for their datacenters?
Google is the largest corporate purchaser of renewable energy in the world
The whole point of having AI ethicists is to identify current indicators of potential future ethical problems so that they can be considered in guiding the direction of development, so that you minimize acute ethical crisis.
You realize the shear scale of co2 output the office the engineers who wrote the model, drove to work, education, etc produced.
I’m actually suprised just how small of a co2 output it was.
Sure, the issue is that the scale of models is increasing by orders of magnitude in a fairly short span of years, as well as the range of applications rapidly expanding. It doesn't take long for that kind of growth to go from a trivial issue to catastrophic one, and it's the exact kind of risk you have ethicists in a field to call out while it still is trivial so that some of the energy of people doing technical development gets directed to mitigate the risk of it ever reaching the catastrophic stage.
Capitalism actually mitigates the risk. Companies won’t pay 1 million in R&D to replace 100k of salary.
In an ideal world, nobody would get hung up on details and everybody would understand that there is a lot of nuance when comparing things. In practice, if Google published a paper which quoted the Strubell paper without the caveats, I can see headlines about how inefficient and bad for the environment Google Translate is. PR would get busy and obtain corrections or follow-up articles to clarify things, but those rarely get the same attention. And it's still extra work that could have been avoided, which in itself is bad optics ("do you folks even review the stuff you send out for publication?").
I'm all for reducing emissions and improving efficiency, but I find the premise a bit of a stretch.
Yes, large and wealthy organizations have a big advantage, but that applies to pretty much anything they do, not just language models. Inefficiencies are bad, but if you ask a number of people why fix them, I think they'd mention financial cost and wasted time well before looking at it as an issue of ethics and fairness.
Reducing costs for language models is already a great idea across all fronts. So is reducing inequalities. It's linking the two that sounds like a strained argument to me. Suppose someone makes models ten times smaller next week. Will marginalized communities' lives improve soon? There must be more. I haven't seen the paper, so I'm curious how it is all framed.
An alternative motivation could be to launder accountability so that when the acute ethical crisis /does/ come, you can throw your hands in the air and say "See? Look at how much resources we poured into this and it still happened! At least we tried!"
Gasoline is a hydrocarbon, but the mass of hydrogen is neglegible compared to that of carbon, so it's not too wrong to say that this mass of carbon came from the same mass of gasoline, 250 kg. The density of gasoline is around 1 kg/l, so we are talking about 250 liters of fuel. The typical tank of a compact car holds about 50 l.
CO2 impact for stuff like this - we probably have bigger fish to fry.
If a key part of Google's claim is that the paper omits relevant research, an author should have simply posted their 128 references and openly asked what work was missing. This whole saga could be easily solved instead of being dragged out for clicks.
> [...] Though Bender asked us not to publish the paper itself because the authors didn’t want such an early draft circulating online, it gives some insight into the questions Gebru and her colleagues were raising about AI that might be causing Google concern.
So, we have on one hand, a researcher who got perilously close to litigation with her employer in the past (IMO because of missteps on both sides).
On the other hand, we have an employer that then was skittish about telling her that they didn't want the paper published (to protect the employer's business interests, mostly, it seems, while maintaining a veneer of open research organizations). And resorted to small statements through HR and intermediaries demanding retraction.
This relationship has broken down; there's no ready process to tidy up the misunderstandings.
I will say that Google's claims to be fostering an open discussion of AI ethics and confronting potentially uncomfortable truths on this path are looking a bit more dubious, though Ms. Gebru doesn't look so particularly easy to work with, either.
5 cars worth of carbon emissions is not a lot given that it is a fixed cost. Very few are retraining BERT from scratch.
EDIT:
The other two points are also disingenuous.
* "[AI models] will also fail to capture the language and the norms of countries and peoples that have less access to the internet and thus a smaller linguistic footprint online. "
NLP in "low resource" languages is a major area of research, especially because that's where the "next billion users" are for Big Tech. Facebook especially is financially motivated to solve machine translation to/from such languages. https://ai.facebook.com/blog/recent-advances-in-low-resource...
* "Not as much effort goes into working on AI models that might achieve understanding, or that achieve good results with smaller, more carefully curated datasets (and thus also use less energy)."
This is also a major area of research. Achieving understanding falls under the purview of AGI, which itself carries ethical and safety concerns. There are certainly research groups working toward this. And reducing parameter sizes of big networks like GPT-3 is the next big race. See https://news.ycombinator.com/item?id=24704952
I think this race has been ongoing for a while now
I think it's quite misleading to compare the energy usage of an industry-wide research effort to individual consumption. The graphs look bad - "wow, 626,000 lbs! that's 284 metric tons of CO2! a plane flight is way less!" - but there's a fundamental difference between "progress on a problem being worked on by thousands of highly-paid researchers" and "I bought a car".
Meanwhile, the worst power plants are generating on the order of 10+ million tons of CO2 every year. There are at least a dozen of these in the US alone. Car factories are emitting hundreds of thousands of tons of CO2 (Tesla is somewhere around 150,000 tons a year, apparently, and it's designed to be efficient). Perhaps activism around CO2 emissions in ML training might be better focused on improving the efficiency of those things instead, seeing as a 1% improvement would outweigh the entirety of the NLP model training industry. It's certainly good to keep in mind the energy costs of training in case things balloon out of control, but right now the costs relative to the results seem small and not worth highlighting as some forgotten sin.
https://www.google.com/about/datacenters/renewable/
Total energy consumption of all computers, mobile phones, datacenters, servers etc, combined, isn't even a percentage point of that.
Yes, CO2 emissions are a problem. You are not going to solve that problem by targeting sectors which entire footprint is not even a significant digit.
But, what if Amazon wants it's own model with its own curation? Maybe we need different languages, maybe countries would like to have their own model with a different world-view. Why shouldn't a researcher train their own model, maybe experiment with different versions? Why should consumers be relegated to pre-trained model with inscrutable preconceptions?
It's not a fixed cost though, it's just how much was spent on this year's iteration of the model. The overall point being made is that model training costs are growing unbounded. Next year it could be 30 cars' worth or whatever for BERT-2, then 600 cars' worth for BERT-3 the year after. That's what it's warning against. At some point it isn't worth it.
I think a more nuanced conversation around these topics will look at exactly what you bring up, how do we properly trade the potential knowledge benefit against the costs?
It pains me that entirely valid avenues of research like this get covered up in nonsense and drama and their message seemingly lost in the midst of it.
Yes if you do run 240x p100s at literally 100% 24/7 for a year you get the power consumption of 5 cars. This run never happened though, this all ran on TPUs at lower precision, lower power consumption and much lower time to converge.
If anything this tells you that electronics are ridiculously green even when operating at 100%. I've never profiled world-wide carbon production but something tells me if you wanted to carbon optimise you'd be better served trying to take cars off the road and planes out of the sky.
We're getting a bit off-topic here, but the #1 target by far in reducing greenhouse emissions is power generation. In transportation it's significantly trickier to replace petrol-based fuels (especially for airplanes), but it's straightforward enough in power plants. And crucially, you can convert all the petrol-powered vehicles to EVs that you want, but if the electricity they're getting from the wall is still provided by burning petroleum then you haven't actually done that much.
Luckily, computation can for the most part be located anywhere (exactly the opposite of transportation), and thus you have a lot of data centers near hydro and other renewable sources so that they can use the cheapest green power available.
I readily admit I don't know any of the numbers associated with carbon production and my comment was solely based on the one GPU vs car figure presented in the aforementioned paper.
Which makes me wonder how far removed AI researchers are from actual production environments. I'm not faulting them, because there's only so much time in your life; the more realistic problem is when someone else takes a paper as gospel and runs headlines with it. Kinda like the trolley problem. Imagine the absurd extreme in this case of governments wanting to regulate large language models because of pollution or to level the competition playing field.
I simply have no idea where the hinge point is. This could inform other questions like, could it be worth to scale up to get a more accurate model (pay up-front in training) to avoid further searches (inference)?
Wind and solar are finite too. The places where they can be harvested are scarce. So if Google using up green energy for bells and whistles on the search page, less homes can use green energy for heating and transport.
Google (probably) wanted to bury it because many of those clickbait headlines would have been negative to google.
https://www.defensenews.com/video/2019/05/31/watch-this-isra... :
Video: Watch this Israeli robot fire a Glock 9mm weapon
Israel’s General Robotics has demonstrated what it says is the world’s first operational armed robot, the DOGO. (Seth J. Frantzman/Staff)
(Also, what were they thinking would happen when they originally hired AI ethics researchers, then? Nobody made them do that.)
The impact of that on AI and the difficulties in counteracting it are not obvious to anyone who has ever used the internet or even—from, among other bits of evidence, public clashes Gebru has had with people who work in AI outside of ethics—not even to everyone building and training AI models on public data that is impacted.
But the research is not even published yet
We're only reading a summary of the paper because apparently the authors aren't confident enough in its quality to release it publicly.
You can't claim that Google dismissed this paper out-of-hand while simultaneously saying "oh, but it's too much of a draft to release publicly". Uh, if it was too much of a draft for the public why shouldn't it be too drafty for Google? Are we really pretending that Google has lower standards of quality than the general public?
Google consumes a vast amount of energy: "10.6 terawatt hours in 2018, up from 2.86 terawatt hours in 2011."[1]
If training and retraining models is a significant and inefficient part of google's energy consumption, the point doesn't seem insignificant(edit: The most advanced AI model involve as much as computing and energy any programs ever created [2], btw). I'm biased by the impression Google's actual search results haven't improved very much but I don't think I'm alone in that impression.
[1] https://www.statista.com/statistics/788540/energy-consumptio...
[2] https://openai.com/blog/ai-and-compute/
I'm guessing the vast majority of the energy usage is for serving billions of requests for various products, such as search, youtube, maps, gmail, photos, etc.., and the cpu, network, and storage requirements for those requests.
Training and retraining ML models is definitely not on the hot path.
https://news.ycombinator.com/item?id=25307167
https://news.ycombinator.com/item?id=25292386
https://news.ycombinator.com/item?id=25285502
https://news.ycombinator.com/item?id=25289445
I’ve recently noticed that this commonly occurs ie multiple relevant threads that make up the top 50 of HN over say a 72 hour period, all related or rifts on a similar discussion.
Wondering if there is a way to group them as part of the same topic/submission? (thus saving you the manual work of posts like this). I appreciate this would (i) require a code change and (ii) would shift the HN model from individual threads based on 1 URL submission, but just thought I’d suggest it to the HN brains trust.
In other news: Keep up the good work that you do on HN to give this online community ‘structure’.
https://news.ycombinator.com/item?id=18183822
As someone who's worked on HN code since then, I can tell you there's no such "mindset", nor is it true that "we don't get any iteration on the feature set here" (11 days ago: https://news.ycombinator.com/item?id=25197418). It is true, though, that most of the changes are subtle enough not to be so visible, including the ones I'm working on this evening. Most of the effort goes into attempting to improve, or at least preserve, the quality of submissions and comments.
This is how much of the internet has felt for a long time. After this nugget, I now wonder if Vox is just a model trained on Piketty and Tumblr.
edit: Also not sold on the CO2 argument. Too many variables! Nerds will calculate and re-calculate such things, with the result jumping all over the place, swearing that they've gotten it right--this time! No humility, in spite of the odds.
There, I've repeated the paper without reading it. But Google did hire her to be an AI ethicist so what else would they expect?
They lose nothing but face.
If a person of color (I am myself non-white) is not performing, what does it take to fire them without the entire world playing the race card on you?
Are we creating a society that makes it impossible to fire a person of color? You know there are bad apples in every race, right? How do we handle such scenarios? Seems unfair to me, myself being a person of color - I don't want the world to treat me like some kind of a hero for being non-white / minority. I want fairness and it is frankly offensive.
I am not pleased with the way we're treating each other. It's supposed to be equal opportunity.
I also want us to have scientific discussion about gender differences (backed by research) and other difficult conversations. Nature doesn't give a fuck about any of this - if our goal is to uncover the way mother nature works, we're going to have to meet difficult truths and not be afraid of it.
We've created an atmosphere of fear. I don't feel comfortable voicing my opinions even after being anonymous on the internet. That's pretty fucked up.
There is no way to prove you are actually PoC. And even if you did, it's incredibly unrelated to the matter at hand.
No one has even claimed she was fired for bad performance and dozens of Google employees have said in no part of their process was academic rigor taken into account.
What you're doing is the literal definition of concern trolling.
Also, I am kind of shocked you would accuse of me something like this. WTF. I am not trolling. I am asking a difficult question that needs to be discussed because no one is discussing it.
Why not make a new HN post and discuss this? How is that discussion point related to this article?
What part of this article or the events surrounding it suggested there is a fear of firing PoC?
There is absolutely no reason to bring this up on this article other than to derail the conversation into FUD.
You also illustrate beautifully the vapidness of this fad notion of "concern trolling". Because they have not conformed with your view of the issues and used your preferred language, they can only be conceived of as trolling.
The other poster seems to be making a genuine effort to express their thoughts and edited their post to be less combative. Yet you can ONLY see them as being dishonest and disingenuous.
If we cannot sit down and have a peaceful conversation, calling trolls and other non-sense, please don't divulge in this thread.
Dropping boilerplate ideological provocations onto unrelated threads isn't good-faith conversation. Whether you mean it to be or not, it has the effect of trolling, and on HN, trolling is a strict-liability offense; your mens rea matters less than the outcome.
Please don't do things like this on HN.
I’m genuinely sorry to hear that you feel uncomfortable voicing your opinions. I know from personal experience how hard it is to feel like you have to keep yourself closed off to the world. As a species generally, and as technologists specifically, we have some way to go to create non-toxic spaces for people to share ideas.
Still, you are engaging in fortune telling[0]—you really can’t know that your comments will be downvoted before you make them. Feeling compelled to add notes to the end of your posts encouraging others to downvote is your brain tricking you into tilting the scales to “confirm” what you “knew” to be true. It may feel like a helpful strategy to blunt the emotional pain of discovering that people don’t agree with you all the time, but from what little you’ve said, it sure seems like it’s just reinforcing your negative outlook. I don’t want you to feel bad all the time, and I suspect it is not actually true that most people here are going to disagree and downvote you to oblivion all the time so long as you avoid self-sabotaging.
I know it can be incredibly hard not to take downvotes personally, and, I hope you are able to try to reframe them as what they are: some random people, some of whom are thoughtful and some of whom are not, pushing a button. It’s not a personal attack, even though our brains can make it feel very much like it is. If you truly are getting downvoted a lot, it may be a signal that some of your opinions aren’t fully thought through and need to be re-evaluated, or perhaps that you just didn’t present your ideas well. On the other hand, your brain can and will exaggerate the negative experiences, make them seem like they are happening a lot more than they are, and make you feel bad even though you’re actually doing just fine.
Anyway, while I’m sure it happens (I don’t think there’s any space that is totally immune to bandwagons), I don’t get the sense that genuine and thoughtful comments regularly get downvoted to oblivion here. It’s trickier than ever these days since there is a lot of bad-faith argumentation going on everywhere online under the guise of innocently “just asking questions”[1], and I think it’s fair to say that there is an growing immune-like reaction which is sometimes attacking genuine posters because it’s just impossible to tell who’s being honest and who’s being a shitty troll.
So just keep doing your best, anonymous internet commenter. :-) If you feel like you can’t, I hope you can find a counsellor or friend who will listen and help you into a more positive head space. At the least, your post has generated some reasonable and civil discussion, and that’s what we’re here for, right?
[0] https://en.wikipedia.org/wiki/Jumping_to_conclusions
[1] https://rationalwiki.org/wiki/Just_asking_questions
Now...try this instead. Think of downvotes as someone anonymously throwing a tantrum with a keystroke because their tender tender feelings were hurt.
That, my friend.. is not your problem.
This is not good.
There are a lot of things that are illegal but people do it anyways.
No, they don't.
A workplace that isn't hostile to people of color and evidence of the cause of dismissal
Perhaps instead, in these situations, employers could provide, at the discretion of the dismissed, information gathered while performing due diligence leading up to the dismissal
Since media cannot cover thousands of individual cases, there should be legal avenues without deep pockets for lawyer fees to sue companies for racist behavior.
The court should look at this situation objectively and factually.
If a company is responsible in how they manage, follows policies, etc they are fine. If executives or others are allowed to misbehave and the company is too cheap to buy silence, things may not be fine.
What’s the real story here? I don’t see evidence of incompetence. But you can be fired for any legal reason in absence of a contract. Maybe there’s some unknown political or other issue. Maybe some conduct crossed a line. Who knows.
The word "fired" often is reserved for terminating someone for cause, where you really broke the rules and get no severance, not even the two weeks minimum that is customary. Non-performers usually aren't treated this way. I'm not sure this is really what happened in this story.
Google has set up a research organization that ostensibly is empowered to ask and ultimately work to openly resolve difficult questions like these... but when the rubber hits the road, they instead throw the researcher under the bus.
I think it is going to be extremely hard. From the same article, I opened a tweet and look at what a high voted reply is: https://twitter.com/PocketNihilist/status/133495412981528985...
This line of reasoning means, you can't be pro diversity and fire someone from the underrepresented groups at the same time for their behaviour.
People are conveniently choosing to forget this is the same company which not long ago fired a person when he complained about the company being too pro diversity in their hiring.
I really hate Twitter and its mob culture.
IMO it enriches HN comment sections to allow for people to bring up adjacent topics, things that came to mind or funky little "this reminds me of..." anecdotes.
For them to be one of 99.9% of the world who can't mobilize a following to create a complaint about this?
Are we creating a society that makes it impossible to fire a person of color
We so far from such a situation like that that your complaint is absurd. A few places with a history of discrimination may have trouble firing the few people of color they might hire. That's about it. In the real world, incompetent people get fired and often competent people as well. A few people may make a career of playing the race card but that's a limited number of people.
Both racism and opportunists "playing the race card" can be real at the same time.
Great question. Part of the reason why the black unemployment rate is so low is that it's virtually impossible to fire them.
This is a pretty superficial take on what is an extremely interesting sociological topic. (To be clear, I’m referring to the article, not the underlying paper which we don’t have.) Obviously just because social movements “have tried to establish ... vocabulary” doesn’t meant that vocabulary has become a “new cultural norm.” Plenty of such efforts end up being cultural dead-ends.
Take for example a term like “LatinX.” This term has been proposed and is used by certain people, but is extremely unfamiliar and often alienating to Latinos themselves: https://www.vox.com/2020/11/5/21548677/trump-hispanic-vote-l... (“[O]nly 3 percent of US Hispanics actually use it themselves.... The message of the term, however, is that the entire grammatical system of the Spanish language is problematic, which in any other context progressives would recognize as an alienating and insensitive message.”).
The article hand-waves away a deeply interesting question: What should an AI do here? Should AI reflect society, or be a vehicle for accelerating change? It seems at least reasonable to say that the AI should reflect what people actually say, in which case a big training dataset is appropriate, instead of what some experts decide that people should say. In some contexts, for example with “LatinX,” researchers seeking to enhance inclusivity could instead end up imposing a kind of racist elitism. (People without college educations—which disproportionately comprises immigrants and people of color—tend to be less knowledgeable about and slower to adopt these changes in vocabulary.)
The paper seems to imply that AIs should not reflect “social norms” but that training data should be selected to accentuate “attempt[ed]” shifts in such norms. Maybe that’s true, but it doesn’t seem obviously true. To return to the example above, is some Google AI generating the phrase “LatinX” (which 3/4 of Latinos have never even heard of: https://www.pewresearch.org/hispanic/2020/08/11/about-one-in...) in preference to “Latino” or “Hispanic” actually the desired result?
What the AI should say is hard given that there is no one right answer to the question of what any individual should say. Different contexts change the equation completely. Seems like a nightmare to define the behavior or test it.
The right solution may involve training a model on everything we have access to and finetuning it based on the context you want to use the model in and the historical examples we will build up of mistakes previous models have made.
But it is a hard problem, and highly context dependent. It doesn’t seem to me like a proper subject for the sort of ultimatum that Gebru gave Google.
I was actually going to turn the website off (I believe it hasn't been working for a couple weeks now) as I didn't think anybody was using it. Glad to see someone is though - it's back on!
If you do turn it off, would you mind sharing your list of websites?
I'm learning django right now and for my first project (after the tutorial project) I will recreate stumblingon. When I do I'll add a logging/metrics system of some kind. That way, I won't make premature decisions about turning it off in future.
I'll also commit to replacing the website with a list of the index for a month or so before (if) I turn it off for good.
Now that I'm thinking about remaking it, is there anything you think is missing from the website?
For example, here is Hispanic Congressman (D) Ruben Gallego on the subject: https://twitter.com/rubengallego/status/1324071039085670401?...
Data centers around the world account only for a tiny fraction of electricity consumption or carbon emissions. Do you even account for the reduced car and air travel due to remote working and online shopping?
How are advanced few-shot learners like GPT-3 even remotely a problem, training less models is somehow worse? Do they even know what they are taking about?
Its all very confusing, but a lot of the work done by grievance studies becomes immediately easier to understand once you realize they are arguing in bad faith.
BTW, "Stochastic Parrots" is a very descriptive name for the problem
> Moreover, because the training datasets are so large, it’s hard to audit them to check for these embedded biases. “A methodology that relies on datasets too large to document is therefore inherently risky,” the researchers conclude. “While documentation allows for potential accountability, [...] undocumented training data perpetuates harm without recourse.”
Since these models are being applied in a lot of fields that directly affects the life of millions of people, this is a very important and underdiscussed problem.
I really want to read the paper.
In particular, it is being applied right now to rank Google search results, and probably responsible for lots and lots of Google's profit. You should be skeptical of Google's appraisal of the paper that is material to Google's profit.
Meh, both parrots and language models are inferior to humans in producing language, but one is more useful than the other. And real parrots are also stochastic, like all living things.
After the stochastic parrots have caused the collapse of civilization, we can eat the real parrots.
I don't understand how living things are "stochasic". Can you please elaborate on the matter?
Gebru is the type of person who defines "bias" as anything that isn't sufficiently positive towards people who look like herself, not the usual definition of a deviation from reality as exists. Having encountered AI "fairness" and "bias" papers (words quoted because the words aren't used with their dictionary definitions), it's not even clear to me they should count as research at all, let alone be worth reading. They take as the starting premise that anything a model learns about the world that is politically incorrect is a bug, and go downhill from there.
All politics aside, this is not even true for toy ML problems. If I’m trying to do digit recognition and “all the data I can find” is a billion hand-written 0’s and a million hand-written 1’s through 9’s, naively training on that data will yield a model that’s pretty close to guessing 0 every time.
I think you're trying to say that a large enough dataset will be free of bias. I don't see how that follows. If I train a model on home mortgage decisions, I will replicate the bias on that currently exists on that dataset - https://news.northwestern.edu/stories/2020/01/racial-discrim... - unless there are conscientious choices to reduce that bias.
Researchers in ethics in ML are specifically trying to enable tech companies to do a better job of not replicating bias and justifiably point out where that is occurring.
Third, I would argue that applying an ML model to do something faster if it replicates the bias of a previously human decision is even worse. The bias has taken the human element completely out and systematized the bias and made it possible with even less friction.
> For example, in about 10% audits in which a white and an African-American auditor were sent to apply for the same unit after 2005, the white auditor was recommended more units than the African-American auditor. These trends hold in both the large HUD (Housing and Urban Development)-sponsored housing audits, which others have examined with similar findings to us, and in smaller correspondence studies
They fail to mention how large is the gap. Is the white auditor recommended 102 vs the black auditor 97, or 150 vs 50, or 200 vs 3? Without such critical information it is hard to form an opinion, unless one already has a large bias in accepting discrimination narratives uncritically.
> In the mortgage market the researchers found that racial gaps in loan denial have declined only slightly, and racial gaps in mortgage cost have not declined at all, suggesting persistent racial discrimination. Black and Hispanic borrowers are more likely to be rejected when they apply for a loan and are more likely to receive a high-cost mortgage.
They fail to mention the magic words 'when controlled for income'. America has a huge income disparity problem, which is conveniently forgotten behind the ongoing race (and gender) hucksterism. Assuming we'd wave a magic wand and fix all disparities across visible populations tomorrow, it will still not fix the fact that huge income disparities exist between individuals. Google engineers and researchers get paid 5 times the median national income or more, and (senior) Google management in the 10x to 1000x range. The vast majority of the population is stuck in dead end precarious jobs, with little social mobility, one medical emergency from bankruptcy.
I haven't seen this "usual definition" of bias as a "deviation from reality" that you give anywhere before. Can you say where it is coming from?
Bah
It seems amusing to read this when the same person and her followers hounded Yann LeCun for pointing out the same thing with image models.
Anyway, this seems interesting. But I am not sure how you solve this. Do we take representative dataset according to population of a place? Also, assuming this is limited to a single language. How can AI generated language account for nuances from regional differences at the same time in a common model. Isn't what the author asking for here is kind of train, en_US, en_GB, en_IN separately here. For things like completion don't language models already account for this?
Maybe if you have an unpopular language, that's just unfortunate and please encourage your kids to learn English (which they probably do at school just about everywhere anyway) so you don't perpetuate the same problem to the next generation.
Basically I know the US better - or the image it projects - than my own country.
Funnily it was a huge letdown when I visited for the first time decades ago, as it felt just like a sitcom and there was nothing new really.
This is akin to a speech writer working for a politician, writing a piece that disagrees with the party platform, and refusing to fix it when asked.
And often it is a damn that need only break once. Corporate ethics as a whole seems contradictory. The only ethical rules which a company may obey in perpetuity are those that do not for long conflict with profit and those which are backed by effective laws.
Even when done with a positive intent, I fear creating positions such as this is begging for eventual corruption.
The response was to decline to meet and fire her.
It’s entirely plausible that the paper is bunk; however, when someone is willing to come down that hard to prevent an idea being published, I tend to err on the side of “worth finding out what”.
I think that is in dispute. Jeff Dean claims she did refuse to fix it.
> It’s entirely plausible that the paper is bunk; however, when someone is willing to come down that hard to prevent an idea being published, I tend to err on the side of “worth finding out what”
But they weren't trying to prevent the idea from being published. They certainly knew by firing her they'd give it far more attention than it would otherwise get. What they wanted was not to put their name on this idea.
This technicality matters for unemployment and COBRA, and it has a concrete definition.
https://twitter.com/timnitGebru/status/1331757629996109824
Are you sure she agreed to fix it? According to her tweets she agreed to take her name off the paper, not to fix it, and only if they agreed to her conditions which included releasing the names of all the reviewers involved. From Jeff Dean's email:
> ... including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback
If I was in their position, I would be extremely uncomfortable releasing the names of the reviewers with the likely result that they would be doxxed by Gebru on Twitter
While I don't really understand the emissions argument, the rest strikes me as very defensible. If the best language models need giant datasets to excel, it is very difficult to ensure your AI is trained on reasonable data.
I wouldn't want an AI therapist to be trained on 4chan. Now obviously nobody would be that stupid, but unfortunately it seems we are not far from it.
Unless we build models that rely on less data, it will be difficult to prevent problematic biases in the AI we put into production.
If that is the argument Timnit makes, I think that's the exact type of work I would expect from an AI ethics department. And good work at that.
I'm not blaming you solely but the recurring idea that it can is worrying.
That's just outright false. She was terminated, effective immediately.
Does anyone else find themselves losing a lot of respect for Jeff Dean in all this?
No, I haven’t lost respect.
Without taking sides (I don't know the full story to understand who is objectively in the wrong here), that's not completely true.
She basically gave an ultimatum - meet my demands or I can work on a last date.[1] It is quite common at big tech that once you resign - officially or unofficially - you could be asked to leave effective immediately. Usually happens when you work in critical areas or moving to a competitor or a disgruntled employee.
Could this have been better handled? Maybe, but no matter who was in the right or wrong, she wasn't technically fired.
[1] https://twitter.com/timnitGebru/status/1334343577044979712
She was fired.
They can't accept the threat itself as a resignation. That's not how threats work.
Firstly, there is ample evidence of her toxic behaviour online. (See the exchange with lecun on twitter, absolutely horrible). So its clear she has issues with how to collaborate and communicate.
She then goes on to threaten the company she works for. Like wtf? Irrespective of what you want to do in response, act like a professional. If you were a CEO/Manager etc and had someone with toxic behavior come in and threaten you if you didnt comply to their demands, wouldnt you go 'ok, see ya'. I certainly would. Everyone is replaceable. Especially if youre toxic when it comes to dealing with situations you dont agree with
Which is fine, but that's firing them for acting up. It doesn't really matter what the demands are.
"Gebru’s draft paper points out that the sheer resources required to build and sustain such large AI models means they tend to benefit wealthy organizations, while climate change hits marginalized communities hardest. “It is past time for researchers to prioritize energy efficiency and cost to reduce negative environmental impact and inequitable access to resources,” they write."
-- Has she calculated the opposite alternatives? What were the costs of the previous models? How about the alternative of not-having ai-backed search at all. Would worse results be actually worse energy wise? If you are not including the substitues/alternative costs it is just sloppy research, or just done purposely on bad faith.
"One is that shifts in language play an important role in social change; the MeToo and Black Lives Matter movements, for example, have tried to establish a new anti-sexist and anti-racist vocabulary. An AI model trained on vast swaths of the internet won’t be attuned to the nuances of this vocabulary and won’t produce or interpret language in line with these new cultural norms."
-- Ok, basically she is pissed as AI is picking normal language, that people use, and not using the vocabulary that is in vogue on certain political circles. Basically censure, and forced speech.
Why does google even bother at this point? It seems that certain circles just don't want to have positive constructive way, but it is becoming: "my way, or you are X", with "X" being whatever prerogative they will through at you.
I think social engineering b.s. should be kept as far away as possible from science. This is turning true ai research into a masquerade to push certain political agendas.
basically: instead of reform, and fixing whats broken; to tear it down, as it is just another means of oppression.... (yes, that's the mindset of the movement).
Her research, looks to be mean spirited, and made to discredit google, so it can be blackmailed into whatever the ideology wants.
It is not meant to fix/improve the situation, but used as political amo. Hence it is just a political paper/propaganda, and not proper ai research.
I find the sequence of decisions made by Google problematic.
Until they describe in intimate detail what their management has been planning to filter they are effectively retracting all research from their institution as having this unknown bias. Like a tabaco lab that blocks all results against their products. That this was shoddy means nothing unless someone would have caught a Google positive piece as shoddy and actually stopped it at this process. I would guess no.
The right thing to do was let it be published, get external critique and end the relationship if most people with no money in the game thought it was bad science.
She didn't... which are the hallmark of bad research.
You're reading a non technical summary of an article, and taking it as gospel.
I think I'm going to wait to see the paper before deciding whether or not it's good.
What do you suggest might the article be getting wrong about her paper? Or are you suggesting that the information contained about the paper in the article is inherently invalid and should be ignored altogether?
The dangers of large language models is an interesting topic but it’s not AI research and it doesn’t advance the state of the art of AI.
When it comes to technical substance in the field of AI, her paper is indeed lacking in that unless the article completely failed to mention something that would directly qualify as original AI research.
I also have to say that “AI” is a vastly different area of study than “ethics.” Very strange that Google has these organized in the same hierarchy.
Damn those Freudian slips.
https://ai.stanford.edu/~tgebru/
Ground-breaking might be a strong word but it does move the industry forward.
https://en.wikipedia.org/wiki/Timnit_Gebru
Take for example a retrospective look at 2017 NeurIPS papers done in 2019 (https://archive.is/wip/77YrB).
You can disagree with how she and/or Google has handled this whole situation, but please do not denigrate work that has been cited (https://scholar.google.com/scholar?oi=bibs&hl=en&cites=14954...) by papers accepted at the most competitive/prestigious ML conferences.
EDIT: I also do not see how in good faith you can say that VentureBeat, a company who makes the bulk of its revenue from running conferences catering to C-suite execs who can shell out thousands of dollars for a ticket, is "leftist".
Most executives of tech and news companies would bend over backwards just to show how leftist they are in 2020.
(I’m only replying to the part of your comment that answers mine)
Even someone like Ben Shapiro recognizes a difference between liberals and leftists (https://twitter.com/benshapiro/status/966081078166421504).
Silicon Valley types would hardly be described as leftists. Numerous studies have been done on the attitudes of Silicon Valley founders and execs (https://www.vox.com/2015/9/29/9411117/silicon-valley-politic...). The distinctions are dramatic.
We see that on average, tech founders are less likely to support vs. even Democrats generally (not just progressives):
* Banning the Keystone XL pipeline (60% vs 78%)
* The individual healthcare mandate (59% vs 70%)
* Labor unions being good (29% vs 73%)
This is to say, the average Silicon Valley type, particularly the C-suite exec or founder, tends not to be on the left wing of the Democratic party.
During the 2020 Democratic primary, even the Silicon Valley billionaires who are openly Democratic-leaning donated to candidates who were not to the left of the field (i.e. Elizabeth Warren and Bernie Sanders) (https://www.cnbc.com/2019/08/13/2020-democratic-presidential...):
* Eric Schmidt -> Cory Booker and Joe Biden
* Reed Hastings -> Pete Buttigieg
* Marc Benioff -> Cory Booker, Kamala Harris, and Jay Inslee
* Reid Hoffman -> Cory Booker, Kirsten Gillibrand, Amy Klobuchar
* Jack Dorsey -> Andrew Yang, Tulsi Gabbard
* Ben Silbermann -> Pete Buttigieg
I'm engaging with you in good faith, and because I was intrigued that in a previous comment you mentioned that you live in Spain (though who's to say you're not a US ex-pat). But calling US tech companies "leftist" is a stretch at best.
I don’t really care about the paper author’s fate, but it seems unsettling to me that she is discussed more than the paper itself.
Your argument sounds like a red herring. Qualified for what? I didn't noticed the OP making any remark on the author's qualifications.
And by the way, you should think things through before pulling appeals to authority to try to defend the credibility of researchers. See James Watson, a nobel laureate and a founder of modern genetics, and his comments on race and gender.
Did you not read the first sentence of OP's post?
"Ok, so basically she is a bullshit social engineer, masquerading as an 'AI ethics researcher'"
That does not mean at all that the author lacks qualifications.
Pretty obtuse to suggest otherwise.
You would do well to work on your literacy and reading comprehension skills.
To start off, be aware that "a bullshit social engineer masquerading as a AI research" does not, by any means, mean the author is "pretending to be an AI researcher".
It means "social engineering pretending to be AI research", doesn't it?
Think about it.
Look at the context. Look at the people involved. Look at the topic of the discussion.
And let's be honest for a second: do you personally believe that anyone would accuse an AI researcher who worked at Google of not being an AI researcher?
Literacy matters, and so does honest and well meaning interpretations of what someone actually did said.
pretend to be someone one is not.
https://www.merriam-webster.com/dictionary/masquerade
"an action or appearance that is mere disguise or show"
As in social engineering masquerading as AI research?
Honestly, is there any room for doubt at this stage?
I don't expect honest and well-meaning people to continue insisting on this at this point. It feels that these attempts to distort what was said are simply attempts to attack the OP based on misreprentations of what he did said.
Cherry-picked? It's literally the first sentence, the central thesis of OP's entire "argument". If they can't get that down on paper correctly, they're not worth taking seriously.
> You would do well to work on your literacy and reading comprehension skills.
Ascribing unsubstantiated sentiments to someone's post in lieu of what they actually said doesn't fall under literacy or reading comprehension, sorry. It's actually the polar opposite of that.
Why are you claiming to know OP's intentions (despite their own words) better than everyone else here? Is OP your alt account?
> I don't expect honest and well-meaning people to continue insisting on this at this point. It feels that these attempts to distort what was said are simply attempts to attack the OP based on misreprentations of what he did said.
It's extremely bizarre that you're questioning the literacy of people when you're projecting your own unsupported sentiment to OP's post while simultaneously claiming that people who are directly quoting the OP's post verbatim are "misrepresenting" what OP said.
You're being very disingenuous here (as well as borderline manipulative) by trying to shut down any dissent against your flawed position by questioning the good faith of the people doing nothing but directly quoting the OP's words.
Amusingly, the OP got shirty at someone* further down this thread for calling them names, but with an unfortunate textual blunder which exposed the hypocrisy of their own name calling (not to mention total lack of self awareness).
I pointed this out but got downvoted to oblivion of course. Classic goose / gander politics.
* (and rightly so btw, that kind of conduct has no place on HN)
Misogyny clearly is a problem for many commenters here.
We call it “FU money,” but these guys aren’t satisfied with being able to say “FU” and still live comfortably; they want to be in charge of things that are seen as significant.
I believe it will always take more than money to get thousands of largely independent adults to do what you want.
What has this got to do with anything? How is driving to a library the "oposite" of AI?
We had very effective search engines for decades before AI.
For example:
> Ok, basically she is pissed as AI is picking normal language, that people use, and not using the vocabulary that is in vogue on certain political circles. Basically censure, and forced speech.
I think a stronger version of this argument, which can appeal more broadly, would be: an automated system that is constantly being trained on data from social media can more quickly pick up new trends in human speech than a manually-programmed expert system. Furthermore, even if humans are constantly updating an expert system, the biases inherent in the composition of the company-hired set of experts will limit the representation of minority trends in the data. On the other hand, a model examining the entire data set can pick these up automatically. For example, I can easily get AI Dungeon (powered by GPT-3) to communicate using language and concepts that are only present in a minority community that I’m part of that I’ve rarely seen represented in literature.
When arguing against Timnit Gebru or their ilk, you don’t need to just give them the support of liberals. We actually make up the majority of techies, so it’s a bad position to put yourself in if you want your cause to succeed.
Exactly this. Framing it in terms of alternate options makes AI the more reasonable choice.
Google AI appears to have drawn unnecessary attention to themselves for relatively benign paper. Most of the arguments in this paper have already been discussed elsewhere and could be refuted in simple terms.
I think it is precisely for these reasons this debate is so important, because it is truly not a clear cut improvement in my opinion.
They didn't say they shouldn't make them, they said "prioritize energy efficiency".
normal language, that people use
As they said, language changes. Do we want a model that acts like the average person of the last 20 years? Or do we want a model that acts like a person who has just been through 2020?
You're free to use the model you believe is best for your goals.
I you use real world data from the last 20 years to generate the model then your model will reflect an amalgamation of the real world data collected in the last 20 years.
If you prefer to focus on last month then go ahead. You'll have less real world data and seasonal patterns will have a deeper impact and the model will be less robust, but go right ahead.
If, instead, you want a model that is not generated from real world data and instead just uses your own artificial data that is blatantly and openly manipulated to reflect your personal biases and political preferences then just call the spade a spade.
Driving to the library, that is the alternative to training these models, really?
> -- Ok, basically she is pissed as AI is picking normal language, that people use, and not using the vocabulary that is in vogue on certain political circles. Basically censure, and forced speech.
If you had a model trained on a large corpus of data from the pre civil war southern American states, it would have been deeply racist, and would even view black people as possible property. If you had one that was trained on data from the 1950 it would be less racist but still problematic viewed by people from today. Is there really something special with today, that removes these kind of concerns with a model trained with current data?
> I think social engineering b.s. should be kept as far away as possible from science. This is turning true ai research into a masquerade to push certain political agendas.
It seems to me that it would be impossible to do any social science research, and specifically any research on racism. With this kind of attitude.
Some of the concerns brought up in the paper seems less consequential than others. Especially the pollution one seems weak to me, that doesn't make it false, and fair enough that is was brought up. I find the racism issue a lot more problematic, and is something I've run up to working on deep learning solutions myself. Even if it worked fine for my group, and most of our customers, that is definitely not fun, and something practitioners should consider when building these things.
I think this argument applies not just to machine learning, but to learning in general. Any kind of knowledge-acquisition process is going to be biased by the environment in which it occurs. That goes not just for digital neural networks, but also those in our human brains, operating on the same racist data the ML models are. If that means we shouldn’t do machine learning, it also means we shouldn’t do human learning either.
Of course, the preceding is absurd. A more reasonable take is that we should adjust the objective function of our learning processes to try to account for the effects of biases. We try to do that subjectively as any decent person operating in a biased society should, but our ML models can do it more accurately. In fact, I’d argue that such techniques are necessary to more carefully analyze and build evidence describing the effects of those biases. They can provide insights that will even improve our ability to correct for biases in the real world.
I think this sort of veiled personal attack resorting to baseless extrapolation is not a productive line of public discourse.
You should take a step back and think about a) the field of study, b) how the main argument completely ignores the basis of said technical field and what it studies, c) how the argument being created lies on the idea that a self-annointed elite should have the right to manipulate the public to force it to fall in place with it's goals and desires.
I thought that my comment was fitting to the tone of the text I replied too, but fair enough, maybe I shouldn't have included that paragraph.
> You should take a step back and think about a) the field of study, b) how the main argument completely ignores the basis of said technical field and what it studies, c) how the argument being created lies on the idea that a self-annointed elite should have the right to manipulate the public to force it to fall in place with it's goals and desires.
It is unclear to me what this means, is it arguing that studying bias in AI and specifically deep learning is not germane? Is it arguing that it is not acceptable to institute policies and action to create a level playing field for minorities and repressed group because that would be social engineering, and forcing the common man to do the elites bidding? If that is what it means, I have to disagree, I find both of those things worthwhile.
Yes, that was one of the problems.
> It is unclear to me what this means, is it arguing that studying bias in AI and specifically deep learning is not germane?
Let me make it clear for you so that a) we are able to talk about things objectively, b) your options to continue using veiled personal attacks is curtailed.
Either your goal is to model reality and real life, or your goal is to model your idealization of what you feel real life should be.
If you pick option #2 then your model does not reflect real life.
Bias is by definition the way the model returns results that don't match the real world and real life, in frequency and in proportion.
If your goal is to use your model to manipulate and control society based on your own personal criteria, by manipulating it to return results that distort the real world and real life, then call it something else, because bias is not it.
In the context of creating models, it's representative data collected from statistically significant observations of a population, for starters.
> In cases where some communities/countries have (to quote from the article) a "smaller linguistic footprint online", is trying to control for this a form of manipulation of society or a way to fix a bias?
In modelling there is no such thing as control. There's the input data and there's the model generated from input data.
If you are looking for a model that is expected to represent a property intrinsic to a specific community then you use data collected from that community to generate that model. That's it.
Models generalize, and reflect the norm. They are like that by design. That's their point.
If your plan is to have a model that does not reflect the input data but instead forces your biases regardless of the input data then your goal is not to model reality but to distort it to comply with your personal goals.
> If you pick option #2 then your model does not reflect real life.
These deep learning models built by corporations are not scientific models, they are engineering solutions, built to solve problems. Reflecting the real world is only useful if it furthers what the company wants to solve. If they for example remove swear words from their training set, that will make them a less accurate model of the world, but make them more useful for building solutions. But it is probably a trade of they would be happy with. We've also seen example of risk scoring application for felons that seem to end up doing racial profiling, because that is what the data seem to indicate makes sense. But that's deeply problematic and runs counter to laws in some places and seem ethically problematic (https://www.theverge.com/2020/6/24/21301465/ai-machine-learn...).
> Bias is by definition the way the model returns results that don't match the real world and real life, in frequency and in proportion.
Getting a good data set without bias is hard, even if you crawl the whole internet like Google does. Not everything is on the internet and there are systematic drivers that make some part of the human condition over represented (English, science, the views of the affluent and educated), and some under presented (small languages, the discourse of people behind the Chinese great firewall, the poor). So just getting a ginormous data set does not fix bias.
> If your goal is to use your model to manipulate and control society based on your own personal criteria, by manipulating it to return results that distort the real world and real life, then call it something else, because bias is not it.
Positive bias is absolutely something that we use, and while it might seem sinister it does not have to be. The example I'm most familiar with a facial recognition technology. Most groups building that ends up with a model that is better at some groups than others. Asian research groups often end up with models that does well with asians and worse with whites, while European groups usually end up with the reverse. In some sense these results do reflect the reality of these groups, most people in Europe is white and most people Asian is asians, so that you training sets ends up like that is not surprising. But no one is happy with these kind of results, and everyone wants to fix that.
To bring it back to speech and text models, let's say you are building a customer service solutions incorporating a deep learning model, the reality might be that you current customer service representatives treat blacks (or people who use "black" dialects), worse than people who sound white. An accurate model built on this data set will then also do that. But is that acceptable? I hope most companies would want to fix that, and be fine with adding some positive bias in their solution.
As opposed to the more direct “approach” in the parent post?
> Ok, so basically she is a bullshit social engineer, masquerading as an 'AI ethics researcher'
If the large language model analyzes everything, it will veer towards normal. It doesn’t seem so far fetched to think they can build something which also tracks and understands language trends and slang, and how to appropriately use this “special vocabulary”.
I don’t know much about how these models work, but I would hope something trained on the entire internet is smart enough to know to speak to me in modern English, not Civil War English or Shakespeare English.
This was just glossed over and maybe slightly off-topic, but what kind of shifts in language did these two movements in particular affect?
Stuff like #MeToo was about raising awareness about sexual abuse, particularly in the entertainment industry, can't really think of how it forced people to change words that they use.
The best example I can think of is people opting to not use "master/slave"... Which is much more debatable and far off from "core tenets" of the BLM movement like police reform and dealing with systemic racism.
I guess what I am trying to say is that when it comes to language, things are a lot less black/white (heh) and we should be careful about people that are setting themselves up as arbiters. Sort of a who guards, the guardians issue I suppose.
The thing is: The language that people use is often enough offensive: racial slurs, sexism and other forms of discrimination, plus seemingly innocent "code words" like "globalist".
AI developers should know about these problems and keep them in mind while developing training data for AI models. The consequences of ignoring such bias is continuing discrimination - with the most obvious and low-tech reminder being the "racist soap dispenser" which had only been trained on white people's skin and failed to dispense soap for people of color.
The danger in such discriminatory models is that while it's an annoyance that a soap dispenser does not dispense soap for a person of color, with algorithms running more and more of our daily lives, the impact is way higher: whether being refused credit or affordable insurance rates because an AI "thinks" the applicant is black and living in a black, poor neighborhood is grounds enough for that, or "predictive policing" AIs sending in massive police responses to everyday calls, this directly hits on people's survival without human oversight.
That is why ethics in AI are so important and why "social justice activism" is so desperately needed!
It’s just basic intellectual honesty, sorely missing in the SJW circles.
It's a white-supremacist dog whistle for "jew": https://www.haaretz.com/us-news/.premium-how-did-the-term-gl...
> the soap dispenser wasn’t “racist”, it was just “bad”
For a person of color who wants to wash their hand and sees that it dispenses soap for white people but not for them it is yet another piece of everyday racism.
As for “globalist”, I don’t see an issue with criticising globalism and those who promote it, be they Jews or others. Clearly a lot of Jews (in Israel) are very “localist”, and I suppose there are some that are “globalist” as well. So?
Just because a view is also supported by some or other “vile” people (that are almost always a tiny minority, if disproportionately loud and/or salient), doesn’t make that view invalid or immoral.
Also, I believe it’s best not to accuse your interlocutor of lacking intellectual honesty unless the evidence is very clear. Perhaps they just see the world differently to you.
I’m happy about people viewing the world differently and having an honest, good-faith discussion about the issues. But I’ll criticise intellectually dishonest arguments, like the moat-and-bailey you just pulled here.
I never argued for that. My point is that researchers (as well as everyone!) have biases, which may manifest in the end as racism or other forms of discriminations. To prevent these biases from manifesting, (AI) ethics experts are needed as oversight.
> the company that employs them is racist
To take the example of the "racist soap dispenser" again: while the individual people who have developed it may not have been racist, they have failed to think outside their biases and ask themselves "can people with non-white skin use the product?". Also, management has failed here, because more diverse staff would have resulted in at least one internal test candidate of color noticing "hey, I can't get soap!".
The result of this lack of diversity and out-of-the-ordinary thinking was that a person of color was rightfully offended at not being dispensed soap. Therefore, the company acted racist even if it never intended to do so.
> the whole country is racist
Just short of half of the country recently elected a President who openly spouted white-supremacist conspiracy theories. For half the people in the US racism is not a dealbreaker for the highest office it has to offer!
... which is a bit different from saying that half the country is racist. Perhaps they thought the alternative was worse.
He was elected by the votes of ~27% of eligible voters, and about ~20% of the population, not “just short of half the country”.
You could say it was a little less than half the people who felt it is worthwhile to vote in a system where votes matter so little that getting the most of them doesn't mean you win, but then the story is about alienation from the electoral system more than support or even indifference to racism.
For the other half, sexism isn’t. (Biden selected his vice-president on the basis of sex.)
In a certain context it just means jews. And they didn't write racist soap dispenser, they wrote "racist soap dispenser".
Do look inwards.
Please define "often". And "offensive" (as in to whom). And of course, you also need to factor in whether being offended is justified.
As an extreme counterexample, a deeply racist person would be offended by a mixed couple. And racists on both sides currently are. I see no reason whatsoever to pander to either side on this, so just there being someone who is or claims to be offended is no guide.
If you find "globalist" offensive, then you are the problem. And, the vast majority of Americans are find this type of PC culture and social justice activism offensive, including the vast majority of "marginalized" groups.
https://www.theatlantic.com/ideas/archive/2018/10/large-majo...
In fact, the only group marginally in favor are white, rich and highly educated. Hmm...
At you the commenter yourself: you seem to be exceptionally harsh on her based on the most minimal information that you’ve purposes to fit your claim of this being “useless things that we shouldn’t care about”. Do you know how much energy goes into training a model? I certain didn’t. Do you think that showing that AI is picking up discriminatory language is something worth looking into? I certainly think having someone look into it would be useful, yes. And Google apparently thought so too. Claiming that these things are idiotic and attacking the author of this paper for bad faith and censure is…very extreme.
One final edit, even her co-authors were not ready for publishing the paper (per article), so why is the reason of censorship, supression etc even being discussed. This whole thing has been blown out of proportion and Google likely did the right thing, purely based on her behavior
So yes, there is more evidence for her toxic behavior than there is for her ability deal with disagreements professionally and work in a team environment.
TBH, I think this whole review things is just a reason for google letting her go. They were probably looking for a reason and she using her threatening (aka toxic) nature was just what google needed.
The people who removed her are executives several skips above her. That is unusual.
You keep bringing up LeCun, but LeCun did not get harmed as you imagine - he only conceded to her point because she was frankly the expert in the topic of discussion and he finally acknowledged that.
Because thats not how I have seen it. She threatened to leave. If any of my team had an issue, and their response 'Do X, Y, X or ill leave', even if they are in the right, I would not align with them purely out of the fact that that is not the right behavior. You can still leave any company with out threats.
Also we have not seen the email to the brain group. I wonder how that compared to her exchange with Lecun and her threatening nature.
Lecun was not harmed? How did she contribute to a healthy debate there? Please also link me. Because all I saw were aggressive tones towards someone she disagreed with (even if she was in the right). Again, he behavior (not if she was right or not) is the problem here. You can be right, but still be an asshole.
Also, how do you know some people in her team didnt have problems with her? Do you have here perf reports? I believe her direct manager is not the only person that can fire her. She can be ok good terms with her direct manager and still behave unacceptably towards higher ups
2. How was LeCun harmed? As far as I'm concerned, he accepted the feedback. And just so we're clear on what happened then - Gebru was only one of many with qualifications who were aggressively critical because of his abdication of research responsibilities. I don't know why you're so focused on Gebru in particular.
3. Yes, if this was about performance, her direct manager who oversees 300 people would know.
4. So yes, glad to see we agree higher ups, who she likely has little to no interaction with to supposedly be exhausted of her behaviors, stepped in to get rid of her.
2. Do you really think lecun wouldn’t have left Twitter if he didn’t find that conversation enjoyable? Do you discount the many other people who found her interactions with lecun horrible? Did any of the other people behave as arrogantly towards lecun as she did? Again, no point made.
3. See 1
4. See 1
So, you can either stop ignoring the fact that her own behavior led to this, or you can continue to think many other respected people in the industry who act way more professionally than her are just against her because they don’t support her views or think google is racist.
You are the only person postulating the intention of her firing. I am merely calling out your assumptions because the consensus from the entire community involved has declared otherwise.
Most other people who found her refutations horrible are outsiders who don’t fully grasp the topic LeCun and Gebru and everyone else was alluding to.
Who are the respected in the industry that are against her? Not even LeCun, who has said he thinks her work is important. Nothing in your post has facts at all.
You dont have to grasp a topic to know if someone is debating respectfully. I dont have to be a doctor to know that the person is an asshole if they act like an asshole towards me.
I think lets just agree to disagree. Focus on the behavior. Not what she is writing about in her papers. And imaging someone acted like that towards you in a workplace. Imagine you are a CEO or a manager and someone comes to you and says 'Do x y or z otherwise im leaving'. Id be like wtf, sure, if thats how you deal with disagreements, cya'. Everyone is replaceable especially toxic people.
How can you objectively comment on her behavior if you have never interacted with the person ever?
This has been discussed in prior threads. A manager is well within his rights to act upon his decisions emotionally. Employees are well within their rights to express concern. If I work in a workplace with severe safety issues, I would most definitely demand an ultimatum for my personal occupational security. Managers DO NOT impulsively fire individuals unless they are mentally unable to navigate through a conflict. It's far more nuanced than that.
https://twitter.com/ylecun/status/1277372578231996424
saying:
"Following my posts of the last week, I'd like to ask everyone to please stop attacking each other via Twitter or other means. In particular, I'd like everyone to please stop attacking @timnitGebru and everyone who has been critical of my posts."
I also find your question about why people attacked her, as if people in the internet always have good reasons to attack others, is an incredibly silly attempt to appeal to the Bandwagon fallacy.
Now go and look at how she has handled it and how she argued with Lecun. Then she also thratens people if she does not get her way. There is ample evidence for her behavior. Does she behave like this all the time? Of course probably not. But the fact that she is openly behaving like this is a clear data point. If you can not see this behavior as not suitable in a work environment, then lets hope we never work together.
Twitter already exists, we don’t need another place online to spend all day spreading false positivity.
A comment that started off with “it seems like Timnit Gebru mat have overestimated the impact…” is not false positivity.
If it is accurate what does it matter?
There are all kinds of research papers; they aren't always describing some new clever trick no one thought of before. Sometimes it's just "hey, we had this question we wanted to know the answer to and so we did some investigation and some math and figured it out." As long as they're questions that some other people somewhere are asking, there's no reason why a respectable conference or journal wouldn't accept such a paper. It's up to them and the reviewers to decide whether the content is a good fit for that venue. If it isn't, it'll get rejected; no reason for Google to involve themselves in the decision.
How rigorously the authors treated these questions in the paper is something we don't know without being able to see the paper itself.
Spot on. Naive scientism is used as cloak to advance political agendas, instead of searching for truth:
"When a domain becomes respected source of Truth, there is incentive for hijackers. There is a huge window of opportunity (from decades to centuries) until the public opinion figures it out. Many (not all) still trust Harvard credentials, as Cardinal credentials in the past." [0]
This is the trend for "Believe in Science", a magical catchphrase evoked to put trust in frauds like Steven Pinker and Paul Krugman:
https://books.google.com/ngrams/graph?content=believe+in+sci...
[0] https://twitter.com/_benoux_/status/1335175823620497413
[0] https://www.academia.edu/26772813/The_Decline_of_Violent_Con...
Please take a look at (Wieringa, R., Maiden, N., Mead, N., & Rolland, C. (2005). Requirements engineering paper classification and evaluation criteria: A proposal and a discussion. Requirements Engineering, 11(1), 102–107) available here: http://www.cse.chalmers.se/~feldt/advice/wieringa_2006_re_pa... Specifically read the section 3 (which is on p. 4). There you will find that research in software engineering can be broadly grouped into the following categories: Evaluation research, Proposal of solution, Validation research, Philosophical papers, Opinion papers, Personal experience papers.
While I agree that the usual way to deal with the criticism of the work is to redo the work with better evaluation (and thus, more strongly supported arguments), I think that in any ethics research you'd need more than just publications that fall under Evaluation research (which is what you are likely referring to as "true AI research").
Your goal here wasn’t to make any useful point about how it was actually a reasonable decision for her to resign/be fired from Google. It wasn’t to comment on the process or difficulty of having these discussions at scale or in public, or even respond to the specific points raised in the article. These would all be valid discussions to have. It was specifically a personal attack based on a paper you haven’t actually read intended to demonstrate how much better you are than everyone else.
Absolutely done with this shit.