the comment you are replying to is saying that if the data is representative of the world, you will then be termed biased because the people doing the labelling think the world is biased.
I can tell a human what they’re doing and they can be aware of their bias and guard against it. This is part of the point of “unconscious bias” training. AI can’t do this.
If you’re saying humans are consciously hiring people named Jared, they shouldn’t be allowed to provide hiring feedback.
AI would probably do it less than humans. If AI also does it, then Jareds probably really are better the employees.
(Or the AI was used incompetently. That's always an option. Was AI the reason why US universities discriminated illegally against non-blacks and non-hispanics?
I really doubt it.)
To those of you thinking of debating the above, remember, you cannot win the word game. The more you oppose someone telling you not to use a word, the more racist you appear, no matter the actual origins of the word in question. It simply doesn't matter. The only way to not be seen as a racist is to apologize and use whatever words they tell you to use.
I don’t think any rational people are offended by the word “blacklist”.
I do think there is some validity to the change.
Black lists are lists of bad things that need to be blocked or prevented, white lists are things that are good and allowed and wanted. That does create an unsavory connotation with the words themselves.
Furthermore, as someone in an industry that used blacklist and whitelist extensively, noobs screw up those words all the time. It’s not a very clear word. Blocklist and allowlist are much less ambiguous.
This is the correct response. The origins of the terms are less important here, and often people get hung up on them.
Whitelist and blacklist are indeed ambiguous, and they also create negative connotations that could reinforce other, actually racist views and language, whether conscious or not.
I most enjoy the terminology updates because anyone that throws a fit about the change outs themselves as a can of worms and a person to keep away from your life or payroll. IME it's a heuristic with 100% accuracy.
This reads like you enjoy antagonizing people, so I'm sure the feeling is mutual. Who would want to work with/for someone who enjoys agitating their peers or subordinates? Do you ask them their stance on abortion and free speech too?
You can also win this game 99% of the time by just not living in California. It’s really just a local power struggle that some people mistakenly believe is a national dialogue.
And the reason for this is because for most people a word substation is a zero effort change that simply doesn't matter one way or another. And if one way is better for whatever reason then sure, why not? One side cares because they're trying to eliminate bias and be inclusive, the other side cares because…?
"words have meaning": this is a right wing dog whistle, even if you are a true language prescriptivist no one will believe you.
"it's stupid": just makes you look like an asshole when someone is trying to do something people see as in good faith.
"it's pointless": then why does it matter one way or another? Goto other reason.
"it's not racist": when we're talking about blacklists it never was, it's about trying to not always associate black with the negative connotation, but you look really racist for wanting to keep it.
"black outside the context of race does have negative connotations in humans, you can't change that we're afraid of the dark": now you look really really racist.
Have you ever considered that people don't like being told how to behave by unlikable people? And that language policing by definition makes you unlikable? When was the last time you told someone not to use a word and they said "you know what, I like being told how not to speak." ? It's maybe the quickest way to tell everyone that you're insufferable.
So basically you're "proving" to yourself that everyone is either racist or an asshole, by creating a situation where people don't want to listen to you because you're insufferable. But you can pretend to be virtuous because you're doing it for some make believe greater good about the word "whitelist."
BTW I'm actually trying to help you see the reason there's resistance. I use "allowlist" and "denylist" voluntarily in my OSS repos, because it doesn't really matter to me. But I still find people who make it their moral imperative to police these words insufferable. From my perspective, you either don't know that, which shows you have no tact, or you don't care, which shows that you're doing this in bad faith.
You're writing this like I don't agree with you and do the exact same thing. Explaining the social dynamics at play doesn't mean I agree with them.
> When was the last time you told someone not to use a word
Literally never. Again you assume a lot about me. I am speaking purely from the perspective of being told to use different words. It's
never really bothered me. My response has always been "sure whatever." Some of them are a stretch but a lot of them make sense. No one has ever made me to feel bad about it.
Clearly someone hurt you and made you feel bad and I'm sorry for that.
You listed every other reason why someone would resist being policed about the word "whitelist", except the "you're being obnoxious" reason. In my experience, that's the far more common reason. Instead, you seem fixated on the racist reasons, since you mention them multiple times.
Believe it or not, people are more likely to be put off by obnoxious word police than they are to be racist.
> The more you oppose someone telling you not to use a word, the more racist you appear, no matter the actual origins of the word in question.
And I expounded why -- literally agreeing with you. Then you called me insufferable. If people took "you're obnoxious" as a valid defense then there would be a way to win the game. Your entire point is that all roads lead to racism and then got defensive when in fact all roads lead to racism. As much as it would be fun to be an omnipotent benevolent dictatress, am I in fact not and have no power to change how people behave.
Also you're clearly mad about word policing but your gripe with them is that you think the way they deliver the message is obnoxious and insufferable. But since you did voluntarily switch up your wording you clearly have no issue with the message itself. Which means you're tone policing. I bet some folks view you the same way you view them.
The problem with "allowlist" is it is an ugly phrase that sounds like it was dreamt up by a bureaucrat. There is a reason we have blacklist and whitelist - they roll off the tongue. The move from master -> main was silly but at least technically superior because it is easier to say and type. "Blocklist" and "Allowlist" are worse words than "blacklist" and "whitelist" because they are the same to type and harder to pronounce aloud.
Whoever cooks up these schemes should have put more effort in. Even something like passlist and stoplist is easier to say. Throughlist. Something that sounds nice in a foreign language, maybe.
That’s all just a matter of preference though. I personally like how allowlist and blocklist sound, and they also immediately convey intention.
You may also be biased towards whitelist and blacklist if you’ve heard them your entire life - if you had grown up with allowlist and blocklist, can you say for certain that someone suddenly saying “we should use whitelist and blacklist” wouldn’t give you the same feeling of clunkiness?
You’re right. They exist in the context in which they’re used. In this case that context is as a technical term that has no association at all to any racist ideas. I would advise you to consider that you do not exist in a vacuum, and perhaps you should reconsider your anti-social impulse to police other peoples’ use of innocent words. Forcing people to remove words from their vocabulary because they remind you of some upsetting concept is not inclusive at all. It is by definition exclusionary.
The move to more neutral language isn't about policing speech. It's about being aware of potential unintentional biases our words may carry and promoting a comfortable, inclusive environment. It's a dialogue, not a mandate
> My attempt to police your speech wasn’t about policing speech. It was about about creating a dialogue, and making sure you know that you are racist if you don’t agree with me, not mandating that you agree with me.
Haha, ok buddy.
If you want to be more inclusive, perhaps you could go to the effort of understanding the vocabulary of my trade and the meaning of those words, rather than try to prevent me from using them. Because I’m feeling very excluded by you right now. Do you care about the lived experience I’m having right now? If not, why?
This isn't about policing or labeling anyone. It's about opening a dialogue for potentially more inclusive terms in our field. No intention to make you feel excluded, only to make room for discussion.
This comment is really the height of gaslighting. These D&I topics have never been an open dialogue, and it’s never been the intention for them to be. Nobody who objects to a D&I project feels as though their input is welcome, and those who are fool enough to voice their objections invariably suffer the consequences for it. These “open dialogues” are about proposing a change, with the implication (or by just explicitly stating) that any objection to the change is racist/sexist/whatever… and would victimise minorities.
The term whitelist is a legitimate piece of technical jargon, and has no racist origin. That is the end of any reasonable discussion on this topic. If you wish to litigate that point any further, that only direction that could possibly take is into wildly unreasonable territory. Renaming anything that people may feel is associated with other problematic concepts has no finish line. There’s an infinite quantity of problems you could invent using this approach, and all of them would be completely equal in their unreasonableness. The only way you could potentially limit this scope is to elevate the unreasonable feelings of one group over the unreasonable feelings of all other groups. Which is an exclusionary practice, and goes to show how subversive the use of language like “inclusion” is here.
Ultimately, the immediate material impact of using term A or term B within a corporate environment is a piddling
While the direct impact of changing a term might seem insignificant, it's about mutual respect and understanding. This isn't language policing, but sparking awareness. Your feelings are valid, your input is welcome. Let's keep talking, even amidst disagreements. Progress lives in dialogue, not silence.
The words in context have no racist history, but consider a young black student interested in network engineering. They start learning that “good services go on the whitelist” and “bad services go on the blacklist”. I can see a potential for that to rub them the wrong way, and while it is a small thing they would likely shrug off, it may also get filed into the box of a million other things that they have to be aware of that are racist in meaning. It’s one more thing that could reinforce the ideals that systemic racism drives into their life.
A case could also be made for people who aren’t aware of the context: are they confused because the terms aren’t familiar? Does their own bias influence them to understand the meaning and infer white = good, black = bad? If so, is that not more reason to change the words? Shouldn’t we move away from language that associates good and bad with colors that are often used as tools of division?
No, I’m saying brains - especially young brains - filter and see the world differently. They make assumptions or correlations that may not be accurate.
Regardless of who the audience is, the terms equate a color to either being good or bad and it’s kind of silly when terms like allowlist and blocklist are immediately parseable into what they mean.
I don’t particularly care other than if there’s some tiny segment who feels the terms could be even remotely offensive, and we can all agree that regardless there are better terms, why not use them?
Does it matter? I also like it when my finances “are in the black” as opposed to “in the red”. I don’t know a ton of African Americans, but I have talked with some of them about this. Although they definitely experience racism, none of them feels terms like “whitelist” is exclusionary. This feels like seeking for reasons to be hurt. Same with “master” and “slave”. There is a lot of history of white slaves in Europe, but it doesn’t hurt my feelings even though my ancestry is from those nations.
Please edit your post to remove these words -
Granting - starts with Grant, same name as a slave-owning US president
Consider - starts with con, (con/ ex-con), used to shame and discriminate against people who have committed crimes but served their sentences
Vacuum - often used with sexist connotations suggesting women should be the primary user of this tool as a means of housekeeping
Pain - perceived differently by different populations, for instance redheads experience more pain than other hair colors
Promote - likely to remind people of unfair corporate ladder practices
Atmosphere - seen as a positive for earth dwellers, suggesting we are superior to planets which do not have atmospheres
White and black are technical terms used in physics and electrical engineering. White means all of the components of the spectrum and black means none of the components of the spectrum.
White noise contains all the frequencies.
Black body radiation is radiation due to heat emitted from an object which does not reflect any light (is black).
> That is the etymology of whitelist and blacklist.
It's hard finding a definitive source for blacklist, though first time reading someone suggesting it's from that sense. Many centuries old examples I've seen when looking this up seem to have already had it as some in-use vocabulary.
Edit: huh, entire parent thread has been entirely hidden on HN. I suppose it was too much of a digression.
The etymology of blacklist in English seems to be more related to a list of his enemies that Prince John (of Robin Hood fame) made of his enemies. Most likely the word is related to the MUCH older association between the color black and death in much of Europe.
Still, nothing to do with black or white skin or any bias against people of different skin colors, so not contesting your actual point.
>Most likely the word is related to the MUCH older association between the color black and death in much of Europe.
I would have thought it would have something to do with darkness/night versus bright days/day, where people experience more danger in the former than the latter for obvious reasons.
Even without the bias issues, if I were a hiring manager how do I verify that the weights "MagicHireAI SaaS Corp" are using aren't leaving swaths of talent on the table that I don't even know exist?
Yes. All these solutions are snake oil. Resumes are BS—it’s the very definition of garbage in garbage out. I don’t need an AI to sort candidates, I need really fast organizing tools.
My wife recently went through multiple applications after a stint of no work permit.
The amount of stuff they make you fill out that is then parsed with AI is insane.
Indeed ruined a lot. Before if you had an ok resume but we're able to talk to them in person you might be able to convince them.
Now with all those aptitude training you have to do well there.
Not to mention the applications that require you to turn on your webcam (!) while you do them. Disgusting.
That last one by the way was the same company where my wife had an in person interview. She drove there. The interview was on Zoom with somebody in another office.
She declined the offer. Wastefulness and already showing that they're not against monitoring by AI means 0 humanity.
> Now with all those aptitude training [...] that require you to turn on your webcam
They are referring to an automated personality or skills test to screen candidates. The webcam is to make sure the applicant is the one actually doing the test. These things are dehumanizing and snake-oil but there’s’s no question that applicant misrepresentation happens, including situations where the person interviewing is a ringer being paid by an agency or the applicant.
I guess I don’t understand how cheating on these things benefits the candidate. Like, you lie about any of that and get put in front of a panel, how’s that going to go?
Generally the people trying to fake their identity or skills in interviews are doing so with remote jobs, with the goal of bumbling along indefinitely once they no longer need to be on camera.
The problem here is that companies have now gone from identity verification to this 'aptitude test' ML bullshit that's just another cover for increasingly weird arbitrary discrimination against applicants.
It doesn't benefit genuine candidates to blatantly lie. They'll get caught later in the process. But you're assuming identity thieves are acting rationally or fear getting caught. They don't, and they're playing a numbers game. If you get past an interviewer and get fired after two weeks, your salary at a US tech company can be $5k, which is the average annual salary in many developing countries.
I don't know about other folks here, but in the past decade, we've encountered recruiters who present one candidate when doing a telephone interview and then a different person shows up for the in-person interview.
The idea is that you keep your webcam on your face while you fill out the application for identity verification, to make sure it is actually the applicant. But if you think that's bad, the hot new thing in these hiring startups is that you have to record yourself answering interview questions with some chatbot interviewer, and then some black box model scores it to give you all kinds of psychometric astrology even worse than Myers Briggs. Except it's a bunch of AI snake oil, and as the article says, you get different personality scores if you put on a headscarf or glasses.
In the US, it is illegal per the EEOC, which has barely any resources to actually enforce the law: "Similarly, employers should not ask for a photograph of an applicant. If needed for identification purposes, a photograph may be obtained after an offer of employment is made and accepted." [1]
It is not illegal to ask for a photo. “Should not ask” is not the same as “cannot ask”.
Asking for a photo opens an employer to claims of discrimination, but it would still have to be proven, so the government is advising against it. But it may well be that in practice, the government will not be spending resources on trying to or be able to prove discrimination, even if a photo is required.
Those of us with authority to hire should reject these snake oil practices. What a waste of time to only get candidates who are okay sitting through these dehumanizing experiences.
Don't pay for this cruft, sponsor local meetups instead.
A core lesson in ML: this is difficult or impossible. See the classic example with wolves, dogs, and snow. [1]
For a resume example, suppose you don't want to be biased again black candidates, or women. Well, good luck trying to filter out resume experience, skills, schools, and interests so that they don't list anything that is statistically more likely to be found among black folks or women. It's impossible.
So now you have a hiring process that looks at... what exactly?
That is not the same problem. That’s a sample bias problem.
The whole point of white listing variables is to only allow things you are comfortable using todiscimrinate. And yes, that likely reduces the power of the model.
Edit: to clarify, your example had the model failing to classify something correctly because the training data did not generalize across real world examples. The problem with diversity bias is that the modes are trained on past data where the labels themselves tend to have some bias. Black candidates do worse. So recreatingthese will generally produce worse outcomes for minority candidates. Now for a lot of these cases the average minority individual is more likely to do worse. Not intrinsically because of race, but still worse.
You can’t realistically expect the model to produce equal outcomes for all classes when all classes are not equal on the actual traits. But you do want to try and make it so that the model doesn’t use race indicators as a crutch that ignores actual good indicators. The common complaint is you hear about a man and a woman applying with the exact same credentials but different outcomes. Well obviously the model had to have additional information besides those credentials if it made different decisions. If this causes problems, you probably should not permit those additional pieces of information.
> The whole point of white listing variables is to only allow things you are comfortable using todiscimrinate.
The problem is that variables where you can safely discriminate don't exist in the real world because everything is intertwined.
For example, if you see that somebody went to school at {Howard, Bryn Mawr, BYU} you can make some pretty solid inferences about their {race, gender, religion} even if they don't explicitly provide them.
> Bryn Mawr is a womens' school and admits only women. However, gender is not always fixed going through life.
Technically true of course, but >99% of the population will identify with a single gender throughout their life. The correlation is so strong that it's not even really a correlation anymore.
If you're going with a probabilistic argument, I think BYU students are more likely to stick with mormonism (let alone Christianity) than Bryn Mawr students with their gender.
That doesn't mean you aren't stereotyping people, though.
Irrespective of the AI discussion (computers don't care if you're stereotyping or not), I really don't think it's stereotyping to assume someone who went to a school which literally only admits women is probably a woman. There's a difference between assuming someone's gender because they're a cheerleader, and assuming because they won the women's cheerleading championship. I also assume that the person who drives a fire truck is a fire fighter.
No, again, it is inevitable that outcomes will not be equitable across race. That doesn’t mean all variables are unjustified to use.
Job applications are kind of dumb. The common use case is credit decisions. Surely, for example, you’re ok looking at credit history and income when deciding to make a loan, right? Even though it won’t be equitable outcomes across races, it is still “fair”
The goal of fairness ML models is not necessarily to have equal outcomes across subgroups.
Other goals can be to have equal model accuracy across subgroups (e.g., models has the same precision/auroc/etc for each subgroup, even if the predictions themselves differ across subgroups) or identical distribution of statistical biases across subgroups (e.g., the model makes the same types of mistakes within each subgroup)
While a lot of people will state those are appropriate goals, they are not. They’re really dumb goals. The model being equally bad for each group is not an indicator that you have achieved anything. Most likely you’ve just made your model for white people worse.
Like, let’s look at that for credit decisions. Black folks have a lower income distribution. So it’s like saying the model is ok if it rejects a Black applicant making 80k if it would also reject a White candidate making 100k. That is the insane bullshit you’re effectively supporting by desiring comparable error distributions.
You have to accept that equal outcomes are not just impossible but not desirable. And then you only use fields that people agree are fair to use. If Black people get disproportionately better or worse outcomes it is either because those outcomes make sense given their stats or it’s not ok because if a White dude had the same info, that you agree is fair to use, they would have passed and the underlying reason for the difference is some bullshit other input.
That might mean that the model is a dumb: you must make $100k for this loan (illustrative) which is going to mean a lot of Black folks cannot get it. Well that model is at least “fair”. Accommodations for race should not be coming from an AI model
Why not simply target the advertising of the job at traditionally black universities, publications, and social networks? Why all the ai mumbo jumbo to filter resumes from a general marketing strategy?
Unfortunately this doesn't work, one of the basic things to know about ML fairness is that you generally can't get the outcomes you want by hiding things from your model.
For example you can infer the race of a candidate from other facts about them. The model could still illegally discriminate by first inferring race, then discriminating on the basis of inferred race.
You will end up adding things to your whitelist that you think are innocuous, but if that's all you do you will very likely end up discriminating and not even knowing it.
While I agree there should be regulations, there should be clear requirements instead of saying "bias". What does bias mean? To clarify, here are a few examples:
1. Gender bias
There is a widely used keyword-based gender bias measurement based on a study from 2011. The research claims that job ads with higher levels have more masculine words. This is based on a pre-defined list of masculine and feminine words [0][1]. Another Harvard study stated that women would only apply for jobs if they meet 100% of the requirements, while men would apply for 60% or less [2].
2. Racial bias
This might be more related to video interviews or based on name, age, or address in resumes. For resumes, there are tools already to anonymise resumes to ensure the focus is on experience instead of the person.
3. Disability
ADA (American Disability Act). According to this FAQ [3], "The ADA does not require employers to develop or maintain job descriptions. However, a written job description that is prepared before advertising or interviewing applicants for a job will be considered as evidence along with other relevant factors."
The list could go on, with age, sexual orientation, native speaker or not ... etc. That's why the expectations have to be clearly defined.
The law, as I understand it, will lock employers to their vendors as they will be required to keep data for auditing, which might incur more charges.
I didn’t directly cite all relevant code that exists, there is more regulation cited by the links I gave you might need to follow. It’s easier to visit https://www.eeoc.gov/ and browse their “discrimination by type” pages under the “employees” and “employers” tabs. I’m not quite sure exactly what you’re getting at, but the EEOC doesn’t really use the word “bias” per se, they talk about discrimination on the basis of sex, age, disability, race, national origin, etc.
The NY law uses the word ‘bias’ pretty much only in the phrase “bias audit”, which is defined in the law to mean a yearly audit of the software to make sure it is not discriminating as far as EEOC standards.
The point is that there isn’t a new law about bias, and it’s not an ambiguous or undefined concept. It’s the existing equal employment opportunity laws, and all employers already know about them, and most (I hope) already adhere to them. I’m not suggesting it’s either complete or perfect, I’m just pointing out what the new NY law is referring to when it uses the words “bias audit”.
Just to add to this, I feel like there's a ton of ways to "accidentally" introduce a lot of bias in hiring: referring everyone from your college's club to the company, brain teasers that require some irrelevant cultural background, mandatory in office/remote cultures, off site decisions. Heck I even saw a company that advertised that they go surfing together every weekend for bonding.
I'm not sure what can be done with these cases, and I know people, of various protected out-groups, that have specifically mentioned those as reasons not to join a company.
The companies that remove the human touch from hiring will rapidly wither and die, leaving the real talent for the rest of us who value people over process.
I hear about job applications and interviews these days and they sound so daunting.
My first job as a developer in 2006 I sat with the lead developer, had a very comfortable chat about things I had built, things I was interested in both related to programming and otherwise. Conversation turned to math and algorithms a little bit. He stepped out and came back with an offer. Whole thing took maybe 45 minutes.
I honestly feel as far as interviews have gone, it did the best of honestly sizing me up as a person.
The problem is that the modern tech interview process was created by large companies like Google. They needed an “unbiased”, scalable process to hire thousands of engineers with the main goal of reducing false-positives.
Most companies don’t operate anywhere close to this scale and should not imitate this same hiring process.
What they've essentially produced might be slightly better than randomly accepting 5% of the applicants that make it to the interview stage. Probably not much better though, and much less efficient.
If Amazon used these techniques to hire its warehouse workers, it won't wither and die a bit.
We software engineers are used to be in high demand and somehow short supply, even now when it's taking me more than two months to find a new position. In many other industries supply noticeably exceeds demand, and people would go to great lengths and jump through really silly hoops just to get hired.
> * Social media scrapers that collect data on applicants to compile personality profiles based on what they’ve found online
> * AI-based chatbots that ask candidates questions about their qualifications, then decide if they’ll proceed in the interview process
> * Algorithmic video platforms that have candidates answer interview questions on camera, record their replies, transcribe their responses, and analyze their vocal or facial patterns for subjective traits like “openness” or “conscientiousness”
> * Logic games that purport to identify qualities like “risk-taking” or “generosity”
Alright, which one of us monsters built any of this dystopian garbage?
(We certainly are monsters with a sense of humor: see the example of deploying Kafkaesque automated snake oil gatekeeping to people's livelihoods... to seek "conscientiousness".)
The focus on bias is a transparent ruse imo. Implying that a dystopian unethical process is ok because it's the same for all demographics and therefore "ok" is absurd, and that's what most of the laws or proposed laws I've seen want to focus on. It's the typical modern grift of throwing up identity politics as a big smoke bomb to obscure way worse stuff.
on startupschool I met several groups that are trying this. One of them, when I asked for a follow up meeting about ethics, instead took a small write up I wrote to respond to their technical needs (...literally a two page write up) and immediately sent it to upwork. Three weeks later, lol, when that didn't work out, they contacted me again. I'm not going to disclose who it was, but I have a new policy now: Absolutely, under no circumstances will I consider working with former McKinsey partners.
Merely anecdotal but my worst experiences over the last year have been with startupschool companies - not saying they’re all bad (maybe I’ve been meeting the statistical minority that aren’t great) but as a part of the community I really expected better
BTW, I totally understand the once-bitten-twice-shy, and McKinsey isn't a marginalized group needing protection, so personal aversions seem OK here. :) (I just bowed-out of an otherwise great co-founder situation, due to realizing a 2-hop connection to a party that I will never willingly go anywhere near again, even though it probably wouldn't affect the business itself.)
But I feel personally obligated to just note, for any HNers reading, that the two ex-McKinsey people I've worked with closely at a startup were both decent, and I'd work with them again.
Heh, maybe we can get McKinsey on our side, on this particular AI dystopia threat. Some hypothetical vendor-selection AI model (comparable to a hiring one) might take one look at certain past scandal involvements of the firm, and denylist anyone associated. But even organizations that don't care about individual fairness might still have to care about some AI pet project sabotaging the quality of their people and vendors.
Not that this would justify it, but I'm curious if it even works. I mean I'm certain the people selling these things have some sleek presentations showing some datums with a million percent improvement, but I'm curious about for companies that have existed for some time. Does this dystopian mess even drive substantially better outcomes than they were getting using 'normal' hiring methods of the sort we've used for millennia? My intuition is screaming no.
I also think the argument that big companies wouldn't do it if didn't work is false. This is the sort of thing you're going to be able to create some big awesome presentation for, demonstrating that not only does it work, but it will end up saving you large amounts of money. Even if it does nothing of the sort. There's just so much room for number jukery, buzzwordery, and general shenaniganry - even if you never overtly lie.
Not that this would justify it, but I'm curious if it even works.
Like most other things companies do in the hiring process (e.g. endless redundant interviewing rounds, ridiculous culture fit questions) -- or the stuff they do in their day-to-day work ("Agile") -- it doesn't need to actually "work" for it to be widely adopted.
It just needs to appear to work, and to give off a warm, buzzy feeling.
> Not that this would justify it, but I'm curious if it even works.
I would think that's one of the primary concerns here, is that we can pretty much guarantee that it doesn't work. Then end result would turn hiring processes into a kafka-esque hellscape where reason has no bearing and candidates are essentially selected at random.
Given how infuriating the current crop of simulated customer service agents are, I can't imagine bringing anything like this to job selection would be good for any of us, save for the leadership team who gets to tout their cost-saving plan and give themselves a bigger bonus.
Edit: ugh after reading the article I'm so disappointed that the regulations amount to some pretty weak auditing and transparency requirements. I was hoping this was an outright ban. It seems the laws have been rewritten by the cooperations. It gives candidate "right" to ask questions about their use and possibly opt out, but all that means is anyone who speaks out will be unhireable. I mean what company is going to hire anyone who starts asserting their rights up front?
> Then end result would turn hiring processes into a kafka-esque hellscape where reason has no bearing and candidates are essentially selected at random
That’s pretty much what it is today isn’t it. You get into a room with a hiring manager and if you happen to have kids the same age or saw the same movie growing up, you’re gonna have a huge leg up.
Can we blame someone for at least trying to make a system that’s more objective
Probably people who after interviewing their 500th bozo throughout their career, thinking "there has to be a way to automate this bullshit", or some similar B2B need that popped up over and over again that they were in a position to observe.
> collect data on applicants to compile personality profiles
I think this is inevitable, in one form or another. It will just be outsourced to some other companies which will rehash and extract whatever data, and sell services like the current background-checking companies, without even exposing any specific data to the hiring company. It's time to bit the bullet and remember that everything you say online publicly is going to remain in your permanent record, even with GDPR and stuff in place. It can't be countered by regulations, much like encryption can't be made transparent only to law enforcement by regulations. What once has been made public cannot be unmade public.
> AI-based chatbots that ask candidates questions about their qualifications, then decide if they’ll proceed
I suppose non-AI analysis of forms is not going anywhere.
> analyze their vocal or facial patterns for subjective traits like “openness” or “conscientiousness”
> Logic games that purport to identify qualities like “risk-taking” or “generosity”
I think it's just so stupid that I'd care more about my right to have disclosed the fact that a particular company does that.
On the other hand, this means that the state need to put its nose much deeper into the hiring process of a private company than it used to. It's the definition of public oversight, of course, but usually it applies to public offices.
I've interviewed a person for a tech job who took about 15 minutes to wrestle a for loop that wasn't obviously broken into an online coding tool. It didn't improve from there. Given that people who fail interview apply to more jobs, and ok candidates get hired and don't apply for too many jobs, the number of bad candidate interviews is disproportionate... I think it would be a great idea to save time with AI.
These horrible processes remind me of applications that include IQ, “personality type” or other phrenological tests. Are those banned in any jurisdictions?
This law appears to define fairness using "demographic parity", aka a model is assumed to be fair if equal fractions of each candidate group is selected.
So if 20 women and 100 men apply, it would be considered unfair to accept 10 women and 20 men (as 10/20 != 20/100).
Where are you getting that? It does no such thing, please don’t concoct absurd stories. You can read the law in its entirety since it was linked in the article. [1] It doesn’t even try to define fairness or say a thing about demographics, it only requires a regular audit, and it relies entirely on already-existing Federal Equal Employment Regulations, which does not model demographics in equal fractions.
I interviewed at a handful of AI powered recruiting software startups and universally they hand waved their way around any questions about bias, so glad to see NYC leading the charge here.
Think of the type of people willing to work at a company that entrusts the most critical action of a company, hiring of employees, to an artificial intelligence? It’s one thing to filter resumes with an ai, but using an ai to rate an in-person interview is dereliction of duty.
I was a candidate in a HireVue interview a few years ago (mentioned in article) and it was a bad experience. I would get prompted with a question and then record myself answering the question (all inside their software), knowing that were using some kind of "AI" analysis on my responses. I didn't actually get to speak with a real person during that interview stage. I did make it to the next stage but thankfully found another job not at that company.
One of my favorite jokes on the topic is "What is the difference between an inquisitor and a bioethicist?" "An inquisitor deserves some forgiveness, he was the product of an unenlightened age and didn't know any better."
I guess I'd have to generalize it from bio- to also apply to tech now.
138 comments
[ 2.7 ms ] story [ 189 ms ] threadi want a highly biased sample of attributes in the extreme tail end which will have all sorts of weirdness in it.
at least 2/3 of the population are unteachable for the management/engineering i want.
i don't want applicants that look anything like normal.
is that now illegal?
Which part of the article did you skip?
Start with the part where AI hiring tools favor people named "Jared."
If you’re saying humans are consciously hiring people named Jared, they shouldn’t be allowed to provide hiring feedback.
(Or the AI was used incompetently. That's always an option. Was AI the reason why US universities discriminated illegally against non-blacks and non-hispanics? I really doubt it.)
I do think there is some validity to the change.
Black lists are lists of bad things that need to be blocked or prevented, white lists are things that are good and allowed and wanted. That does create an unsavory connotation with the words themselves.
Furthermore, as someone in an industry that used blacklist and whitelist extensively, noobs screw up those words all the time. It’s not a very clear word. Blocklist and allowlist are much less ambiguous.
Whitelist and blacklist are indeed ambiguous, and they also create negative connotations that could reinforce other, actually racist views and language, whether conscious or not.
But hey, i can do that, too:
> no matter the actual origins of the word in question
> The only way to not be seen as a racist is to apologize and use whatever words they tell you to use.
Please stop using the words "debating", "win", "game", "oppose", "racist", "apologize". They are highly offensive and racist vocabulary.
"words have meaning": this is a right wing dog whistle, even if you are a true language prescriptivist no one will believe you.
"it's stupid": just makes you look like an asshole when someone is trying to do something people see as in good faith.
"it's pointless": then why does it matter one way or another? Goto other reason.
"it's not racist": when we're talking about blacklists it never was, it's about trying to not always associate black with the negative connotation, but you look really racist for wanting to keep it.
"black outside the context of race does have negative connotations in humans, you can't change that we're afraid of the dark": now you look really really racist.
So basically you're "proving" to yourself that everyone is either racist or an asshole, by creating a situation where people don't want to listen to you because you're insufferable. But you can pretend to be virtuous because you're doing it for some make believe greater good about the word "whitelist."
BTW I'm actually trying to help you see the reason there's resistance. I use "allowlist" and "denylist" voluntarily in my OSS repos, because it doesn't really matter to me. But I still find people who make it their moral imperative to police these words insufferable. From my perspective, you either don't know that, which shows you have no tact, or you don't care, which shows that you're doing this in bad faith.
> When was the last time you told someone not to use a word
Literally never. Again you assume a lot about me. I am speaking purely from the perspective of being told to use different words. It's never really bothered me. My response has always been "sure whatever." Some of them are a stretch but a lot of them make sense. No one has ever made me to feel bad about it.
Clearly someone hurt you and made you feel bad and I'm sorry for that.
Believe it or not, people are more likely to be put off by obnoxious word police than they are to be racist.
Okay let's go back to the beginning here.
> The more you oppose someone telling you not to use a word, the more racist you appear, no matter the actual origins of the word in question.
And I expounded why -- literally agreeing with you. Then you called me insufferable. If people took "you're obnoxious" as a valid defense then there would be a way to win the game. Your entire point is that all roads lead to racism and then got defensive when in fact all roads lead to racism. As much as it would be fun to be an omnipotent benevolent dictatress, am I in fact not and have no power to change how people behave.
Also you're clearly mad about word policing but your gripe with them is that you think the way they deliver the message is obnoxious and insufferable. But since you did voluntarily switch up your wording you clearly have no issue with the message itself. Which means you're tone policing. I bet some folks view you the same way you view them.
Whoever cooks up these schemes should have put more effort in. Even something like passlist and stoplist is easier to say. Throughlist. Something that sounds nice in a foreign language, maybe.
You may also be biased towards whitelist and blacklist if you’ve heard them your entire life - if you had grown up with allowlist and blocklist, can you say for certain that someone suddenly saying “we should use whitelist and blacklist” wouldn’t give you the same feeling of clunkiness?
You’re right. They exist in the context in which they’re used. In this case that context is as a technical term that has no association at all to any racist ideas. I would advise you to consider that you do not exist in a vacuum, and perhaps you should reconsider your anti-social impulse to police other peoples’ use of innocent words. Forcing people to remove words from their vocabulary because they remind you of some upsetting concept is not inclusive at all. It is by definition exclusionary.
Haha, ok buddy.
If you want to be more inclusive, perhaps you could go to the effort of understanding the vocabulary of my trade and the meaning of those words, rather than try to prevent me from using them. Because I’m feeling very excluded by you right now. Do you care about the lived experience I’m having right now? If not, why?
The term whitelist is a legitimate piece of technical jargon, and has no racist origin. That is the end of any reasonable discussion on this topic. If you wish to litigate that point any further, that only direction that could possibly take is into wildly unreasonable territory. Renaming anything that people may feel is associated with other problematic concepts has no finish line. There’s an infinite quantity of problems you could invent using this approach, and all of them would be completely equal in their unreasonableness. The only way you could potentially limit this scope is to elevate the unreasonable feelings of one group over the unreasonable feelings of all other groups. Which is an exclusionary practice, and goes to show how subversive the use of language like “inclusion” is here.
While the direct impact of changing a term might seem insignificant, it's about mutual respect and understanding. This isn't language policing, but sparking awareness. Your feelings are valid, your input is welcome. Let's keep talking, even amidst disagreements. Progress lives in dialogue, not silence.
A case could also be made for people who aren’t aware of the context: are they confused because the terms aren’t familiar? Does their own bias influence them to understand the meaning and infer white = good, black = bad? If so, is that not more reason to change the words? Shouldn’t we move away from language that associates good and bad with colors that are often used as tools of division?
Regardless of who the audience is, the terms equate a color to either being good or bad and it’s kind of silly when terms like allowlist and blocklist are immediately parseable into what they mean.
I don’t particularly care other than if there’s some tiny segment who feels the terms could be even remotely offensive, and we can all agree that regardless there are better terms, why not use them?
Both terms of phrase have just as much racist connotation which is to say none.
I seriously 100% agree.
White noise contains all the frequencies. Black body radiation is radiation due to heat emitted from an object which does not reflect any light (is black).
That is the etymology of whitelist and blacklist.
It's hard finding a definitive source for blacklist, though first time reading someone suggesting it's from that sense. Many centuries old examples I've seen when looking this up seem to have already had it as some in-use vocabulary.
Edit: huh, entire parent thread has been entirely hidden on HN. I suppose it was too much of a digression.
Still, nothing to do with black or white skin or any bias against people of different skin colors, so not contesting your actual point.
I would have thought it would have something to do with darkness/night versus bright days/day, where people experience more danger in the former than the latter for obvious reasons.
Even without the bias issues, if I were a hiring manager how do I verify that the weights "MagicHireAI SaaS Corp" are using aren't leaving swaths of talent on the table that I don't even know exist?
The amount of stuff they make you fill out that is then parsed with AI is insane.
Indeed ruined a lot. Before if you had an ok resume but we're able to talk to them in person you might be able to convince them.
Now with all those aptitude training you have to do well there.
Not to mention the applications that require you to turn on your webcam (!) while you do them. Disgusting.
That last one by the way was the same company where my wife had an in person interview. She drove there. The interview was on Zoom with somebody in another office.
She declined the offer. Wastefulness and already showing that they're not against monitoring by AI means 0 humanity.
I know I have to.
I know I don’t have to.
I don't understand what this means. Did she have to turn on her webcam to upload her cv? How does that even make sense.
Also, how is that legal? I thought requiring a photograph to apply for a position is illegal (except for stuff like acting).
The problem here is that companies have now gone from identity verification to this 'aptitude test' ML bullshit that's just another cover for increasingly weird arbitrary discrimination against applicants.
It doesn't benefit genuine candidates to blatantly lie. They'll get caught later in the process. But you're assuming identity thieves are acting rationally or fear getting caught. They don't, and they're playing a numbers game. If you get past an interviewer and get fired after two weeks, your salary at a US tech company can be $5k, which is the average annual salary in many developing countries.
In the US, it is illegal per the EEOC, which has barely any resources to actually enforce the law: "Similarly, employers should not ask for a photograph of an applicant. If needed for identification purposes, a photograph may be obtained after an offer of employment is made and accepted." [1]
[1] https://www.eeoc.gov/prohibited-employment-policiespractices
Asking for a photo opens an employer to claims of discrimination, but it would still have to be proven, so the government is advising against it. But it may well be that in practice, the government will not be spending resources on trying to or be able to prove discrimination, even if a photo is required.
Don't pay for this cruft, sponsor local meetups instead.
A core lesson in ML: this is difficult or impossible. See the classic example with wolves, dogs, and snow. [1]
For a resume example, suppose you don't want to be biased again black candidates, or women. Well, good luck trying to filter out resume experience, skills, schools, and interests so that they don't list anything that is statistically more likely to be found among black folks or women. It's impossible.
So now you have a hiring process that looks at... what exactly?
[1] https://www.researchgate.net/figure/A-husky-on-the-left-is-c...
The whole point of white listing variables is to only allow things you are comfortable using todiscimrinate. And yes, that likely reduces the power of the model.
Edit: to clarify, your example had the model failing to classify something correctly because the training data did not generalize across real world examples. The problem with diversity bias is that the modes are trained on past data where the labels themselves tend to have some bias. Black candidates do worse. So recreatingthese will generally produce worse outcomes for minority candidates. Now for a lot of these cases the average minority individual is more likely to do worse. Not intrinsically because of race, but still worse.
You can’t realistically expect the model to produce equal outcomes for all classes when all classes are not equal on the actual traits. But you do want to try and make it so that the model doesn’t use race indicators as a crutch that ignores actual good indicators. The common complaint is you hear about a man and a woman applying with the exact same credentials but different outcomes. Well obviously the model had to have additional information besides those credentials if it made different decisions. If this causes problems, you probably should not permit those additional pieces of information.
The problem is that variables where you can safely discriminate don't exist in the real world because everything is intertwined.
For example, if you see that somebody went to school at {Howard, Bryn Mawr, BYU} you can make some pretty solid inferences about their {race, gender, religion} even if they don't explicitly provide them.
Howard is a historically black college, which means their student body today is majority black, not 100% black.
Bryn Mawr is a womens' school and admits only women. However, gender is not always fixed going through life.
BYU is a Christian school, but also admits several non-religious people too, and people regularly become apostates to their respective religion.
Technically true of course, but >99% of the population will identify with a single gender throughout their life. The correlation is so strong that it's not even really a correlation anymore.
That doesn't mean you aren't stereotyping people, though.
If the training data shows a history of bias against Howard graduates, the model will learn to replicate.
Job applications are kind of dumb. The common use case is credit decisions. Surely, for example, you’re ok looking at credit history and income when deciding to make a loan, right? Even though it won’t be equitable outcomes across races, it is still “fair”
Other goals can be to have equal model accuracy across subgroups (e.g., models has the same precision/auroc/etc for each subgroup, even if the predictions themselves differ across subgroups) or identical distribution of statistical biases across subgroups (e.g., the model makes the same types of mistakes within each subgroup)
Like, let’s look at that for credit decisions. Black folks have a lower income distribution. So it’s like saying the model is ok if it rejects a Black applicant making 80k if it would also reject a White candidate making 100k. That is the insane bullshit you’re effectively supporting by desiring comparable error distributions.
You have to accept that equal outcomes are not just impossible but not desirable. And then you only use fields that people agree are fair to use. If Black people get disproportionately better or worse outcomes it is either because those outcomes make sense given their stats or it’s not ok because if a White dude had the same info, that you agree is fair to use, they would have passed and the underlying reason for the difference is some bullshit other input.
That might mean that the model is a dumb: you must make $100k for this loan (illustrative) which is going to mean a lot of Black folks cannot get it. Well that model is at least “fair”. Accommodations for race should not be coming from an AI model
For example you can infer the race of a candidate from other facts about them. The model could still illegally discriminate by first inferring race, then discriminating on the basis of inferred race.
The problem you describe affects blacklists that try to remove only the things you obviously don’t want to use.
1. Gender bias
There is a widely used keyword-based gender bias measurement based on a study from 2011. The research claims that job ads with higher levels have more masculine words. This is based on a pre-defined list of masculine and feminine words [0][1]. Another Harvard study stated that women would only apply for jobs if they meet 100% of the requirements, while men would apply for 60% or less [2].
2. Racial bias
This might be more related to video interviews or based on name, age, or address in resumes. For resumes, there are tools already to anonymise resumes to ensure the focus is on experience instead of the person.
3. Disability
ADA (American Disability Act). According to this FAQ [3], "The ADA does not require employers to develop or maintain job descriptions. However, a written job description that is prepared before advertising or interviewing applicants for a job will be considered as evidence along with other relevant factors."
The list could go on, with age, sexual orientation, native speaker or not ... etc. That's why the expectations have to be clearly defined.
The law, as I understand it, will lock employers to their vendors as they will be required to keep data for auditing, which might incur more charges.
[0] https://gender-decoder.katmatfield.com/static/documents/Gauc...
[1] https://gender-decoder.katmatfield.com/about
[2] https://hbr.org/2014/08/why-women-dont-apply-for-jobs-unless...
[3] https://adata.org/faq/does-ada-require-employers-develop-wri....
https://www.govinfo.gov/content/pkg/USCODE-2010-title42/html...
https://www.ecfr.gov/current/title-29/subtitle-B/chapter-XIV...
The NY law uses the word ‘bias’ pretty much only in the phrase “bias audit”, which is defined in the law to mean a yearly audit of the software to make sure it is not discriminating as far as EEOC standards.
The point is that there isn’t a new law about bias, and it’s not an ambiguous or undefined concept. It’s the existing equal employment opportunity laws, and all employers already know about them, and most (I hope) already adhere to them. I’m not suggesting it’s either complete or perfect, I’m just pointing out what the new NY law is referring to when it uses the words “bias audit”.
I'm not sure what can be done with these cases, and I know people, of various protected out-groups, that have specifically mentioned those as reasons not to join a company.
This group will keep shrinking though.
My first job as a developer in 2006 I sat with the lead developer, had a very comfortable chat about things I had built, things I was interested in both related to programming and otherwise. Conversation turned to math and algorithms a little bit. He stepped out and came back with an offer. Whole thing took maybe 45 minutes.
I honestly feel as far as interviews have gone, it did the best of honestly sizing me up as a person.
Most companies don’t operate anywhere close to this scale and should not imitate this same hiring process.
We software engineers are used to be in high demand and somehow short supply, even now when it's taking me more than two months to find a new position. In many other industries supply noticeably exceeds demand, and people would go to great lengths and jump through really silly hoops just to get hired.
> * AI-based chatbots that ask candidates questions about their qualifications, then decide if they’ll proceed in the interview process
> * Algorithmic video platforms that have candidates answer interview questions on camera, record their replies, transcribe their responses, and analyze their vocal or facial patterns for subjective traits like “openness” or “conscientiousness”
> * Logic games that purport to identify qualities like “risk-taking” or “generosity”
Alright, which one of us monsters built any of this dystopian garbage?
(We certainly are monsters with a sense of humor: see the example of deploying Kafkaesque automated snake oil gatekeeping to people's livelihoods... to seek "conscientiousness".)
Buddy, it takes a village.
But I feel personally obligated to just note, for any HNers reading, that the two ex-McKinsey people I've worked with closely at a startup were both decent, and I'd work with them again.
Heh, maybe we can get McKinsey on our side, on this particular AI dystopia threat. Some hypothetical vendor-selection AI model (comparable to a hiring one) might take one look at certain past scandal involvements of the firm, and denylist anyone associated. But even organizations that don't care about individual fairness might still have to care about some AI pet project sabotaging the quality of their people and vendors.
I also think the argument that big companies wouldn't do it if didn't work is false. This is the sort of thing you're going to be able to create some big awesome presentation for, demonstrating that not only does it work, but it will end up saving you large amounts of money. Even if it does nothing of the sort. There's just so much room for number jukery, buzzwordery, and general shenaniganry - even if you never overtly lie.
Like most other things companies do in the hiring process (e.g. endless redundant interviewing rounds, ridiculous culture fit questions) -- or the stuff they do in their day-to-day work ("Agile") -- it doesn't need to actually "work" for it to be widely adopted.
It just needs to appear to work, and to give off a warm, buzzy feeling.
I would think that's one of the primary concerns here, is that we can pretty much guarantee that it doesn't work. Then end result would turn hiring processes into a kafka-esque hellscape where reason has no bearing and candidates are essentially selected at random.
Given how infuriating the current crop of simulated customer service agents are, I can't imagine bringing anything like this to job selection would be good for any of us, save for the leadership team who gets to tout their cost-saving plan and give themselves a bigger bonus.
Edit: ugh after reading the article I'm so disappointed that the regulations amount to some pretty weak auditing and transparency requirements. I was hoping this was an outright ban. It seems the laws have been rewritten by the cooperations. It gives candidate "right" to ask questions about their use and possibly opt out, but all that means is anyone who speaks out will be unhireable. I mean what company is going to hire anyone who starts asserting their rights up front?
That’s pretty much what it is today isn’t it. You get into a room with a hiring manager and if you happen to have kids the same age or saw the same movie growing up, you’re gonna have a huge leg up.
Can we blame someone for at least trying to make a system that’s more objective
I think the entire history of business backs you up on this.
That seems like harsh characterization. If all of the candidates are poor matches then perhaps the recruiting system isn't working properly.
I think this is inevitable, in one form or another. It will just be outsourced to some other companies which will rehash and extract whatever data, and sell services like the current background-checking companies, without even exposing any specific data to the hiring company. It's time to bit the bullet and remember that everything you say online publicly is going to remain in your permanent record, even with GDPR and stuff in place. It can't be countered by regulations, much like encryption can't be made transparent only to law enforcement by regulations. What once has been made public cannot be unmade public.
> AI-based chatbots that ask candidates questions about their qualifications, then decide if they’ll proceed
I suppose non-AI analysis of forms is not going anywhere.
> analyze their vocal or facial patterns for subjective traits like “openness” or “conscientiousness”
> Logic games that purport to identify qualities like “risk-taking” or “generosity”
I think it's just so stupid that I'd care more about my right to have disclosed the fact that a particular company does that.
On the other hand, this means that the state need to put its nose much deeper into the hiring process of a private company than it used to. It's the definition of public oversight, of course, but usually it applies to public offices.
So if 20 women and 100 men apply, it would be considered unfair to accept 10 women and 20 men (as 10/20 != 20/100).
[1] https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=4...
I would like to see a distribution of salaries for jobs in NYC before saying ~100k
I guess I'd have to generalize it from bio- to also apply to tech now.