I don't think you're phrasing your idea well. Perhaps you mean that any piece of software that allows surveillance has the propensity to trend toward being misused? It seems to me spyware is a loaded word and has its connotations. What has happened here has been on a massive scale very quickly in a way we have not seen before.
Edit: For example, remote sensors placed in a power plant or foundry where people work also would constitute surveillance. But it is in an environment where carefully calibrated machines can otherwise fail catastrophically.
I would say if it does not spy all the time it runs it can't be spyware - does Teamviewer spy all the time or can it just be made to spy as a side effect of its main purpose?
Teamviewer came to my mind as well but I'm not sure desktop monitoring is quite on the same level as a camera pointed at one's face. Unless teamviewer made some strides in the years I haven't used it..
To add, I suppose what I am really getting at is that it may be more useful to address ProctorU in particular rather than bring it under the same umbrella as other less used surveillance software.
I spent years running a school and am an edtech developer: education needs to (and probably will) evolve to suit the nature of remote learning. We're in this weird phase where we're trying to shoehorn models and constraints from the in-person learning paradigm into remote learning.
My wife has taught for 3 years at an online only state charter school (US). The single most difficult issue to solve (waste of time) is integration between foundation school management software such as Infinite Campus (where grades are kept) and third party learning packages (where assignments come from).
Funny you mention that. My work is building course content and assessment delivery systems that can be plugged into any LMS. It’s the leading (modular) solution to this exact problem.
I've seen it done successfully at some uni's. Zoom for the lecture, and a far less invasive proctoring tool for the tests. You can even have in person and remote in the same class if the material is made available online.
All of these testing suites are absolutely intrusions of privacy. Proctorio had a portion of their EULA where they essentially stated that they will retain all of your information (your test results, your webcam footage, your microphone recording, etc.) for an undisclosed amount of time, and if the Proctorio brand were to ever be purchased by another private entity, that footage would become their property by extension. Pretty unbelievable stuff, being forced to take a test in an environment like that would probably cause me to spiral out into a nervous breakdown after a few minutes.
Don’t read the EULA; just mindlessly click ‘Agree’ like everybody else. The corporations have already won, so there’s no need to give yourself anxiety over it.
> Surrender to the power of complex technological systems — allow them to oversee, track, quantify, guide, manipulate, grade, nudge, and surveil you — and the system will offer you back an appealing share in its spoils. What is good for the growth of the technological system is presented as also being good for the individual, and as proof of this, here is something new and shiny. Sure, that shiny new thing is keeping tabs on you (and feeding all of that information back to the larger technological system), but it also lets you do things you genuinely could not do before.....The danger, however, was that “once one opts for the system no further choice remains.”
My favorite moment when my institution was evaluating proctoring software was when a faculty member asked: "What if a student's naked underaged sibling/child walks into frame during the exam? What is your corporate policy on evaluation and retention of child pornography?"
Shockingly, smarmy ed-tech hucksters don't have a good answer to this one.
This sort of thing is why we sometimes need a platform regulator / "App Store".
Customers can't defend themselves against such intrusions of privacy from their school/government.
And employers too. They want you to respond to emails at all hours, and ask for way too much control over your device in exchange. This is slowly being changed with Android work profile and Apple "user enrollments", thankfully.
It needs to be a legal requirement with teeth: these tools would never be allowed through the app store approval process but that's not a problem as long as they're allowed to simply say you have to buy a laptop instead.
One alternate way to prevent this would be liability: if the institutions using this had to reimburse all of their users for every security hole in their mandatory software or the risk due to the security settings they require you to disable, it'd complete change the calculation for them.
I don't really understand your point. ProctorU has apps on both Play Store and App Store, how does Apple and Google save us here? Seems they have no problem hosting software for ProctorU.
App stores are actually an enabler for this sort of cancer; Apple is already speaking about reporting people to the police, they can obviously add an anti-cheat as well. And there will be no way escaping that.
I had the same issue with ProctorU. Installing Windows on a 64GB USB and booting off that anytime I had to take an exam solved the problem to my satisfaction.
I suppose I'm extra paranoid because I have a dedicated (older) computer for courses that require some sort of installed software, including Zoom. I don't want my unmounted hard drives available to the software.
KVM is probably your best bet on Linux and VMware the best on Windows.
https://github.com/hzqst/VmwareHardenedLoader works for VMware but doesn't work against some modern anti-cheats, but KVM universally works against anti-cheats when configured properly with RTDSC spoofing and such
It's harder than you think, and remember that the consequence is not “I can't play a game until I revert my config” but “I was reported to my college for an ethics violation and now my $$$ degree is in question” or “My professional organization has been told that I attempted to cheat and the certification I need to keep my job is in jeopardy”.
There are many things which are technically possible which are not a favorable cost-benefit for most people. This is in the same category as those guys who relied on technically being able to fly without showing ID to the TSA — there's a reason why it was mostly affluent white men flying solo, because the potential downsides are much greater for most other categories.
I opted out of using spyware in the university physics courses I taught last year, and caught my cheating students the old fashioned way. Proctor-spyware, like airport security, is more about theater than effectiveness. You aren't giving the USMLE or a Bar Exam, so you can take the time to write a good exam and evaluate it correctly.
I've never graded tests or papers, but I always assumed cheating would be obvious because if you go to chat with a student about the problem, they will not have anything to say.
Well, if you put up an online exam that consists of only basic multiple choice tests, then you're making it a lot easier to cheat. That's the source of a lot of the trouble.
That's true, but questions that are difficult to cheat on are both difficult (time-consuming) to create, and difficult (time-consuming) to grade. Which means less time for other stuff, like actually teaching.
So I only have experience creating/grading tests as a TA, not a prof, but that wasn't my experience at all.
The total time spent in creating a few good, easy to discuss questions, the answering of which would demonstrate understanding, and then reading them, thinking about them, and coming up with a grade, was probably actually less than the amount of time it took to create meaningful multiple choice questions that didn't have any ambiguity, and which weren't easy to intuit the answer even without understanding. Doubly so when we did away with "partial credit" answers, but instead made it so each question was 1 point (or otherwise all or nothing), you had, say, 5 of them, and what we really were looking for was a paragraph that showed understanding (rather than checking off boxes in a rubric of "mentioned A, B, and C"; short essay questions, if you will), an expectation we communicated to the students.
And that's aside from the actual project based grades, which were better still.
As a teacher you're pretty expected to know the problem you just asked, and there's no amount of body language which can replace an answer to the asked problem.
True, but that is not the same as proving someone cheated. Besides, when you have too many students, you cannot talk to all of them one-by-one in any meaningful way. Or assess them in a meaningful way, to be honest.
By the way, where I work, management pushes for stuff like proctoring, and more students, and "measurable" results, and so on. As a teacher, I don't care much about the whole grading show.
> “It’s become clear to me that algorithmic proctoring is a modern surveillance technology that reinforces white supremacy, sexism, ableism, and transphobia. The use of these tools is an invasion of students’ privacy and, often, a civil rights violation."
Must all of the cards in the deck be played at each turn? Things cannot simply be bad/user hostile/privacy invading, etc.
Yes. Since its A) incredibly easy to play all the cards and b) its pretty much impossible to attempt to refute without being labeled all of the above and c) gives the accuser woke points
Of course! Calling this out makes you a free-spirit and free-thinker. Definitely not brainwashed at all to be oppositional to that list of things that are obviously terrible.
You're anonymous online. You have the ability to create a throwaway and refute the actual arguments in the article all you want with literally 0 repercussion. Honestly seems more like you're chasing your own version of woke points.
> It’s not clear to me that algorithmic proctoring is a modern surveillance technology that reinforces white supremacy, sexism, ableism, and transphobia. The use of these tools is an invasion of students’ privacy and, often, a civil rights violation."
He's not anonymous, he's posting with his real name and has a KeyBase identity in his profile. Honestly it takes serious guts to call out Woke BS with proof of identity in your profile.
Don't you normally expect people to make the strongest case for preventing something? It's pretty common for a lawsuit to bring every claim a lawyer can come up with on the hopes that enough will stick to get the outcome they want.
I would especially consider that in the United States we do not have a broad legal right to privacy but there are potentially much stronger tools available if that software skews negative outcomes towards protected categories like sex, disability, or race. From the perspective of a student, job applicant, etc. being asked to use this, if the legal risks cause an organization to stop using it they'll enjoy that as a win even if the outcome isn't a blanket ban.
>Don't you normally expect people to make the strongest case for preventing something?
Not at the expense of grounded reasoning. When I see poorly substantiated claims, it shouldn't, but it really drags the whole rest of the argument down. The argument presented about lawsuits is actually a great example of why I think that's a broken system. They resolve that issue of lost credibility by considering each issue with total, clear, and mandated separation. Outside of that legal world with very well-defined rules, using such tactics reduces credibility.
I should note, that in this particular case, the claim of racial biases is at least substantiated by a believable anecdote.
EDIT: To clarify why I think the legal methodology is broken, its only because the same principles apply to criminal trials. IMO, prosecutors should NOT be throwing poorly substantiated charges at a defendant just to increase their winning probability and make the required defense more expensive.
You're talking about a letter from a U.S. Senator citing published reports in e.g. MIT Technology Review and concerns raised by professional organizations. I think dismissing that as “wild, unsubstantiated” would require at least some discussion of the linked claims.
Why should I reference the particulars of the link when refuting a generalization?
If the parent comment had said something to the effect of "I should give the benefit of the doubt to a sitting US senator" then you'd have a point, but that context wasn't part of their statement.
Edit: Also, frankly, I wouldn't give the benefit of the doubt to a US senator. If anything, it makes the identity politics feel even more irrelevant.
Okay, trying engaging intellectually with the reports rather than just reacting to someone’s one sentence summary of thousands of words. It’s kind of hard to see any definition of “identity politics” which includes the reports but not your emotional reaction to accurate words.
> Okay, trying engaging intellectually with the reports rather than just reacting to someone’s one sentence summary of thousands of words
I wasn't reacting to a summary, I was reacting to an independent premise. The sentence I reacted to, at least the way I read it, was broad to the point of being more of an axiom with which the commenter interpreted the post. As I see it, disagreeing with an axiom is an intellectual engagement. Feel free to counter my disagreement.
> It’s kind of hard to see any definition of “identity politics” which includes the reports but not your emotional reaction to accurate words.
Not entirely sure what you're saying here, but none of what I said was emotional. Simply pointing out that pattern matching is a viable way of filtering other people's thoughts and ideas. If someone makes wild claim, it should change the way you view other claims which may have seemed more rational in their absence. Not really an emotional statement in my mind, but again, feel free to point out which part of this you disagree with and I'd be happy to engage.
As it stands now you aren't really responding to anything I've said, but rather disagreeing that what I'm responding to warrants responding, which is rather tangential.
If you think the accusations are poorly substantiated, then make that case, rather than just complaining that too many accusations were included in the same paragraph.
No, but I think that raises the question of why you think the quoted bit above is a bad argument. It tracks with various reports I've seen linked in HN on this topic over the past couple years.
I know that 95% of the time I see a Gish gallop of identity politics, it's not an argument to engage in at all because even when you do, you're called racist yourself unless you subserviently agree with every aspect of the argument. The identity politics argument is often a tempting one because it allows people to act righteously indignant and feel powerful.
Case in point, here's someone saying, 'How can you even handwave away systemic racism?' in reply to a comment agreeing that bias exists but is not deliberate: https://news.ycombinator.com/item?id=29164295
Also, the identity politics argument seems to hinder a simple, moral argument against surveillance software as a violation of privacy. The logical implication of the IP argument is that this surveillance software would be okay to use if we manage to work out all the kinks.
The difference is that lawyers act in a structured environment with specific rules on how things should be considered - you bring up all the possible claims because if any of them get thrown out, it doesn't impact the others.
It's not the same with general discourse - when you raise a bunch of issues that aren't especially relevant and seem designed to be inflammatory, you damage the credibility of your other arguments. Arguing that test proctoring software is transphobic is such a stretch that it makes you question whether they author has such strong biases against the software that their evaluation of it is just generally too biased to be trustworthy.
> Arguing that test proctoring software is transphobic is such a stretch that it makes you question whether they author has such strong biases against the software that their evaluation of it is just generally too biased to be trustworthy.
It seems like the software is matching people against existing images, based on the issue with the black student, and trans people are I would assume more likely to change their appearance, including as a result of taking hormones and having facial feminization/masculinization surgeries.
I have found that two to three strong points far out weight a list of 5-10 weaker points. This extends to the case when the original two points are included in a longer list.
Things are bad in particular ways - software that is more likely to falsely punish people of color does support the continued dominance of white people in a real, material way. Noting how things are bad seems useful and important to me.
That being said, there's a rotating list of "badnesses" that are in the zeitgeist and I agree that it's annoying to see them flogged at every opportunity (often w/o much insight).
"software that is more likely to falsely punish people of color does support the continued dominance of white people in a real, material way."
The statement is at best extremely misleading, and at worse, mostly false. It also represents a juvenile, immature, and myopic perspective on reality.
I would suggest reading "Wealth, Poverty, and Politics" or "Discrimination and Disparities" by Thomas Sowell. He has been debunking the "inequality of outcome therefore racism" logical fallacy for decades.
It’s more likely to fail because…physics. The sensor on your webcam is only so big, and can only capture so much light. Darker faces require more lighting to capture details. Photography isn’t racist, it’s physical limitations that come into play.
A) The major issue is not the sensors, it's the lack of emphasis of ML training data for darker skin.
B) Even if it were simply physics at play, requiring the use of a system known to have physical constraints against darker skin causing failing grades purely on that metric is still pretty racist.
> Photography isn’t racist, it’s physical limitations that come into play.
I wanna pull this apart a bit because I think it's a good opportunity to talk about how systemic bias gets started. Digital sensor evolution is path-dependent. Technologists developed photo-sites that have "enough" dynamic range for most uses before moving on to increase the resolution on a sensor. What exactly is "enough" depends on your test data.
The sensor on a webcam is only "so big" as you say - but how that sensor balances resolution and photo-site count depends on what conditions they consider acceptable. We could build web cams that would see more pigmented faces better - there is no fundamental limitation in the technology itself. It's that a series of decisions have been made over years of development, generally without people thinking specifically about race at all, and we've arrived at a status quo that has adverse outcomes for people with different skin tones.
There was a similar process that happened with film photography[1]. Not that film, as a technology, is unable to capture dark skin - but that the development standards that were tested and distributed were designed for lighter skin.
Like, I agree that the webcams we have aren't intentionally 'racist.' But I do think that the status quo that has led everyone to accept this balance of dynamic range and resolution is reflective of valuing people with lighter skin more.
A closely related story that will also resonate with you: early microphones were tuned for male voices, which led to truly a lot of harm to women. It's just a matter of physics, as people are fond of saying. Mediocre technologists make mediocre products that were only validated for people like them.
Where did I say that unequal outcomes must, necessarily, be caused by racism? What is misleading about what I said?
It seems like you're reading a specific thing I said about a specific scenario and universalizing it in a way you imagine I might universalize it. I'd love to hear a critique of what I actually said, or we could talk about our views of society in a wider way, but I can't respond to this combination of generally dismissing what I said and attacking what you imagine I might think.
I stand at least partially corrected. I am not familiar with biases in facial recognition software but it looks like a real thing in some cases, caused for instance by lack of diversity in training data sets.
The quoted person expands on their arguments on those topics in the linked article. If you think they're needlessly "playing a card" why not engage with the actual argument?
It’s hard to make people care with “bad/user hostile/privacy invading” because those terms have saturated descriptions of behavior that users are okay with. Example: tons of articles mention FB as a privacy invading or user hostile service but it continues to be used by people who don’t really care. Using the same terminology for something that is arguably worse with much higher stakes (algorithmic proctoring that “reinforces white supremacy, sexism, ableism, and transphobia”) is appropriate because it gets the reader to care by illustrating exactly what is possible with algo proctoring.
I sense you’re tired of discussions that mention the “cards in the deck”, likely because you aren’t affected by them and therefore care little for them. That’s honestly fair, but there’s value in writing that way to channel outrage into action.
My criticism is that including a laundry list of "isms" is a polarizing, low-effort rhetorical device that elides a lot of nuance.
White supremacy: darker skin tones tend to photograph worse than lighter skin tones. Laptop webcams are notoriously crappy and can make this effect worse. Is this white supremacy or physics? Can the test instructions be modified to ask all users to have an appropriate lighting setup (ie lit from above and from the side to ensure that your face is foregrounded properly)
Ableism: for users who require assistive technologies, is it better to take a test in their own space with their own equipment or to travel to a test center and use shared lab equipment? For users with mobility challenges, is it better to be in their own space or travel to a potentially non-accessible testing center?
Sexism: for working mothers, better to take a test in your own home or travel to a testing center and arrange for child care?
Two "isms" seem more relevant but weren’t mentioned: ageism - because fuck boomers, right? Socioeconomics - not everyone has access to a PC that meets the specifications or can acquire one on short notice.
Lots of context is discarded when one engages in polarizing categorical rhetoric. I’m not here shilling for proctoring software but rather for nuanced discourse.
> White supremacy: darker skin tones tend to photograph worse than lighter skin tones. Laptop webcams are notoriously crappy and can make this effect worse. Is this white supremacy or physics?
I’m not a photography expert, but it seems to me that cameras are physically equally capable of overexposing an image and underexposing an image. If a particular camera which is used in a facial monitoring system tends to do one rather than the other, I would ask why that is the case.
Doesn't work like that. Overexposing (with the same light) means having worse SNR due to higher gain.
The choice to use typically terrible cameras in a proctoring system disregarding that it might work ever worse than normally for a subset of people is suspect, yes.
But wouldn't it be just as easy to create a cheap camera that tends to correctly expose dark skin in normal lighting conditions, and cannot dial down exposure enough to prevent light skin from being blown out in normal lighting conditions? If that's possible but cheap cameras tend to not work this way, then why is that not the case?
Yes and no, but mostly no. Lighter skin reflects more light (which means more signal), so it's inherently easier to image (it would be harder in extremely intense light, but getting fast shutter speeds is a lot easier than dealing with not having enough photons). Auto-exposure algorithms tend to work more accurately on lighter skin, too, which could be improved but is generally not something implemented at the hardware level in a webcam as far as I know (software can ask for different iso sensibility and shutter speeds).
The "isms" referenced are less about the fact you can work around these complaints line by line and more about the fact no one bothered to check them before rolling out required surveillance technology for education.
White supremacy is not just bad faith things done by bad people, it is also the assumption that whiteness is the default experience and the failure to account for that not being the case. Similarly with ableism and sexism.
That being said, complaining about the accessibility & inclusiveness of our required surveillance technology does have a dystopian feel to it, lol.
> White supremacy is not just bad faith things done by bad people, it is also the assumption that whiteness is the default experience and the failure to account for that not being the case.
This only works with "new" definitions of racism. It is in fact plainly racist on it's face to demonize a group based on immutable characteristics. It is even worse when actual diversity of though is ignored and people of color are demonized because they don't agree with a race-marxist ideology.
Would these webcams have shipped if you couldn't see white faces well on them? I don't think they would, and the test proctors would not think of requiring one that didn't. Both the camera makers and the proctors are putting in an assumption that these products are only/primarily for white people, and thus discouraging non-white people is not an issue.
The claim of "transphobia" is extremely weak, for one. Clicking through the maze of links, the argument appears to be that because algorithms that attempt to guess gender based on photos sometimes guess wrong when trans people are involved, all attempts by computers to look at faces must therefore be transphobic, even if those computers do not attempt to guess the gender of the person whose face they're looking at.
Misgendering people intentionally is transphobic, and systems of this nature behave this way. For many trans people getting an ID to correctly reflect their name and gender is extremely difficult and can go for months/years without it being corrected; companies like Proctor U don't care about the problems this causes: https://www.teenvogue.com/story/exam-surveillance-tools-remo...
I followed the references to find the actual argument behind calling surveillance proctoring all of the above, and these are the relevant bits[0]:
>At the beginning of a test, these products ask students to verify their identity by matching their appearance with a photo ID. As Os Keyes has demonstrated, facial recognition has a terrible history with gender[x]. This means that a software asking students to verify their identity is compromising for students who identify as trans, non-binary, or express their gender in ways counter to cis/heteronormativity. If a student’s gender expression or name on their ID are different from their current gender expression or name, the algorithm may flag them as suspicious. When this happens, they may have to undergo another level of scrutiny to authenticate their identity, an already common and traumatic experience for trans and gender non-conforming students. If these students are not alerted of this possibility before the test begins, it may force them to either discontinue the test and risk their grade, or out themselves to their course owner when they may not want to, risking more trauma and discrimination including being denied financial aid, being forced to leave their institution, or have their lives put in physical danger.
>The Eugenic Gaze is a combination of white supremacy, sexism, ableism, cis/heteronormativity, and xenophobia. When we apply the Eugenic Gaze using technology, the way we do with algorithmic test proctoring, we’re able to codify and reinforce all of those oppressive systems while avoiding equity-based critiques because of our belief in the neutrality of data and technology.
Their recommendations are quite reasonable:
>Don’t use algorithmic test proctoring. Instead, focus on pedagogical techniques that you can use to design assessments, online or in person, that draw from personal experience or require students to apply concepts in unique contexts. If you have to use algorithmic test proctoring, make sure students know about the test settings and ID requirement well before they take a test, and assure them that you will not take any behavior flagged as “suspicious” into consideration that isn’t described explicitly in the syllabus.
The GP link[1] instead calls out "Facial Recognition Tech", and "Algorithmic Proctoring" as being too biased and follows up with a petition[2] to ban these entirely.
It's not news that facial recognition is best at white male-presenting faces and bad at all the others. ProctorU is also pretty hard to use for the differently abled.
I think you just reflexively dismissed this because you saw a basket of words that normally go along with things you disagree with, but the argument is pretty solid.
> It's not news that facial recognition is best at white male-presenting faces
Really? On the male part? I'd have expected it to do best with women, because I'd have figured facial hair is more difficult to deal with than a wider varieties of hair styles.
It's important to note that none of these things are necessarily done deliberately (though "white supremacy" is perhaps a bad way to express "racism that benefits white people specifically"). Other than transphobia, either the linked article, or the letter linked in that article provide evidence for all of the accusations. Facial recognition software that doesn't handle dark skin well intrinsically treats different ethnicities differently, in this case disadvantaging non-white people. Many of the markers for "suspicious behaviour" that are used to detect cheating are also present in people with both mental and physical health conditions. Dealing poorly with headware has an outsized influence for non-white women (who are likeliest to wear headwear that obstructs the face).
Again, I wouldn't chalk any of this up to deliberate bias against any of these groups, but it's all bias anonetheless.
> (though "white supremacy" is perhaps a bad way to express "racism that benefits white people specifically")
Can you explain how racism that benefits white people specifically, as enabled through either direct, explicit bias in policies or laws, or indirect, implicit bias through a lack of diversity when conducting research on human computer interactions, or training corpus used for machine learning is not explicitly white supremacy?
These types of issues have been mainstream and well documented that they are at the centre of a 2009 episode of a sitcom (Better Off Ted, Racial Sensitivity), among many other more conventional studies of the phenomenon.
After decades of research (and centuries of practical observation) how is it even possible to handwave away systemic racism and bias "that benefits white people specifically" as anything other that white supremacy?
Edit: I accidentally left out the italicized part of the first question.
I think "white supremacy" tends to imply direct, explicit bias, and may sort of exclude the built-in "unrealized" biases that exist in the current culture, where white supremacy was the foundation but not necessarily explicitly imbued.
White supremacy is the idea that white people are inherently better and more capable and therefore more deserving.
Hence, for something to be white supremacy, it doesn't just need to have a bias favoring white people. It should also have a justification of this favoring based on white people deserving better or being better.
Racist systems that have encoded societies biases are generally not white supremacist. The 'justification' for those systems is often things like "this is just the way it is" or "this was easier to do like this" or "I went with my own experience".
These days a lot of racism does not come from white supremacy. It either comes from something like familiarity bias of people in power, or from following the status quo mindlessly. Calling those acts white supremacist can be dangerous. It allows the real white supremacists to hide among the unknowing. It also pushes people who unintentionally did something racist way into the defensive if you tell them they are white supremacist. And pushing people who unintentionally did racist things into defending their actions is not going to make things better.
> White supremacy is the idea that white people are inherently better and more capable and therefore more deserving.
That's a common dictionary or encyclopedia definition of the white supremacy. More broadly, white supremacy also refers to the systems and structures of power that are built into most of "western" (a better term might be post-colonial) societies that favor both white people, and people who support or uphold the balance of power in those post-colonial societies.
> Racist systems that have encoded societies biases are generally not white supremacist
I agree, however I think that those racist systems that are not inherently white supremacist in nature are largely rooted in non-colonial countries (basically countries other than the European colonial powers, and the countries that grew out of those colonies).
> It also pushes people who unintentionally did something racist way into the defensive if you tell them they are white supremacist. And pushing people who unintentionally did racist things into defending their actions is not going to make things better.
That is just not true. If someone does or says something racist, they can and should be challenged on it. If they become defensive, there are multiple reasons that could happen, but if the reason is that they simply didn't know better, it's just the way their society is, or if the reason is that they are opposed to "wokeness" (which is a catch-all for intersectionality, critical race theory, and many other modern perspectives and ideologies that are largely centered on dismantling power structures and reducing bias and discrimination), then it's likely that they are supporting white supremacy out of ignorance (whether that ignorance is from being uninformed or uneducated, or the more malicious willful ignorance of people who choose to use or engage in racist norms because they are opposed to "wokeness" from an ideological or other perspective).
Pushing people who do things that could be cast as unintentionally racist is the only way to a) educate them, so they can do better, or b) determine if it was an intentional act. I know this from practical experience, and it was only from going through the hard and painful experience of being called out on harmful "unintentionally" racist jokes and behaviour that I learned to do better after being raised by a family that had (and for the most part, still has) some pretty racist and discriminatory views.
Lets not devolve into a semantic discussion about white supremacy. Under your definition I agree with a lot of what you are saying.
I still think we should not tell people who do racist things "you are being a white supremacist". Instead we should say "that thing you did was a bit racist" or even "that thing you did could be seen as racist". We should _definitely_ challenge those people. But if you are trying to get someone to change their mind or behavior, you gotta be real careful about their feelings.
Calling someone a white supremacist is going to hurt their feelings quite a bit. And that isn't going to make them more likely to consider if they should change or did something wrong. That doesn't mean don't challenge people on their actions. That means challenge people on their actions very gently.
If we don't treat these people gently, then we lose the less introspective part of the population. They start getting reactionary, start 'banding together' against these annoying people. They start 'fighting back in the culture war'. Basically, getting this wrong is how you create republicans.
All of the above means I think we should be considerate in how we challenge racists. And I think part of that involves being really careful about the term white-supremacy.
There's an old saying: "It is difficult to get a man to understand something when his salary depends upon his not understanding it" (often attributed to Upton Sinclair, not sure if that's true or not tho).
Id like to suggest that what you are calling "familiarity bias" might have a component of the quote in it too. Not salary in this case, but social position. That is in the racist system, one race of folks get better treatment, and if they want to maintain better treatment, the status quo must be maintained. The group of people at top of a racial hierarchy (that is in the supreme position), are incentivized to keep the racist system. When race is considered a bad reason to judge a person, they still are incentivized to maintain the system, just find different words to justify the status quo.
I guess a different way of saying this is - white supremacy describes a race based social hierarchy where white people are at the highest level. It has also been used to describe the lowlife Nazi or KKK wannabes that advocate for it in the baldest terms, but they are bigots who advocate for white supremacy using racist terms like "inferior genetics" or worse.
Compare the term racist itself - there are folks who would have you believe that the term is limited to personal bigotry against people of a different race, and has nothing to do with the rules and actions of systems (a position I think you don't hold due to your description of racist systems).
I'm not handwaving anything away. It's just that I don't think all racism is the same.
White supremacy, to me, implies explicit, militant, proactive racism. People who might very well be proud of the fact they're racists.
This story is about a software system that (among many other issues) doesn't work well with darker skin tones in low light. Especially in light of all the other failure modes, I'd ascribe that to carelessness or indiference, mixed with pressure to reduce false negatives at the expense of more false positives. I wouldn't be surprised if training data and/or testing were filmed in an office setting with the amount of light you expect there, and they never ran across the issues with dark skin interacting poorly with the amount of lighting a student would have a home.
> White supremacy, to me, implies explicit, militant, proactive racism. People who might very well be proud of the fact they're racists.
That is only half of the story of white supremacy though. The other half is the entrenched systems and biases baked into those systems that largely benefit white people, and that train people, through experience, to prioritize preserving the existing systems and status quo.
Not considering the fact that there is a well documented history over the last 20 years of tech companies and business in general prioritizing the experiences of the white majority, at the expense of people of colour, is largely the reason why you can "wouldn't be surprised if training data and/or testing were filmed in an office setting with the amount of light you expect there, and they never ran across the issues with dark skin interacting poorly with the amount of lighting a student would have a home.", and not consider that being the norm, or even acceptable as being indicative of white supremacy.
Those biases may not always, and only impact people of colour, but they do overwhelmingly benefit white people. That's the entire point of the article that OP shared, and the references the author of that post uses to back their claims.
It seems clear that a dev team could whip up a product that tested well (using folks in office, family members, friends, etc.); was trained with datasets that - for whatever reasons - weren't sufficiently varied; and hit some mark of success and pushed it out the door to refine the rest later. It also seems clear that the resulting product could do poorly when recognizing black skin, due not to ill intent but lack of polish with the resources on hand.
But something I always wonder when accusations like "white supremacy" are thrown around: is it falsifiable? What evidence would dissuade you from that?
- What if both ends of the spectrum do poorly and extremely pale people have problems, too?
- What if the threshold is dark black and lighter-skinned black people, Asians, Middle Easterners and other non-white people are able to use it successfully?
- What if only a narrow band of light levels work, making it clear their testing range was generally too narrow, not just in skin color?
- What if they took care to incorporate black models in testing, but the photo quality (and their own in-house cameras and lighting) overestimated the quality of most home users'?
And what of the myriad other things that were done poorly in the software: limited OS support, bugs, excessive memory usage, overall intrusiveness, browser limitations, disallowed mobile devices, lack of multi-monitor support? Do they likewise arise from systematic oppression of some group? What if we dig in and find that white people are more likely to use iPads, Linux, and multiple displays?
Most often these accusations flow in only one direction, and that all other flaws or problems are taken to be simply happenstance and noise. Certainly anything that impacts white people negatively will not be automatically seen as anti-white, although in a world with activist devs, such a result isn't incomprehensible.
Claims of white supremacy (among other accusations of character) are thus, to my mind, wildly speculative and carry a very heavy burden of proof.
> how is it even possible to handwave away systemic racism and bias "that benefits white people specifically" as anything other that white supremacy?
Your way of grouping people by race is kind of arbitrary. It puts together rich and poor when they only share a skin color. How is white supremacy working in Bulgaria, for example?
This is speculation, but many transgendered folks tend to present gender in an non-traditional way, as well as people in the midst of transitioning that may be "in between" presentations in a traditional sense. If you only train on cisgendered faces, you may only be training on sort of "default" gender presentations, facial structure, and the like, which may give a similar disadvantage.
I can totally 100% see how transgender people might trip up some of these things. For the point of this discussion, though, all the other biases had a specific example in the linked article, whereas that one didn't.
I mean, it's an article about how a technology is negatively impacting marginalized groups, and cites research that backs those claims. It makes sense to cite the groups and practices most impacted by it.
You could read the research and refute it, or you could just bluster about things. I know you tried to expand on it in your comment below, but minimizing the specific concerns raised to "isms" and ignoring that at least two of the references in the articles linked and their references for the actual research addressed at least the socioeconomic portion of it, illustrates that you only applied your surface level perspective and criticism.
> Things cannot simply be bad/user hostile/privacy invading, etc.
In this case, the algorithm is actually bad for all ethnicities [0]. It's just that it's extremely bad for black students (fails half the time) and just regular bad for everyone else (fails a quarter of the time).
> …reinforces white supremacy, sexism, ableism, and transphobia.
Article proceeds to use a lot of words to not show any of these being true.
I am against it for the general dystopian surveillance normalization it encourages. We don’t need to throw a word salad of made up progressive insults against it to resist its implementation.
I’ve never seen a critique of proctoring software that offers an alternate solution to the problem (remotely delivering a skills assessment that is used to allocate resources in an environment known to have rampant cheating)
There's a discussion upthread right now about making the whole problem irrelevant by switching to open book tests and/or having a 5-minute conversation with the student to make sure that they actually have some clue what they're talking about.
Have students sit for an exam and then follow up with a question or two (from a pool of N), live, to see if they actually understand the material. We're paying hundreds or thousands each, they can easily afford to spend the time.
Best way is do away with exams and have projects graded instead. Math write a survey paper on the subject that demonstrates expertise, then orally defend it on live chat where you can't easily cheat, that's what my school did but only had european accreditation whereas regional US likes 20th century examination style.
And the ones that do propose alternatives do so from a very narrow point of view. Oral exams are much more time consuming and simply not practical in many situations. Projects and seminar papers have similar problems, especially in early semesters.
Create an exercise that contains a technical term that does not exist, and create a page containg it with a plausible, but wrong solution for the excercise. Make sure that the page is easily found with Google. Give everyone who solved it using the wrong solution a failing mark.
Personalize every exam. Create a pool of exercises and choose n exercises per student, based on the student id.
Easily done if the sheets are already LaTeX anyway.
Create heavy time pressure. Cheating is very hard when even completing all exercises regularly is almost impossible.
(Lovingly called "Zeitklausur" in German, lit. "time exam", it's normal that students are unable to finish those in time)
Create exams that don't just test the ability to vomit knowledge, but test the ability to use that knowledge, and let students explain in their own words.
Replace the exam with multiple small projects and presentations.
All of those things were used by different chairs / departments in my university :-)
Nothing will prevent "someone else writes the exam for another student" with absolute certainty, yes. But neither does proctoring software.
How do enroll in a Msc program and not read how exams are done? I took remote university for a while, you either go to some exam invigilator and pay the $20 for them to monitor you (some campus libraries it's free) or you have someone volunteer to do it who meets the criteria. Big deal this article is a twitter quality rant.
The problem is of course regional accreditation rules of proctored exams
This looks like something the UK Gov could use for SELT exams, visa & immigration tests or other similar secure tests approved by the government.
At the moment these are taken in person in secure buildings where you are identified at the entrance, your belongings are stored in a locker, and the exam itself is taken in a secure room on computers with no/limited internet connection, etc. You are timed and monitored.
I'm waiting for the world to catch up with the fact that looking up information while you are working is a core part of any real job. Do I care if my Linux Security Professional spends a few minutes looking up information on the internet before taking some action? It's not the case that anyone can solve any problem so long as they have a search engine. Without domain knowledge, an open ended web search is not going to lead to a convincing answer except for the most trivial questions.
This extends to coding interviews as well. Using the resources at one's disposal to get a sense of the landscape before diving into algorithms must surely be part of the job, right? What do I care if a developer needs a quick reminder before diving into a solution, or even reads up a bit and scans someone else's code before answering?
What is the value in ensuring that people have perfect recall if this is something that will almost never be necessary in a real world job?
It's always seemed a little off. I've been coding for 15 or so years and I still sometimes completely blank on certain javascript array functions and need to google it.
In consulting I produce better thought-out and constructed recommendations if I parse a book and previously-delivered decks of slides. Hell, even ISO standards.
On the other hand, at least a basic level of recollection is necessary for quick thinking in meetings, you don't always have the time to look up documentation.
And you can perfectly evaluate that by setting a realistic (!) time limit and judging the quality of the answers. It's absolutely irrelevant how much recollection you have if you still manage to solve the problem efficiently.
I took an astronomy course a few years ago which had a fairly forward-thinking (if a bit lazy) instructor. The initial tests were fairly conventional. But for the intense test right before the final, he gave us a comprehensive take-home. With the full assumption we'd be hitting Google hard for the more difficult questions. He knew this was a complex topic. Thought we'd learn more and retain more with a test where we had to show some initiative in finding the right answers without the stress of having to remember it on the spot.
Plus, it doubled as a study guide for the actual final which was only a couple of weeks later. I thought it was a remarkably kind thing to do. Took out a little stress. Gave even the struggling students an easy "A". And it worked as a comprehensive guide to almost everything we covered.
You are not thinking about the real risk. It is not about preventing a candidate to Google a few things on the side. It is to prevent a completely different person from doing the exam instead of the candidate and simply sending them the answers. And don't think it is just an abstract threat, there are whole businesses built around that. Unfortunately there is not much you can do to have exams remotely and be sure the candidate is the one doing the exam without being extremely invasive.
I bombed once an interview at a major bank when the hiring manager insisted what I would do if something happened to the system, and there would be no internet to search for answers. When I answered that nobody would notice the system was crashed if there was no internet didn't please him very much.
Ironically enough, you might not have internet access at your datacenter (reception issues, so no wifi or phone data, and switchport connections are often secured or don't route to a public internet). And things get really entertaining when your whole office network is down.
It's not an odd question. "Okay, the whole subnet where your credentials server used to be is now a smoking hole in the ground, and IT forgot to pay the fiber bill last month. What do you wish you'd done three years earlier to address this problem?"
I invite candidates to "error out" to internet resources and use their own professional judgement about what is and is not okay.
To set people at ease I tell them up front that only one candidate has crossed the line (googled the solution) and everyone else has made perfectly appropriate choices; "elseif or elif?" and small details like that.
They also lie about supporting Firefox. A family member needed to use this for a professional license. Following the instructions (which is basically turning off most security warnings and installing a bunch of malware) didn't work and the first thing support said was to install Chrome.
Shockingly, this was due to some JavaScript relying on an older Chrome proprietary API so there's no possible way they actually tested it against their alleged support matrix.
Sure, got a few million dollars to bring a lawsuit knowing that if it starts go somewhere they'll issue a 3-line patch and blame the intern for not testing it?
I know it's just a saying, but I feel the need to de-mistify this.
Lawsuits don't cost millions. Court fees are absolutely never that high, and lawyers, while some may be expensive, are generally affordable for ~middle class (or even lower class if someone wants to do pro-bono work for you)
The whole "lawsuits cost millions" thing is a myth perpetuated by big corporations and further relied by normal folk who hear it from somewher else, which probably heard it from somewhere else, and so on.
When you read in the news "X company wasted $XX million in legal fees", what it actually means is "they stretched out the case with a team of very expensive corporate lawyers whoses price ranges are in the millions".
> The whole "lawsuits cost millions" thing is a myth
That may be, but they can easily cost many tens or hundreds of thousands of dollars. Lawyers typically bill at multiple hundreds of dollars an hour so it doesn't take a lot of hours to rack up five- or six-figure costs. That's high-stakes poker for most people.
I once sued a neighbor for their barking dog. It cost me over $10,000 before I pulled the plug.
I don't understand how you think this is solely one-sided.
> "they stretched out the case with a team of very expensive corporate lawyers whoses price ranges are in the millions".
Yes - The company stretched the case out with expensive lawyers: Do you think the other side is somehow not obligated to also continue dealing with that case?
Who pays my lawyer while the company stretches the case out? Oops - that's still me.
----
As someone who has actually retained a lawyer for dealing with a previous employer:
1 - Most places had zero interest if the money at play was less than 100k (ie: They would not take the case unless I had a potential win of 100k or more)
2 - They charge ~$350 an hour. Sometimes billing for "intern" work at ~$150 an hour instead. I make good money (~200k) and I can afford less than 23 days of lawyer time a year, assuming I spend my ENTIRE yearly income on it.
You're right that lawsuits don't always cost millions. However, they will cost at minimum tens of thousands of dollars. Filing fees are generally a few hundred dollars per document, and median lawyers' fees are somewhere around $300/hr, depending on jurisdiction. And--in the US--it is generally expected that you pay your lawyer's fees whether you won or lost.
The advice I have gotten from actual lawyers is that it's literally not worth it if you expect to get only a few thousand dollars.
This is why we need robot laywers that can Sue-as-a-Service for $5/hr. Just log into the website, type in who you want to sue, why you want to sue them, and it should take care of the rest. With enough proceedings from past cases it should be possible to train an algorithm to create the defense that is most likely to win.
Unfortunately, that doesn't really work. While there's a lot of court documents that are going to be highly formulaic and could plausibly be written almost Mad Libs style, there are several court documents that are going to rely very heavily on the unique factual nature of the case. Responses and replies to motions are going to fall into that latter category almost universally.
Okay, yes, hopefully that's hyperbole but it's still a LOT more than most people are going to want to spend — that's why this works: if they were trying to take your house, sure, you'd lawyer up but when it's more like a principled stand on privacy, an awful lot of people are going to reasonably conclude that it's not worth the cost. This is the advantage to having, say, a government privacy regulator which has lawyers on staff whose entire job is to do things like this.
This is especially work considering with this particular company, which has a history of using legal threats to silence critics:
I once did a security review for a site that claimed to only support Chrome. I tried Firefox and used a UA switcher to fake being Chrome, and sure enough, the site didn't work. The page would load, but nothing would interact.
Turns out, their JS minifier was creating code that contained a syntax error. Chrome was able to make it work, but Firefox would silently error out. Rather than try to solve the problem, they blocked any browser that wasn't Chrome.
Me, to <litigious tech company>: “Your JavaScript fails on IE8 because it now throws an exception when it attempts to set an invalid CSS value. I made a tiny patch but do you have an ETA for the fix?”
LTC support: “We don't beta test Microsoft's products for them!”
Me: “Okay, it was released this week. How's testing going?”
[a week passes]
LTC support manager: “Hey, can we get a copy of that patch to give to other customers?”
My employer at the time paid 7 figures annually for support.
At least Proctorio (despite suing a college student under DMCA for reversing their software to show the extent of its capabilities [1]) doesn't go to this far a level. It's a browser extension that I can install for an exam, and remove afterward. It gets microphone, screen, and camera inputs, and permissions are handled through the browser.
They also are terrible at how they treat your local machine. I had to use them for a course I was in. I was having issues connecting because they kept insisting I used some Flash based tool. Their proctor started going into various settings on my Mac, clicking rather carelessly and what seemed like at random then declared the platform was unsupported. I immediately said "ok, so now your going to return my machine back to the state that it was in before you started messing around..." They immediately dodged the question and ended the chat. It took 3 more attempts to even get the test started and like a week to find all the damage they did to my settings. Terrible company.
For the certification program at Redis University (https://university.redis.com/), we previously used ProctorU because that's what other folks were doing -- proctored exams for certification.
Buuut, after a while, we were like, why subject people to this? This is crazy. And why even charge for certs anyway? I'm happy we're done with proctoring!
I'm another reluctant ProctorU user, and using it thrust upon me if I wanted to complete my online master in CS. It's spyware, and it really is that awful. At least for my tests, it's required that you buy an external camera, and scan the entire room before taking the test, it records you the entire time, and runs in the background (and foreground) with system level privileges. Taking a test this way is very stressful this way, compared to just walking into a building with just a pen and your phone on silent.
Nearly every college student during the pandemic had to use ProctorU in order to complete their classes, or a similar alternative. Quite disturbing the experience is normalized, and I wish there were an official OS level feature for "report all activity on the system from time X to Y", without having to use a sketchy third party app.
I wish the author the best of luck fighting the requirement to use ProctorU.
The thing is, it still wouldn't be enough. Ex. I currently use magisk to tell Google Play and all apps that my phone isn't rooted, allowing me to use app features they would otherwise lock me out of.
As a workaround, someone could buy a new laptop from a major retailer right before the exam, install the software, take the test, uninstall the software, then return the laptop for a refund.
This is my first comment on this site after lurking for a few years, I thought I would add my personal experience using ProctorU and how weird it feels to willing give up my personal privacy to take a test online.
I go to UOPeople, which is a tuition free online school and I am getting a 4 year computer science degree (for like $5k which is crazy), anyway of the 40 courses you have to take only 11 are proctored. UOP offers 2 choices for proctoring, A) find a real life proctor, or B) use ProctorU. I don’t have the luxury of finding a real person + with covid its more unlikely, so ProctorU it is.
All in all its not the worst thing in the world, but when I first read the requirements for the ProctorU testing environment and technical requirements I almost quit school completely to look for alternative paths. Some of the crazy requirements include:
Testing space must have nothing on the walls, or floors.
You can’t wear glasses while taking the exam.
Your desk must be clear of everything besides the specific testing materials (calculator if your lucky, and maybe a pencil and paper)
Your device needs a webcam, so that they can not only watch you for the entire 1 hour and 30 minutes where you take a test that determines 40% of your grade, but also so that you can show them each wall, floor and under your desk. And thats just the physical space requirements. I had to empty my closet and use my laptop to take this test because their is no way I just have an extra room for testing…
The digital requirements were pretty intense as well, access to folders they had to right too, chrome settings and a whole bunch of wack stuff. I created a dummy account just to take tests.
When you go into the program they have you download it acts like a 1 way mirror, you can hear the proctor(if you are lucky to have a human proctor) and they can watch you, your screen and hear you. I had some tech issues once and I was grateful to have a proctor with a sense of humor who was able to help me through it. I can’t say my privacy is worth a cheaper degree, but I hope that this doesn’t become normalized, because it is not a pleasant experience.
> All in all its not the worst thing in the world, but when I first read the requirements for the ProctorU testing environment and technical requirements I almost quit school completely to look for alternative paths. Some of the crazy requirements include: Testing space must have nothing on the walls, or floors. You can’t wear glasses while taking the exam. Your desk must be clear of everything besides the specific testing materials (calculator if your lucky, and maybe a pencil and paper) Your device needs a webcam, so that they can not only watch you for the entire 1 hour and 30 minutes where you take a test that determines 40% of your grade, but also so that you can show them each wall, floor and under your desk. And thats just the physical space requirements. I had to empty my closet and use my laptop to take this test because their is no way I just have an extra room for testing…
Talk about security theatrics ... I'd just stick the answers I want on a piece of paper just under the cupboard in reach of your feet or with a tape on the bottom of your desk and get it off when scratching your crotch.
> access to folders they had to right too
Why? They are recording all processes and the screen already. Again just theatrics. And if you can hide it from that they are never going to find it anyway.
How is this not massively illegal? This is a clear ADA violation. I cannot see without glasses. Not like, things are a bit blurry, but like I have 20/800 vision that's correctable to 20/20 with glasses. Forcing me to take an exam without glasses is forcing me to fail an exam for a reason that has nothing to do with my academic abilities.
I’d say it’s because glasses can hide your eyes when the lenses can reflect light which means their “are they looking in a suspicious direction?” crap can’t work.
I know you were joking about the naked thing, but another commenter above mentioned their girlfriend was told wearing a sleeveless top was ‘inappropriate’ and was asked to cover up. I just think that’s insane.
I also had an issue with not being able to wear glasses. It was my fourth Sans cert and never had the issue before. The proctor also was sabotaging me by saying I requested "Technical support" 5 times during the exam, each time the timer running and some dude distracting me, despite me telling him I have no tech issues and to let me continue my exam. They would spend 5 minutes verifying that I indeed had no issues and then leave...
Very unprofessional, if not illegal due to discrimination and even though it was one of Sans's entry level certs, I barely passed, versus 90 + on all of their advanced ones without these issues.
Their reasoning was "The rules say no facial obstructions, your glasses block your face.". They have to hire the dumbest people to do these proctors.
I've never had any issues with other proctoring services, and things like pearson for Comptia and Microsoft were actually enjoyable. With proctoru each proctor seems to find some issue and you have to argue with them since it's completely unreasonable.
If it's anything like my experience it's luck of the draw on the proctor and their "interpretation" of things. I used the same room and setup for every exam with them. Second to last one, the proctor said my room was 100% unacceptable. I protested and was told that there was no way this very same room ever was considered acceptable. So I moved into a new space and finally moved on to the test. My last exam with them? Used the old room and had no issues.
The proctors are random, poorly trained, people being paid near minimum wage in the American South that probably turnover every few weeks. I can't expect a high degree of judgment.
Most of these proctoring software easily detect Virtualbox, VMware, etc.
But QEMU/KVM which is the de-facto hypervisor on Linux is harder to detect. Even the others which I mentioned before can be hardened to evade detection.
And if you do a little bit of tinkering and intercept traffic, you can make it so that all the cheating reports from the "AI" never leaves you computer. I've never played with ProctorU but have experimented with a couple of other similar software. They usually send regular reports every five minutes and some anomaly reports (some extra software running on your computer, another person in room, face not visible, etc) when something happens. You need to intercept and modify traffic to not send these anomaly reports. This is easier if its browser based, but you need to install systemwide certs if its install-able software, and a lot more work if they utilize certificate pinning inside binary install-able software. I have never encountered the last one though.
When I had to deal with Proctor-U, the software refused to run under a KVM VM. Detection of anything remotely VM hardware related, made it alert and the proctor refuse to continue.
That's after fighting with the software to have it installed in the VM to begin with.
Most virtual machine detection boils down to checking the CPUID hypervisor bit and vendor string. Luckily, it is possible to configure VMWare, VirtualBox and QEMU to spoof those values in the guest machine.
One big problem with QEMU is that its virtual hard drives has the word "QEMU" hardcoded in their name, which proctoring software easily detects. When I checked there was a patch 1 year ago to make that configurable, but it was unmerged.
Reminds me that I still want to develop that camera driver that replays fake footage. Maybe I will build a small dongle with some storage for video.
I heard that MS is requiring notebooks to have a HD front facing camera. Maybe they would still sign my device driver? If they don't, wouldn't that be a lawsuit waiting to happen?
if this is paid for and this requirements were only disclosed after the fact, there might be enough money to be had here to entice a lawyer into making the course provider abandon the software.
That's more or less the only silver lining I can think of here.
Stop testing people on memorization and then you don't have to worry about cheating. Allow people to recall data with resources typically available to them in the real world. This is the same issue I have with code/interview challenges that say don't use the internet. I would be a fool to not use the resources readily available to me or at least validate what I think I already know.
nearly every exam I had at University, as a Computer Engineer, allowed 1 sheet of notes. So they were already not focusing on memorization even 20 years ago.
All of my B.Sc. Applied Physics exams including the finals were open note (anything in your own hand plus any duplicated sheets handed out in lectures). My finals were in 1977, Exeter Uni.
There was no limit on how many notes one brought in to an exam. Some of the weaker students turned up with rucksacks full of ring binders and the invigilators had frequently to admonish them to make less noise rustling the papers! Those students almost all failed or attained only a pass degree.
In my opinion this successfully weeded out those who thought that memorisation was enough. The exams typically never asked anything that could be answered simply by looking up the answer in notes or even the textbook.
I just finished up my masters in CS from OMSCS at GT and some of the classes allow a sheet of notes. It's still a thing, even in the days of online education and proctor software.
I graduated that program a few years ago. The best final I had was Intro to HPC, you were allowed to use book, notes, internet, etc. The questions were open-ended and in-depth enough that the average on the test was still around 60-70 IIRC. You need a very deep understanding of the material to answer the questions sufficiently.
Yes, the sheet of notes means you're not memorizing formulae, but I'm 95% sure the real motivator for professors to allow them is you learn when you put the notes together.
I think this common argument underrates the extent to which core factual mastery informs your ability to perform analysis and to synthesize arguments. For example, if given the exam question “Discuss the role of demagoguery in Athenian democratic politics in the Peloponnesian War.” and you need to look up whether the Sicilian Expedition happened before or after the death of Pericles, then you probably don’t really understand the role of Athenian political dysfunction during the war either.
I have to look up all of those things, which means I didn't gain anything in class/assignments/reading, and no amount of Googling is going to help me if the person reading my answer is an informed person, whether my test is proctored or not.
I very much agree with you that knowing a lot of facts is very helpful for developing arguments. But I do wonder what students should be expected to produce on an exam.
Perhaps it's about the type of question asked. Knowing the Sicilian Expedition happened after the death of Pericles is different than knowing it happened in (checks Wikipedia) 413 BC.
But then knowing the year is very important for the world historical context. At that era in Greek history, events in Persia had more impact than events in Italy and Britain was essentially unknown to them. Yet here we are living in a world where the Parthenon friezes are in the British Museum. It's hard to put together the different moving parts without dates.
I am looking for an exam proctoring solution at my employer (an accredited online university). What are my best solutions? Any solution can be hacked (i.e. workaround it’s limitations). And without proctoring there is no guarantee students will not have someone else attempt the exam on their behalf or send the exam to someone to solve it for them. The only approach I see, but not favored by the Deans, is testing centers (prometric, etc.).
I think I would rather let a dozen cheaters "get away with it" than let even one innocent person have their academic career damaged by a false positive.
Cheating is like crime. A criminal needs to be right every time to not be caught but enforcement only needs to be right once. Even if you let once instance fall through the cracks, it's very unlikely it's an isolated instance.
Academic cheating also doesn't seem like a sustainable life plan. Even if you get your degree and land a job, what are you going to do when your boss and co-workers realize you don't actually know how to do the things you have a degree in?
371 comments
[ 3.2 ms ] story [ 297 ms ] threadEdit: For example, remote sensors placed in a power plant or foundry where people work also would constitute surveillance. But it is in an environment where carefully calibrated machines can otherwise fail catastrophically.
To add, I suppose what I am really getting at is that it may be more useful to address ProctorU in particular rather than bring it under the same umbrella as other less used surveillance software.
For me, it made me realize that the word 'dystopian' in the title adds no facts to anything, its just a judgement. A negative one.
Coming up with a replacement is the hard part.
> Surrender to the power of complex technological systems — allow them to oversee, track, quantify, guide, manipulate, grade, nudge, and surveil you — and the system will offer you back an appealing share in its spoils. What is good for the growth of the technological system is presented as also being good for the individual, and as proof of this, here is something new and shiny. Sure, that shiny new thing is keeping tabs on you (and feeding all of that information back to the larger technological system), but it also lets you do things you genuinely could not do before.....The danger, however, was that “once one opts for the system no further choice remains.”
Shockingly, smarmy ed-tech hucksters don't have a good answer to this one.
Customers can't defend themselves against such intrusions of privacy from their school/government.
And employers too. They want you to respond to emails at all hours, and ask for way too much control over your device in exchange. This is slowly being changed with Android work profile and Apple "user enrollments", thankfully.
One alternate way to prevent this would be liability: if the institutions using this had to reimburse all of their users for every security hole in their mandatory software or the risk due to the security settings they require you to disable, it'd complete change the calculation for them.
> Seems they have no problem hosting software for ProctorU.
The restrictions are on what their apps can do, not on who published it.
There are many things which are technically possible which are not a favorable cost-benefit for most people. This is in the same category as those guys who relied on technically being able to fly without showing ID to the TSA — there's a reason why it was mostly affluent white men flying solo, because the potential downsides are much greater for most other categories.
I opted out of using spyware in the university physics courses I taught last year, and caught my cheating students the old fashioned way. Proctor-spyware, like airport security, is more about theater than effectiveness. You aren't giving the USMLE or a Bar Exam, so you can take the time to write a good exam and evaluate it correctly.
The total time spent in creating a few good, easy to discuss questions, the answering of which would demonstrate understanding, and then reading them, thinking about them, and coming up with a grade, was probably actually less than the amount of time it took to create meaningful multiple choice questions that didn't have any ambiguity, and which weren't easy to intuit the answer even without understanding. Doubly so when we did away with "partial credit" answers, but instead made it so each question was 1 point (or otherwise all or nothing), you had, say, 5 of them, and what we really were looking for was a paragraph that showed understanding (rather than checking off boxes in a rubric of "mentioned A, B, and C"; short essay questions, if you will), an expectation we communicated to the students.
And that's aside from the actual project based grades, which were better still.
By the way, where I work, management pushes for stuff like proctoring, and more students, and "measurable" results, and so on. As a teacher, I don't care much about the whole grading show.
Then the students are not getting what they pay for.
Must all of the cards in the deck be played at each turn? Things cannot simply be bad/user hostile/privacy invading, etc.
Its win win win for the accuser
It does require you to engage with people directly.
"This" clearly means the original accusation in the OP.
> It’s not clear to me that algorithmic proctoring is a modern surveillance technology that reinforces white supremacy, sexism, ableism, and transphobia. The use of these tools is an invasion of students’ privacy and, often, a civil rights violation."
I would especially consider that in the United States we do not have a broad legal right to privacy but there are potentially much stronger tools available if that software skews negative outcomes towards protected categories like sex, disability, or race. From the perspective of a student, job applicant, etc. being asked to use this, if the legal risks cause an organization to stop using it they'll enjoy that as a win even if the outcome isn't a blanket ban.
Not at the expense of grounded reasoning. When I see poorly substantiated claims, it shouldn't, but it really drags the whole rest of the argument down. The argument presented about lawsuits is actually a great example of why I think that's a broken system. They resolve that issue of lost credibility by considering each issue with total, clear, and mandated separation. Outside of that legal world with very well-defined rules, using such tactics reduces credibility.
I should note, that in this particular case, the claim of racial biases is at least substantiated by a believable anecdote.
EDIT: To clarify why I think the legal methodology is broken, its only because the same principles apply to criminal trials. IMO, prosecutors should NOT be throwing poorly substantiated charges at a defendant just to increase their winning probability and make the required defense more expensive.
It should. If someone is willing to make wild, unsubstantiated, claims, it should detract from their credibility.
If the parent comment had said something to the effect of "I should give the benefit of the doubt to a sitting US senator" then you'd have a point, but that context wasn't part of their statement.
Edit: Also, frankly, I wouldn't give the benefit of the doubt to a US senator. If anything, it makes the identity politics feel even more irrelevant.
I wasn't reacting to a summary, I was reacting to an independent premise. The sentence I reacted to, at least the way I read it, was broad to the point of being more of an axiom with which the commenter interpreted the post. As I see it, disagreeing with an axiom is an intellectual engagement. Feel free to counter my disagreement.
> It’s kind of hard to see any definition of “identity politics” which includes the reports but not your emotional reaction to accurate words.
Not entirely sure what you're saying here, but none of what I said was emotional. Simply pointing out that pattern matching is a viable way of filtering other people's thoughts and ideas. If someone makes wild claim, it should change the way you view other claims which may have seemed more rational in their absence. Not really an emotional statement in my mind, but again, feel free to point out which part of this you disagree with and I'd be happy to engage.
As it stands now you aren't really responding to anything I've said, but rather disagreeing that what I'm responding to warrants responding, which is rather tangential.
Case in point, here's someone saying, 'How can you even handwave away systemic racism?' in reply to a comment agreeing that bias exists but is not deliberate: https://news.ycombinator.com/item?id=29164295
Also, the identity politics argument seems to hinder a simple, moral argument against surveillance software as a violation of privacy. The logical implication of the IP argument is that this surveillance software would be okay to use if we manage to work out all the kinks.
It's not the same with general discourse - when you raise a bunch of issues that aren't especially relevant and seem designed to be inflammatory, you damage the credibility of your other arguments. Arguing that test proctoring software is transphobic is such a stretch that it makes you question whether they author has such strong biases against the software that their evaluation of it is just generally too biased to be trustworthy.
It seems like the software is matching people against existing images, based on the issue with the black student, and trans people are I would assume more likely to change their appearance, including as a result of taking hormones and having facial feminization/masculinization surgeries.
Yes, but with some evidence. Otherwise, as a society, this is a bad direction to head in.
https://www.technologyreview.com/2020/08/07/1006132/software...
https://www.nytimes.com/2020/09/29/style/testing-schools-pro...
That being said, there's a rotating list of "badnesses" that are in the zeitgeist and I agree that it's annoying to see them flogged at every opportunity (often w/o much insight).
The statement is at best extremely misleading, and at worse, mostly false. It also represents a juvenile, immature, and myopic perspective on reality.
I would suggest reading "Wealth, Poverty, and Politics" or "Discrimination and Disparities" by Thomas Sowell. He has been debunking the "inequality of outcome therefore racism" logical fallacy for decades.
B) Even if it were simply physics at play, requiring the use of a system known to have physical constraints against darker skin causing failing grades purely on that metric is still pretty racist.
I wanna pull this apart a bit because I think it's a good opportunity to talk about how systemic bias gets started. Digital sensor evolution is path-dependent. Technologists developed photo-sites that have "enough" dynamic range for most uses before moving on to increase the resolution on a sensor. What exactly is "enough" depends on your test data.
The sensor on a webcam is only "so big" as you say - but how that sensor balances resolution and photo-site count depends on what conditions they consider acceptable. We could build web cams that would see more pigmented faces better - there is no fundamental limitation in the technology itself. It's that a series of decisions have been made over years of development, generally without people thinking specifically about race at all, and we've arrived at a status quo that has adverse outcomes for people with different skin tones.
There was a similar process that happened with film photography[1]. Not that film, as a technology, is unable to capture dark skin - but that the development standards that were tested and distributed were designed for lighter skin.
Like, I agree that the webcams we have aren't intentionally 'racist.' But I do think that the status quo that has led everyone to accept this balance of dynamic range and resolution is reflective of valuing people with lighter skin more.
[1] https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-b...
https://www.newyorker.com/culture/cultural-comment/a-century...
It seems like you're reading a specific thing I said about a specific scenario and universalizing it in a way you imagine I might universalize it. I'd love to hear a critique of what I actually said, or we could talk about our views of society in a wider way, but I can't respond to this combination of generally dismissing what I said and attacking what you imagine I might think.
All you can do is laugh at this point.
I sense you’re tired of discussions that mention the “cards in the deck”, likely because you aren’t affected by them and therefore care little for them. That’s honestly fair, but there’s value in writing that way to channel outrage into action.
Great. Good to know where the zeitgeist is.
White supremacy: darker skin tones tend to photograph worse than lighter skin tones. Laptop webcams are notoriously crappy and can make this effect worse. Is this white supremacy or physics? Can the test instructions be modified to ask all users to have an appropriate lighting setup (ie lit from above and from the side to ensure that your face is foregrounded properly)
Ableism: for users who require assistive technologies, is it better to take a test in their own space with their own equipment or to travel to a test center and use shared lab equipment? For users with mobility challenges, is it better to be in their own space or travel to a potentially non-accessible testing center?
Sexism: for working mothers, better to take a test in your own home or travel to a testing center and arrange for child care?
Two "isms" seem more relevant but weren’t mentioned: ageism - because fuck boomers, right? Socioeconomics - not everyone has access to a PC that meets the specifications or can acquire one on short notice.
Lots of context is discarded when one engages in polarizing categorical rhetoric. I’m not here shilling for proctoring software but rather for nuanced discourse.
I’m not a photography expert, but it seems to me that cameras are physically equally capable of overexposing an image and underexposing an image. If a particular camera which is used in a facial monitoring system tends to do one rather than the other, I would ask why that is the case.
The choice to use typically terrible cameras in a proctoring system disregarding that it might work ever worse than normally for a subset of people is suspect, yes.
White supremacy is not just bad faith things done by bad people, it is also the assumption that whiteness is the default experience and the failure to account for that not being the case. Similarly with ableism and sexism.
That being said, complaining about the accessibility & inclusiveness of our required surveillance technology does have a dystopian feel to it, lol.
This only works with "new" definitions of racism. It is in fact plainly racist on it's face to demonize a group based on immutable characteristics. It is even worse when actual diversity of though is ignored and people of color are demonized because they don't agree with a race-marxist ideology.
>At the beginning of a test, these products ask students to verify their identity by matching their appearance with a photo ID. As Os Keyes has demonstrated, facial recognition has a terrible history with gender[x]. This means that a software asking students to verify their identity is compromising for students who identify as trans, non-binary, or express their gender in ways counter to cis/heteronormativity. If a student’s gender expression or name on their ID are different from their current gender expression or name, the algorithm may flag them as suspicious. When this happens, they may have to undergo another level of scrutiny to authenticate their identity, an already common and traumatic experience for trans and gender non-conforming students. If these students are not alerted of this possibility before the test begins, it may force them to either discontinue the test and risk their grade, or out themselves to their course owner when they may not want to, risking more trauma and discrimination including being denied financial aid, being forced to leave their institution, or have their lives put in physical danger.
>The Eugenic Gaze is a combination of white supremacy, sexism, ableism, cis/heteronormativity, and xenophobia. When we apply the Eugenic Gaze using technology, the way we do with algorithmic test proctoring, we’re able to codify and reinforce all of those oppressive systems while avoiding equity-based critiques because of our belief in the neutrality of data and technology.
Their recommendations are quite reasonable:
>Don’t use algorithmic test proctoring. Instead, focus on pedagogical techniques that you can use to design assessments, online or in person, that draw from personal experience or require students to apply concepts in unique contexts. If you have to use algorithmic test proctoring, make sure students know about the test settings and ID requirement well before they take a test, and assure them that you will not take any behavior flagged as “suspicious” into consideration that isn’t described explicitly in the syllabus.
The GP link[1] instead calls out "Facial Recognition Tech", and "Algorithmic Proctoring" as being too biased and follows up with a petition[2] to ban these entirely.
[0]: https://hybridpedagogy.org/our-bodies-encoded-algorithmic-te...
[1]: https://library.auraria.edu/news/2021/why-online-test-procto...
[2]: https://www.sheaswauger.com/post/petition-to-ban-facial-reco...
[x]: https://ironholds.org/resources/papers/agr_paper.pdf "The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition"
I think you just reflexively dismissed this because you saw a basket of words that normally go along with things you disagree with, but the argument is pretty solid.
Really? On the male part? I'd have expected it to do best with women, because I'd have figured facial hair is more difficult to deal with than a wider varieties of hair styles.
Again, I wouldn't chalk any of this up to deliberate bias against any of these groups, but it's all bias anonetheless.
Can you explain how racism that benefits white people specifically, as enabled through either direct, explicit bias in policies or laws, or indirect, implicit bias through a lack of diversity when conducting research on human computer interactions, or training corpus used for machine learning is not explicitly white supremacy?
These types of issues have been mainstream and well documented that they are at the centre of a 2009 episode of a sitcom (Better Off Ted, Racial Sensitivity), among many other more conventional studies of the phenomenon.
After decades of research (and centuries of practical observation) how is it even possible to handwave away systemic racism and bias "that benefits white people specifically" as anything other that white supremacy?
Edit: I accidentally left out the italicized part of the first question.
Racist systems that have encoded societies biases are generally not white supremacist. The 'justification' for those systems is often things like "this is just the way it is" or "this was easier to do like this" or "I went with my own experience".
These days a lot of racism does not come from white supremacy. It either comes from something like familiarity bias of people in power, or from following the status quo mindlessly. Calling those acts white supremacist can be dangerous. It allows the real white supremacists to hide among the unknowing. It also pushes people who unintentionally did something racist way into the defensive if you tell them they are white supremacist. And pushing people who unintentionally did racist things into defending their actions is not going to make things better.
That's a common dictionary or encyclopedia definition of the white supremacy. More broadly, white supremacy also refers to the systems and structures of power that are built into most of "western" (a better term might be post-colonial) societies that favor both white people, and people who support or uphold the balance of power in those post-colonial societies.
> Racist systems that have encoded societies biases are generally not white supremacist
I agree, however I think that those racist systems that are not inherently white supremacist in nature are largely rooted in non-colonial countries (basically countries other than the European colonial powers, and the countries that grew out of those colonies).
> It also pushes people who unintentionally did something racist way into the defensive if you tell them they are white supremacist. And pushing people who unintentionally did racist things into defending their actions is not going to make things better.
That is just not true. If someone does or says something racist, they can and should be challenged on it. If they become defensive, there are multiple reasons that could happen, but if the reason is that they simply didn't know better, it's just the way their society is, or if the reason is that they are opposed to "wokeness" (which is a catch-all for intersectionality, critical race theory, and many other modern perspectives and ideologies that are largely centered on dismantling power structures and reducing bias and discrimination), then it's likely that they are supporting white supremacy out of ignorance (whether that ignorance is from being uninformed or uneducated, or the more malicious willful ignorance of people who choose to use or engage in racist norms because they are opposed to "wokeness" from an ideological or other perspective).
Pushing people who do things that could be cast as unintentionally racist is the only way to a) educate them, so they can do better, or b) determine if it was an intentional act. I know this from practical experience, and it was only from going through the hard and painful experience of being called out on harmful "unintentionally" racist jokes and behaviour that I learned to do better after being raised by a family that had (and for the most part, still has) some pretty racist and discriminatory views.
what a vague concept lumping together unrelated things
the systems of power in one place are different from another place, they are unrelated, can't be reified like this
I still think we should not tell people who do racist things "you are being a white supremacist". Instead we should say "that thing you did was a bit racist" or even "that thing you did could be seen as racist". We should _definitely_ challenge those people. But if you are trying to get someone to change their mind or behavior, you gotta be real careful about their feelings.
Calling someone a white supremacist is going to hurt their feelings quite a bit. And that isn't going to make them more likely to consider if they should change or did something wrong. That doesn't mean don't challenge people on their actions. That means challenge people on their actions very gently.
If we don't treat these people gently, then we lose the less introspective part of the population. They start getting reactionary, start 'banding together' against these annoying people. They start 'fighting back in the culture war'. Basically, getting this wrong is how you create republicans.
All of the above means I think we should be considerate in how we challenge racists. And I think part of that involves being really careful about the term white-supremacy.
Id like to suggest that what you are calling "familiarity bias" might have a component of the quote in it too. Not salary in this case, but social position. That is in the racist system, one race of folks get better treatment, and if they want to maintain better treatment, the status quo must be maintained. The group of people at top of a racial hierarchy (that is in the supreme position), are incentivized to keep the racist system. When race is considered a bad reason to judge a person, they still are incentivized to maintain the system, just find different words to justify the status quo.
I guess a different way of saying this is - white supremacy describes a race based social hierarchy where white people are at the highest level. It has also been used to describe the lowlife Nazi or KKK wannabes that advocate for it in the baldest terms, but they are bigots who advocate for white supremacy using racist terms like "inferior genetics" or worse.
Compare the term racist itself - there are folks who would have you believe that the term is limited to personal bigotry against people of a different race, and has nothing to do with the rules and actions of systems (a position I think you don't hold due to your description of racist systems).
White supremacy, to me, implies explicit, militant, proactive racism. People who might very well be proud of the fact they're racists.
This story is about a software system that (among many other issues) doesn't work well with darker skin tones in low light. Especially in light of all the other failure modes, I'd ascribe that to carelessness or indiference, mixed with pressure to reduce false negatives at the expense of more false positives. I wouldn't be surprised if training data and/or testing were filmed in an office setting with the amount of light you expect there, and they never ran across the issues with dark skin interacting poorly with the amount of lighting a student would have a home.
That is only half of the story of white supremacy though. The other half is the entrenched systems and biases baked into those systems that largely benefit white people, and that train people, through experience, to prioritize preserving the existing systems and status quo.
Not considering the fact that there is a well documented history over the last 20 years of tech companies and business in general prioritizing the experiences of the white majority, at the expense of people of colour, is largely the reason why you can "wouldn't be surprised if training data and/or testing were filmed in an office setting with the amount of light you expect there, and they never ran across the issues with dark skin interacting poorly with the amount of lighting a student would have a home.", and not consider that being the norm, or even acceptable as being indicative of white supremacy.
Those biases may not always, and only impact people of colour, but they do overwhelmingly benefit white people. That's the entire point of the article that OP shared, and the references the author of that post uses to back their claims.
But something I always wonder when accusations like "white supremacy" are thrown around: is it falsifiable? What evidence would dissuade you from that?
- What if both ends of the spectrum do poorly and extremely pale people have problems, too?
- What if the threshold is dark black and lighter-skinned black people, Asians, Middle Easterners and other non-white people are able to use it successfully?
- What if only a narrow band of light levels work, making it clear their testing range was generally too narrow, not just in skin color?
- What if they took care to incorporate black models in testing, but the photo quality (and their own in-house cameras and lighting) overestimated the quality of most home users'?
And what of the myriad other things that were done poorly in the software: limited OS support, bugs, excessive memory usage, overall intrusiveness, browser limitations, disallowed mobile devices, lack of multi-monitor support? Do they likewise arise from systematic oppression of some group? What if we dig in and find that white people are more likely to use iPads, Linux, and multiple displays?
Most often these accusations flow in only one direction, and that all other flaws or problems are taken to be simply happenstance and noise. Certainly anything that impacts white people negatively will not be automatically seen as anti-white, although in a world with activist devs, such a result isn't incomprehensible.
Claims of white supremacy (among other accusations of character) are thus, to my mind, wildly speculative and carry a very heavy burden of proof.
Your way of grouping people by race is kind of arbitrary. It puts together rich and poor when they only share a skin color. How is white supremacy working in Bulgaria, for example?
You could read the research and refute it, or you could just bluster about things. I know you tried to expand on it in your comment below, but minimizing the specific concerns raised to "isms" and ignoring that at least two of the references in the articles linked and their references for the actual research addressed at least the socioeconomic portion of it, illustrates that you only applied your surface level perspective and criticism.
In this case, the algorithm is actually bad for all ethnicities [0]. It's just that it's extremely bad for black students (fails half the time) and just regular bad for everyone else (fails a quarter of the time).
[0] https://www.theverge.com/2021/4/8/22374386/proctorio-racial-...
Still, ProctorU it's a major deterrence to cheating, I'm just not sure it's worth the cost.
Article proceeds to use a lot of words to not show any of these being true.
I am against it for the general dystopian surveillance normalization it encourages. We don’t need to throw a word salad of made up progressive insults against it to resist its implementation.
Create an exercise that contains a technical term that does not exist, and create a page containg it with a plausible, but wrong solution for the excercise. Make sure that the page is easily found with Google. Give everyone who solved it using the wrong solution a failing mark.
Personalize every exam. Create a pool of exercises and choose n exercises per student, based on the student id. Easily done if the sheets are already LaTeX anyway.
Create heavy time pressure. Cheating is very hard when even completing all exercises regularly is almost impossible. (Lovingly called "Zeitklausur" in German, lit. "time exam", it's normal that students are unable to finish those in time)
Create exams that don't just test the ability to vomit knowledge, but test the ability to use that knowledge, and let students explain in their own words.
Replace the exam with multiple small projects and presentations.
All of those things were used by different chairs / departments in my university :-)
Nothing will prevent "someone else writes the exam for another student" with absolute certainty, yes. But neither does proctoring software.
The problem is of course regional accreditation rules of proctored exams
At the moment these are taken in person in secure buildings where you are identified at the entrance, your belongings are stored in a locker, and the exam itself is taken in a secure room on computers with no/limited internet connection, etc. You are timed and monitored.
This extends to coding interviews as well. Using the resources at one's disposal to get a sense of the landscape before diving into algorithms must surely be part of the job, right? What do I care if a developer needs a quick reminder before diving into a solution, or even reads up a bit and scans someone else's code before answering?
What is the value in ensuring that people have perfect recall if this is something that will almost never be necessary in a real world job?
On the other hand, at least a basic level of recollection is necessary for quick thinking in meetings, you don't always have the time to look up documentation.
Plus, it doubled as a study guide for the actual final which was only a couple of weeks later. I thought it was a remarkably kind thing to do. Took out a little stress. Gave even the struggling students an easy "A". And it worked as a comprehensive guide to almost everything we covered.
It's not an odd question. "Okay, the whole subnet where your credentials server used to be is now a smoking hole in the ground, and IT forgot to pay the fiber bill last month. What do you wish you'd done three years earlier to address this problem?"
To set people at ease I tell them up front that only one candidate has crossed the line (googled the solution) and everyone else has made perfectly appropriate choices; "elseif or elif?" and small details like that.
Shockingly, this was due to some JavaScript relying on an older Chrome proprietary API so there's no possible way they actually tested it against their alleged support matrix.
Lawsuits don't cost millions. Court fees are absolutely never that high, and lawyers, while some may be expensive, are generally affordable for ~middle class (or even lower class if someone wants to do pro-bono work for you)
The whole "lawsuits cost millions" thing is a myth perpetuated by big corporations and further relied by normal folk who hear it from somewher else, which probably heard it from somewhere else, and so on.
When you read in the news "X company wasted $XX million in legal fees", what it actually means is "they stretched out the case with a team of very expensive corporate lawyers whoses price ranges are in the millions".
That may be, but they can easily cost many tens or hundreds of thousands of dollars. Lawyers typically bill at multiple hundreds of dollars an hour so it doesn't take a lot of hours to rack up five- or six-figure costs. That's high-stakes poker for most people.
I once sued a neighbor for their barking dog. It cost me over $10,000 before I pulled the plug.
https://blog.rongarret.info/2009/07/dog-days.html
> "they stretched out the case with a team of very expensive corporate lawyers whoses price ranges are in the millions".
Yes - The company stretched the case out with expensive lawyers: Do you think the other side is somehow not obligated to also continue dealing with that case?
Who pays my lawyer while the company stretches the case out? Oops - that's still me.
----
As someone who has actually retained a lawyer for dealing with a previous employer:
1 - Most places had zero interest if the money at play was less than 100k (ie: They would not take the case unless I had a potential win of 100k or more)
2 - They charge ~$350 an hour. Sometimes billing for "intern" work at ~$150 an hour instead. I make good money (~200k) and I can afford less than 23 days of lawyer time a year, assuming I spend my ENTIRE yearly income on it.
The advice I have gotten from actual lawyers is that it's literally not worth it if you expect to get only a few thousand dollars.
This is especially work considering with this particular company, which has a history of using legal threats to silence critics:
https://www.gofundme.com/f/stand-against-proctorio
I would DEFINITELY not jump at the chance to incur a similar reaction.
Turns out, their JS minifier was creating code that contained a syntax error. Chrome was able to make it work, but Firefox would silently error out. Rather than try to solve the problem, they blocked any browser that wasn't Chrome.
-_-
Me, to <litigious tech company>: “Your JavaScript fails on IE8 because it now throws an exception when it attempts to set an invalid CSS value. I made a tiny patch but do you have an ETA for the fix?”
LTC support: “We don't beta test Microsoft's products for them!”
Me: “Okay, it was released this week. How's testing going?”
[a week passes]
LTC support manager: “Hey, can we get a copy of that patch to give to other customers?”
My employer at the time paid 7 figures annually for support.
[1] https://news.ycombinator.com/item?id=26898651
Buuut, after a while, we were like, why subject people to this? This is crazy. And why even charge for certs anyway? I'm happy we're done with proctoring!
Nearly every college student during the pandemic had to use ProctorU in order to complete their classes, or a similar alternative. Quite disturbing the experience is normalized, and I wish there were an official OS level feature for "report all activity on the system from time X to Y", without having to use a sketchy third party app.
I wish the author the best of luck fighting the requirement to use ProctorU.
Oh please don't. This type of espionage should be discouraged, not officially supported.
https://www.amazon.com/American-Plating-502N-Valve-Instrumen...
I go to UOPeople, which is a tuition free online school and I am getting a 4 year computer science degree (for like $5k which is crazy), anyway of the 40 courses you have to take only 11 are proctored. UOP offers 2 choices for proctoring, A) find a real life proctor, or B) use ProctorU. I don’t have the luxury of finding a real person + with covid its more unlikely, so ProctorU it is.
All in all its not the worst thing in the world, but when I first read the requirements for the ProctorU testing environment and technical requirements I almost quit school completely to look for alternative paths. Some of the crazy requirements include: Testing space must have nothing on the walls, or floors. You can’t wear glasses while taking the exam. Your desk must be clear of everything besides the specific testing materials (calculator if your lucky, and maybe a pencil and paper) Your device needs a webcam, so that they can not only watch you for the entire 1 hour and 30 minutes where you take a test that determines 40% of your grade, but also so that you can show them each wall, floor and under your desk. And thats just the physical space requirements. I had to empty my closet and use my laptop to take this test because their is no way I just have an extra room for testing…
The digital requirements were pretty intense as well, access to folders they had to right too, chrome settings and a whole bunch of wack stuff. I created a dummy account just to take tests.
When you go into the program they have you download it acts like a 1 way mirror, you can hear the proctor(if you are lucky to have a human proctor) and they can watch you, your screen and hear you. I had some tech issues once and I was grateful to have a proctor with a sense of humor who was able to help me through it. I can’t say my privacy is worth a cheaper degree, but I hope that this doesn’t become normalized, because it is not a pleasant experience.
Talk about security theatrics ... I'd just stick the answers I want on a piece of paper just under the cupboard in reach of your feet or with a tape on the bottom of your desk and get it off when scratching your crotch.
> access to folders they had to right too
Why? They are recording all processes and the screen already. Again just theatrics. And if you can hide it from that they are never going to find it anyway.
How is that legal? My face would need to be less than 10cm away from the screen. So - there goes using the camera to monitor where I am looking.
How is this not massively illegal? This is a clear ADA violation. I cannot see without glasses. Not like, things are a bit blurry, but like I have 20/800 vision that's correctable to 20/20 with glasses. Forcing me to take an exam without glasses is forcing me to fail an exam for a reason that has nothing to do with my academic abilities.
Actually surprised they don't require you to take the test naked.
I know you were joking about the naked thing, but another commenter above mentioned their girlfriend was told wearing a sleeveless top was ‘inappropriate’ and was asked to cover up. I just think that’s insane.
This shit is crazy.
Very unprofessional, if not illegal due to discrimination and even though it was one of Sans's entry level certs, I barely passed, versus 90 + on all of their advanced ones without these issues.
Their reasoning was "The rules say no facial obstructions, your glasses block your face.". They have to hire the dumbest people to do these proctors.
I've never had any issues with other proctoring services, and things like pearson for Comptia and Microsoft were actually enjoyable. With proctoru each proctor seems to find some issue and you have to argue with them since it's completely unreasonable.
Don't know what class/proctor you took, but I took several UoP tests with ProctorU and never had issues with glasses.
But QEMU/KVM which is the de-facto hypervisor on Linux is harder to detect. Even the others which I mentioned before can be hardened to evade detection.
And if you do a little bit of tinkering and intercept traffic, you can make it so that all the cheating reports from the "AI" never leaves you computer. I've never played with ProctorU but have experimented with a couple of other similar software. They usually send regular reports every five minutes and some anomaly reports (some extra software running on your computer, another person in room, face not visible, etc) when something happens. You need to intercept and modify traffic to not send these anomaly reports. This is easier if its browser based, but you need to install systemwide certs if its install-able software, and a lot more work if they utilize certificate pinning inside binary install-able software. I have never encountered the last one though.
That's after fighting with the software to have it installed in the VM to begin with.
Defeating malware's VM detection is very interesting.
Links for others if they're interested:
https://github.com/a0rtega/pafish collects all the best-known detection methods into a test suite.
This issue is interesting/has links for sure: https://github.com/spender-sandbox/cuckoo-modified/issues/45...
I heard that MS is requiring notebooks to have a HD front facing camera. Maybe they would still sign my device driver? If they don't, wouldn't that be a lawsuit waiting to happen?
That's more or less the only silver lining I can think of here.
There was no limit on how many notes one brought in to an exam. Some of the weaker students turned up with rucksacks full of ring binders and the invigilators had frequently to admonish them to make less noise rustling the papers! Those students almost all failed or attained only a pass degree.
In my opinion this successfully weeded out those who thought that memorisation was enough. The exams typically never asked anything that could be answered simply by looking up the answer in notes or even the textbook.
Which is a long way to "yes, I agree."
Perhaps it's about the type of question asked. Knowing the Sicilian Expedition happened after the death of Pericles is different than knowing it happened in (checks Wikipedia) 413 BC.
But then knowing the year is very important for the world historical context. At that era in Greek history, events in Persia had more impact than events in Italy and Britain was essentially unknown to them. Yet here we are living in a world where the Parthenon friezes are in the British Museum. It's hard to put together the different moving parts without dates.
I'm not following. If you don't know the material, you have same incentive to cheat.
Any suggestions?
ProctorU is absolutely, positively insane, but the alternative sounds quite reasonable.
[1] https://www.nytimes.com/2021/05/09/technology/dartmouth-geis...