A teacher at my university was fired for failing students who were caught cheating in an introductory CS class. The ones in charge didn't like that he was causing a disturbance. They would rather have had the cheaters get away with it. The system is broken.
That is indeed some disproportionate reaction to just failing some students in introductory CS class. (So much that I feel there must be more to the story.)
> The system is broken
This is truer than it sounds. The very design of Examinations itself is broken in most places. It shouldn't be possible by design to cheat in my opinion.
"Open-book" exams are harder to set, but I think the whole cheating-prevention infrastructure is harder, costlier, and pointless.
Even open book exams can be gamed. There are whole industries set up around enabling in-exam communication (e.g. concealed subvocal mics/headphones) and outsourcing schoolwork.
It is definitely possible to design assessments on which cheating is difficult. For example, oral examinations or personalized per-student projects or exams. The problem is that this style of personalized assessment fundamentally does not scale past a few dozen students in a classroom.
You want a classroom that small, it's going to cost you. Just instructor salaries for would run each student 10-30k per year, and that's before paying for infrastructure (classrooms, tech, offices) and (admittedly not always useful) administration.
Smaller class sizes is definitely a huge boon for everybody involved. I went to a mid-sized private uni, where a large class size was maybe in the low 30 students range, and many were closer to 10-15, and the experience was dramatically better than my friends at other universities got. Professors knew the vast majority of us by name, and had at least an approximate idea of your grasp on the course material.
Cheating was solved with a pretty straightforward approach: don't make it trivial to cheat (re-using test questions, question-bank multiple choice tests, etc) to keep the honest students honest, then trust the students. If someone's caught violating that trust, send them to the business school. (I don't know what they did with cheaters in the business school. I assume promote them)
Our best computer networks teacher from secondary school decided to leave following complaints from parents, who believed that the tests were too difficult. As I remember, about 3/4 of our class had a tough time. Honestly, the tests weren't all that hard. I suspect there was some crowd effect taking place, which resulted in people collectively giving up. This turned out to be a significant blow to the standard of our education. The subsequent teachers were neither as knowledgeable, nor as effective in teaching the subjects.
That may be university-by-university thing. I briefly managed an academic department at an old, highly-ranked university. Professors regularly caught cheaters and there was never any question that they would fail the course. The only question was whether the incident was mild enough that they would be permitted to try again next semester or severe enough to leave the university.
The administration supported us 100% as this was seen as essential to defending the value of the degree conferred on the other students. Without some kind of standards we'd eventually become just another degree mill.
What surprised me most was how many students who never finished the degree (not just cheating -- some just got job offers and decided not to stick around) still listed the degree on their resume/CV. The problem is that many companies now call the school to check as part of their hiring process, and the school would inform them that they had enrolled but had not been granted the degree. In retrospect I wish I had instituted a policy of letting the ex-student know in those cases, because I'm not sure if hiring companies tell them and I wonder how many people think they're just unlucky or something.
I once worked as a part-time visiting lecturer in CS.
I routinely got assignments handed-in that students had evidently copied from one-another. More than half of the students were handing in copied work. In at least one case, they hadn't even botherd to change the name at the top. Usually they had the sense to change variable- and function-names, but not always.
As a newbie lecturer, I asked my colleagues what to do. They said: "You can fail them. You'll be accused of racism (most of the students were brown-skinned). They will appeal; you'll then have to sit on exam boards through the summer, which is unpaid for a visting P/T lecturer. There's a good chance the school will overrule you, because these are paying overseas students."
"Or you can tell them that you've noticed the 'sharing' that's been going on; that collaboration and sharing is encouraged, but that they must never do it in marked assignments."
I adopted the latter course of action.
Being a part-time visiting lecturer is a crap job.
The linked article and this entire discussion is addressing the ethical issues that surround grading, originality, and plagiarism. How is AI going to be able to apply the "correct" ethical code when all of us can't seem to agree?
It's a decent exercise in ethics to point out what could be unethical about it. If you only consider the goal (catching cheaters) it's easy to convince yourself it can't possibly be unethical, but you shouldn't disregard the methods.
- Publishing wrong answers to a previous exam could confuse students who were simply looking up old exams to study.
- Identifying 'cheaters' as anyone who had a less than 1% chance of arriving at the answers randomly. This is wrong for 2 reasons, one is that they aren't answering randomly the other is that even assuming they are answering randomly you'd falsely accuse at least 1 student of wrongdoing on average.
- Not sure if a teacher has a moral duty to make good tests, but if they do then reusing multiple choice questions on a complex topic like ethics isn't ideal.
> - Publishing wrong answers to a previous exam could confuse students who were simply looking up old exams to study.
In any decent institution which does exams, an example exam sheet with questions similar to what you’re going to get, or even one of actual old exams, is usually published for just this purpose.
Sniffing around the sites the express (or commonly agreed upon) purpose of which is enablement of cheating, on the other hand, reveals the intent.
The claim that Quizlet is solely designed for the purposes of cheating is unfounded in my opinion. It doesn't look like it at first glance anyway.
I'm not even sure what a website designed to enable cheating would look like to be honest, but that's because I have no expectation that I'd be able to access the internet during an exam, and because I consider any study of related materials fair game before the exam begins. You'd have to get an answer sheet of the actual exam up front before I start to consider it iffy.
I once wanted to be a university professor. Stories of rampant cheating at some schools make me think it would've had to have been somewhere with a more honorable culture.
I've heard of some schools where supposedly the students take the honor code very seriously.
If you have at least a masters degree (bachelors in some states), it's not terribly difficult to get an adjunct lecturer job. You're generally committed for a whole semester, but that's basically it.
I guess it depends on how obviously wrong the given answers were. At my university it was allowed and encouraged to practice for exams using old exams. I could see someone having studied that way and assuming the fake answers must be correct.
Nah, the professor isn't in the wrong. A study guide was provided, the school disallows looking at other tests, and he told the students not to cheat-- and the consequences. I don't agree that there is nuance here.
"If you get caught cheating, there will be serious consequences, ranging from X to Y. The decision will be made by the board together with the ethics committee."
I think there’s a difference between looking at an old exam and its answers, figuring out why each was right and wrong, and learning from it, and just finding an exam online and using the answer key as your answer in a test because you couldn’t be bothered learning.
The interesting part here is that at least some of the students might have thought it was an old exam from the same course, used it for studying, learned totally nonsensical answers, and then gotten confused in the actual final. Evidently not all of them straight-up copied the answers, or they would have gotten much higher scores. This is apparently still a violation of his institution's academic honesty policy, but there are interesting questions about what happens if he e.g. poisoned one member of a study group who went on to teach bullshit to the other members, giving them a higher chance of picking the wrong answers and putting them over the threshold without ever knowingly cheating.
Of course this is irrelevant, as the p<0.01 test gives an up to 62%[0] chance that someone in his class of 96 students could be falsely accused of cheating, and is unethical in itself. Apparently one student managed to be right on the threshold, which the professor has chosen to interpret as "the student possibly cheating" rather than "me possibly ruining my student's academic life for no reason by choosing a test with lots of false positives and applying it to a class of 96 students".
[0] 1-(0.99^96), and though the real chance is probably a lot lower if the questions are obviously wrong, it's still a stupid threshold to choose
For all of my major exams in the UK, we were encouraged to study by practising on old papers and my score would have been abysmal if I hadn't. So not being allowed to look at past exams feels strange to me too.
Unrelated to the details of this case, but my school had an official, department associated organization/club called the Society of Software Engineers that would run study sessions for CS, SE, and related courses that the students would have to take. They also took in old exams from students to create a knowledge base that you could study against. The rule was that you weren't allowed to leave the room.
It was an interesting grey spot since most professors were all for using old exams to study, but some weren't (I don't think they liked writing new exams every semester, but I wouldn't either). Some would give threats for giving your exams to this university recognized academic club, but they'd black out names upon receiving them so there wouldn't be any proof that it was you (at least easy proof).
It was an interesting case there but most staff were all for it. Making that old data available to future students, but in a more restricted way seemed fine to me, and it absolutely helped me study, but I wonder what others would think of that situation.
It is at the universities whose policies I'm familiar with, yes. At both universities I have attended it was included in the policy alongside all the typical cheating and plagiarism rules.
> who recently caught 40 of the 96 students [...] cheating
That 40 is reported suspected, by one individual. The same individual claims that only 2/3 of those were admitted at time of writing.
Also, suspicion was by only a 1:100 coincidence probability standard of evidence, and by some imperfect metric. IMHO, that threshold would be too low to "prove" guilt in such a potentially serious matter (negative mark on student record, reputational damage among college social and professional networking peers, and potentially including suspension or expulsion).
He says that only one student was at roughly the hundred to one level, so he thought that one might be honest. The rest were at the billion to one level.
He also said that he is talking to others about what he should do. It seems like he very sensibly is using this as a starting point rather than accepting his work as the final verdict
> Also, suspicion was by only a 1:100 coincidence probability standard of evidence, and by some imperfect metric.
...only? In a sample size of 96, what's the value in analysis beyond that level?
No, this seems pretty solid to me. The math to do this is routine freshman statistics stuff, something every practicing scientist knows. The assumptions just require that wrong answers be roughly evenly distributed across the choices (e.g. you could construct a test where everyone who was wrong would be led to choice C and rarely B or D, but that's a little pathological; and regardless it's something that would be evident in the data set and seems not to have been).
I mean, standards vary but I'll bet in most jurisdictions 100:1 odds count as "beyond a reasonable doubt" for jury instructions in criminal trials. At the very least you'd bring the trial, which is what happened here. If I know there's only a 1% chance I'm wrong, it's absolutely valid for me to accuse you of cheating.
...I have no idea where I heard this, but for some reason I was under the impression "beyond a reasonable doubt" meant ~95% confident. (Whereas "preponderance of evidence" meant >50% confident.)
Per Wikipedia, the original framing is Blackstone's ratio from the enlightenment, where he used 10:1. Real courts don't put numbers on this, of course, but that's the thinking that produced the philosophy. Basically: no, given the level of innaccuracy we already know to be present, I think it's very clear that a sincerely-believed 99% confidence will send someone to jail pretty much anywhere in the US.
The quoted sentence I objected to was the sloppy writing, conflating of "suspected" with "caught". I think "caught" would be read by most readers as implying guilt.
Suspected at 1:100, certainly. Quietly confronted and questioned at 1:100, sure. Convicted at 1:100, absolutely not.
I'm not sure this is the best analogy, but imagine that, for every 100 people in society, 1 of them is wrongly convicted, at random. Further imagine that this lottery happens repeatedly, so that even more than 1 in 100 end up getting hit. That sounds like a miserable society in which to live.
Actually, I can't seem to get the maths to work. Isn't it just the Binomial distribution? Each question has 5 options so the probability of 'success' is 1/5 and so to get 19 questions 'specifically wrong' out of the 45 planted questions by chance is just (1/5)*19 * (1-1/5)*26 * 45!/(19!26!) = 1 in 2588 but in the article it is 1:100. What am I doing wrong ?
Here's my best guess: 19 is the cutoff point for a binomial test [1] where the probability of at least that many answers to match those in the honeypot test goes below 0.01. But this holds only if you assume p=0.25.
Why would you use 0.25 instead of 0.2? I guess it would make sense if you only looked at wrong answers - that is, you wouldn't be asking "what's the probability of your answers matching those on the fake test" but rather "what's the probability of your wrong answers being wrong because you used the fake test". Since you are only looking at wrong answers, your probability is 1 in 4 instead of 1 in 5.
"1% false positive rate => 99% confidence about a positive signal" is a really intuitive step to take, but you should be careful because it turns out to be horribly wrong in general: https://en.wikipedia.org/wiki/Base_rate_fallacy
(Perhaps not in this case but in general this can get you into trouble)
Though personally I'd say that it's quite likely that the p=0.01 threshold falsely flagged a non-cheater, which to me says it is too lax.
The 1/100 chance is assuming that students choose all answers with equal likelihood, but supposedly these answers were obviously wrong. If the probability of accidentally choosing the obvious wrong answer drops even by a few percent, that would significantly decrease the likelihood of innocence.
I'd rather avoid the question about ethics in this case. Is it ethical to accept payment for teaching? Is it ethical to charge as much for as (some of) US univerities do? Is it ethical to devise a test in such a way that a number of students will certainly fail? Is it ethical to prevent students from cheating? You can bend the word just as long until it means something like "might it cause any harm at all?"
But if you ask yourself: why do the students take this course, you can think the reason is to learn something. If they want a piece of paper that proves that, they can't cheat. If you --cynically-- assume that students only take the course to get their diploma with as little effort as possible, then cheating is allowed.
As long as schools want to teach something, they should stay away from online or take-home exams and essays. If they want something else (and usually the motivator here is money), the value of their education will drop.
> in his online Introduction to Ethics course cheating on a take-home final exam.
Sounds funny that an ethics class gets heavy cheating. But it might be a required or elective course that students don't take seriously, or it might be disproportionately Philosophy majors (which major might get more than their share of frosh starting to wield off-the-wall ethical rationalizations).
> Sounds funny that an ethics class gets heavy cheating.
Good observation - got me thinking. I see lots of suggestions here about requiring ethics training/classes for developers, so it made me wonder-- can ethics be effectively trained into a person predisposed to behave unethically if consequences for the behavior don't exist?
Edited for clarity and moved under the right comment
One problem doing this for tech is that we've normalized so much bad behavior.
For example, go back 30 years, and make ordinary software spy on its users, or otherwise act against users' interests, and I think you'd be seen as evil and possibly criminal.
One promising thing, though, is that college students might still be be more open to confronting issues of ethics and morality. (Though often it's only "Revolutionaries till graduation", because of sheltered circumstances or social fashion.) But college is a bit late, and students are already being driven to mercenary behavior by e.g., competition for FAANG jobs (like I used to mainly hear about desperate/ruthless behavior by some pre-meds). Ideally, we teach better values from early age, including by example.
> I decided to ‘poison the well’ by uploading [to Quizlet] a copy of my final with wrong answers. (The final is 70-80 questions, all multiple choice, 5 options each.) Most of these answers were not just wrong, but obviously wrong to anyone who had paid attention in class. My thinking was that anyone who gave a sufficient number of those same answers would be exposing themselves, not only as someone who cheated by looking up the final online, but who didn’t even pay enough attention in class to notice how wrong the answers were.
I think that’s reasonable. I would even consider expelling students who did this; specially in college, specially in an ethics course.
I had a prof that did something like this. It was back in the 80s, so no internet. Instead, because it was a large class, he had to give the mid-terms to different parts of the class. He would make all the tests look similar, but with small differences. If the right wrong answers turned up, he would fail the student. (I hope that makes sense.)
At the end of the semester, he explained what was going on. The course was required for the degree (EE), you had to have a C or above to pass, and it was a bear. He said right out, the course was designed to cull the heard.
A few students were outraged. I'm not surprised by the reaction Merriam got. Speaking for myself, I was indifferent to what he had done, though I didn't speak up. I think he should ask the students that he determined hadn't cheated what they think.
This feels harsh because if the tests were similar but with small differences then getting the right wrong answers seems likely to happen anyway. Also, he went the extra step and outright failed the student but I'm not sure why that extra step was justified, surely the cheaters were automatically going to get very poor marks.
>> He said right out, the course was designed to cull the heard.
That seems unethical too. Students are paying to learn material. Professors are ostensibly there to teach it. Making it extra hard seems to go against that.
Are you trying to teach kids to not cheat? Or are you trying to teach them how to conceptualize, illustrate (if not demonstrate), and structure ethics such that they can make critical and thoughtful deductions or contributions to the field?
Because, if it's the former: great job, Ranger Rick. You definitely used a method that will root out cheaters and give them some (small/limited) incentive not to cheat anymore.
But, if it's the latter, you've failed your students in every respect. It's not even a clean example of the ethics of cheating, because you've tipped the scales in ways that affect multiple variables, instead of just one.
Neither of which is ethical or unethical in the vacuums of consideration that any subject remains neutral in. But if the context is that it's a philosophy class, I would expect the teacher teach me philosophical ethics and let the chaff fall where it may, rather than try to "prove" some nebulous idea of what it means to "know" something, and why one method of being able to 'prove' it is inferior to some other method. Put simply: I'm in this class to learn. If you're giving me the information and then I pass the test, that's your entire responsibility. Whatever third parties are doing - so long as it's not infringing on you - is not relevant. Not to your class and my grades. Sorry you're one of THOSE teachers, but learning isn't a test. It's a lifelong pursuit and you can't force people to pursue what they're A) not interested in or B) deft enough to use digital memory for.
I think both? At least, when I was in school, I thought one of the first-order lessons they were trying to teach us was to not cheat, while also teaching us other things.
Yeah, but WHY would they want to teach you not to cheat? It's not a virtue, in and of itself (cheating has to be defined for each context it's in so "don't cheat" is like saying "don't be bad"). They teach you not to cheat because if you cheat, you won't know the answer when you need it.
If you're a doctor, and don't know the answer to a question when you need it, you might kill someone. A lawyer has problems there, too. There are great values in keeping people from relying on deferred knowledge, in a great many applications.
I just don't see how philosophical ethics - especially not a mid-level course - benefits from those same lessons/limitations. If a software developer isn't expected to know every language pattern; only how to look up the patterns they need, when they need them, then why would then be disallowed from doing that on a test? A better test would target different metrics than "correct answer" or "incorrect answer". The same thing applies here. You can do software dev without being able to pass an "internal knowledge only" test. You can be productive with ethical philosophy without being able to pass an "internal knowledge only" test.
So yes, both is the goal. What I'm getting at is that both SHOULDN'T be the goal. In a lot of cases. Most, I'd hazard to guess.
That's not why they teach you not to cheat. That's why they teach doctors (and other kinds of students) that they need to memorize some things in order to have fast recall without requiring a reference.
But they teach you not to cheat because cheating is bad, on its own terms. The purpose of education is not just to teach facts, but also to teach young people how to be net-positive members of society. (This is why society subsidizes education.) Teaching not to cheat to get ahead falls within that part of the curriculum.
Nothing I said implies I think education only teaches fact. Obviously I believe the contrary, as evidenced by my comments. As far as education teaching people how to be net-positive members of society, I agree that's what the education we subsidize is intended for, but that doesn't apply to this situation. A mid-level psychology course at a higher learning institution. If you honestly think that the GOAL of that course is to teach children not to cheat, I think you're grossly mistaken. First and foremost by the fact that not all students are "young people". And then by the fact that we have other courses, even in higher learning institutions, that deal with teaching the students ethics. In the other classes, as with all other areas of life, you're just expected to employ the ethics of the society. It's a meta-reinforcement of those lessons, but the reinforcements are not the lessons of the other classes. If a student fails in their ethics classes due to their ethical decisions, that makes sense. If a student fails math with the right answers but the wrong "ethics", that doesn't make sense. Unethically sourced math answers can still provide the solutions to problems. Just ask NASA.
So then, if a professor of an unrelated or tangentially related field takes it upon themselves to make a lesson out of what should be a meta-reinforcement, in a field that they readily admit that they are unprepared to work in or do data analysis for, that is not a reasonable source of ethical education. Especially not when done without prior social experiment approval and safeguards.
There are two different questions: 1. Are these the right rules? and 2. Should students be taught to follow well defined rules?
You're almost entirely talking about #1, and I totally agree. The vast majority of classes should not forbid looking stuff up because it is an artificial constraint that doesn't reflect the reality outside school.
But I think #2 is also important. I know it's passe, especially in the silicon valley milieu that we swim in here, but I think "not following rules you personally think are stupid is good" is a bad lesson for students to learn.
(FWIW: I thought the opposite when I was a student, but I was a short-sighted idiot when I was a student.)
> But they teach you not to cheat because cheating is bad, on its own terms. The purpose of education is not just to teach facts, but also to teach young people how to be net-positive members of society. (This is why society subsidizes education.) Teaching not to cheat to get ahead falls within that part of the curriculum.
That is highly debatable. Cheating most often affects the cheater, if anything, because when they need a skill they pretended to have, they'll fail at real-life tasks. But nobody else is affected by cheating.
> Cheating most often affects the cheater, if anything, because when they need a skill they pretended to have, they'll fail at real-life tasks. But nobody else is affected by cheating.
Except the people standing on the bridge designed by the civil engineer who cheated, or getting radiation therapy from a machine programmed by a cheating software engineer, or being treated by a doctor who cheated in med school.
Cheaters getting degrees from an institution that's supposed to produce high-quality graduates is bad for the value of that degree, which is bad for the institution, anyone else who went there, and anyone who relies on that degree as a signal that the graduate has the skills they're supposed to have.
Also, the lesson is broader than this. In the real world, "cheating" is fraud. The lesson I'm saying educational institutions have a valuable social role in teaching is: follow the rules. I know that's naive and passe, but it's also important and a lesson most people actually learn well, to all our benefit. People who think they are entitled to cheat and get away with it are bad for society.
> The purpose of education is not just to teach facts, but also to teach young people how to be net-positive members of society. (This is why society subsidizes education.)
If only this were true. If you read the history of education, it was generally invented to create compliant factory workers that were adjusted to rigid schedules and strict authorities. (Its not a coincidence that so many schools are named after robber barons. Rockefeller, Carnegie, Vanderbilt, etc) University used to be a bit different, but now that everyone is expected to go and its about vending credentials it's arguable that creating well rounded individuals is at best an occasional bonus.
Thank you for this comment. It's fascinating how everyone else in this thread seems to view education as a tool purely to acquire qualifications rather than for actually learning anything. I wonder if this is a US thing that has come about as the result of education becoming a financial instrument.
I honestly think it's more of a reactionary thing. Cheating is bad. That's a pretty uncontroversial statement. Whatever the criteria for "cheating", if we're all agreeing that you did it, that's bad and deserves 'correction'.
Once you're there, there's not a lot of reason to go further. "Did he cheat? Yes? Then he bad." Easy peasy, next topic of consideration. Why spend more time on something that has a simple answer?
In this scenario, the students did what would have unequivocally been a "cheat", in generations past. They looked up answers and used them to prove knowledge. That's what every person who ever failed a math test for cheating did.
So, again, even going deeper, this is cheating, plain and simple. No need to wade in any further, when you've already given the benefit of the doubt.
Of course, this not being math, and technology replacing the need for such conventions in math are both important considerations that change the context. But, you have to give it now a third thought before you even get to this point, and of course, here's where things are quite subjective. I bet people would still pretend that knowing math is important for whatever corner of life they inhabit ("I work construction and geometry makes you a pro", "gotta be able to calculate tips", "you'll get killed on contracts with tricky rates"). Most would also quickly concede that if they were just taught how to get to interfaces (apps/websites/etc) that solve those problems, that would be effective as well (even if they couldn't help adding "but knowing is faster/better/convenient/etc"). And so, not only are we dealing with subjectivity and historical bias, but now also a confirmation bias. And all of that is on top of the authority bias that a lot of people START with ("the authority said don't do it; the students did it; they should fail."
So, personally, I don't chalk up to indoctrination/conspiracy/malice anything that can be assumed to be myopia/selfishness/fear/distrust/etc. Humans are capable of the formers, but far more susceptible to the latters. That said, I'm never going to defend the US educational system, so I'm not disagree with you.
Your point about it being an authority bias is really interesting and in line with a lot of the political views I encounter on this venture capital forum. It might also tie in a bit to the US disposition towards punitive vs rehabilitative imprisonment.
Everyone is very quick to suggest brutal punishment for the people who have contradicted the will of the authoritative system, but people aren't really so interested in discussing both why they've done so, and the societal consequences of that disobedience. And even then, not considered the impact on the individual being punished; many people seem to be wishing permanent banishment from the university system for cheating in what is probably a random side unit elective class.
Cheating is extremely common, so much so that students cheat during an ethics exam. People overwhelmingly find justifications for their actions, and cheaters are no exception.
From the comments in this thread you can draw the obvious conclusion: many people here justify cheating because they are cheaters themselves and they don’t want to acknowledge cheating is shameful.
I think you have to verify that cheating did not occur in order to verify if they have learned the ethics espoused by the course. Tests function to prevent those who have not understood the course material from claiming they have. In that situation verification of truthfulness is required before you can verify their skills
That assumes that a student is incapable of understanding the ethics without being able to practice the ethics. I think you would have a high bar in proving that to be the case. A lot of people do shit they, personally, find unethical, so long as they can square it with their own contexts of exceptions.
The provided test tries to have it both ways, but that's invalid. If they are supposed to be learning the answers to the questions on the test, they can prove that even if they looked those answers up. But if the test is to not look up answers and then they get a score reflecting their ethics, then the class is meant to make more ethical students; not to have more people understand the answers to the questions on the test.
At best, you could make the case for the ethical test being a component to the class. Give it a rigorous score, just like you do for any other answer on the test (pass/fail seems reductive, in this scenario, but it's reductive for a lot of tests answers, so whatever. bless the teachers who give partial points). But verifying what a student KNOWS does not, in any way, involve what ACTIONS they take.
You make a lot of good points and it's made me think that there is a fundamental assumption that tests are useful. I think that that is not the case. As you said this test attempt to both validate knowledge and ethics and ends up invalidating itself. It seems to be a flaw intrinsic to this style of test.
I have taken some very well constructed tests. They were not takehome and only one was multiple choice (it was a systems class so it became sets of binary answers which became more than multiple choice). I think that ensuring the integrity of answers in necessary for gaging learnings from a test like this but that need, as you've implied, would indicate that the test is the wrong tool
I think you're right in that most test aren't useful for what they are trying to assess. And that you can't validate anything that you can't trust the integrity of.
But I think the takeaway is that "critical application" is what constitutes a valuable test. In math, that just means "can you use the procedures of mathematics correctly, resulting in the correct answer?" But in, say, English, memorization has very little value. A valuable test would be in testing whether or not a student can match events in a dramatic story to, say, modern real world events and critically discuss the similarities and differences. And, unfortunately, writing that out clinically and expecting a student to both understand it and be able to do it on command while under pressure of failure is not exactly conducive to positive results. So English tests really shouldn't look anything like Math tests, once you get past the point of memorizing definitions.
The same is true of all disciplines. Tests should be generated (and often updated) based on what is currently useful to the discipline in a real and practical sense, at a fundamental level, and then on a more philosophical or exploratory sense once you get into higher learning.
With all due respect, this is nonsense. This is a teacher and a human being that exists and works within a system. You seem to be one of those people who thinks that teachers are some kind of supernatural, super-powerful heroes of humanity who have the responsibility on their backs to be larger than life. This guy has to apply exams; it's not something he wants, it's what he needs to do in the system. And he has to grade 96 of them. And he is a human being. Do you know what it would take to assess 96 students in the way you are describing?
Plus, cheating or not cheating is an ethical matter, not a pedagogical one. He is not trying to teach anyone not to cheat. The expectation of not cheating comes as a pre-requisite of any formal education. He has the social duty to capture cheaters because if not he is failing the society and system who have given him the responsibility to not only educate, but to certify someone's skills. If the cheaters get out there with a diploma in their hands, certified by him (and his institution), he has failed those who believe in that diploma's validity.
Everything you said about learning being a lifelong pursuit that can't be forced, etc. are personal pursuits which are independent and orthogonal to this teacher's formal responsibilities in the educational system.
I think it amusing and I don't have a problem with what he did, but I guess I don't really understand why teachers get so fired up about the subject in the first place. If people are in the class to fill out a general education requirement or something and otherwise don't care about the topic or will never use it again, why get so vigilant about it? And the students that do want to continue in philosophy would just be punishing themselves by cheating.
It’s literally his job to care about that. Chances are the students in his university classroom got their place in competition and other people weren’t so lucky. Plus everything else the previous reply said and also in mine.
No it's not, it's his job to teach people that want to learn. If uninterested people that are forced to be there drift through without creating a fuss, who cares?
Let me ask you people something. Of the college graduates that have been out longer than 2 years, what do you actually remember about non-major classes? My guess is: basically nothing. It's a waste of everyone's time. I don't support cheaters because at the end of the day you agreed to the rules, but I'm not upset that people break dumb rules to save their own time.
Grading on a curve also strikes me as bad. Either you know it or you don't. If I score 50% in the class, but 60% of the class cheated so I pass based on the curve.. is that a good thing?
I was assuming that calibration of difficulty implied some sort of curve. I'm admittedly not a teacher so there's much I don't know, but I don't really understand why difficulty would be a useful metric, which is what I was getting at. The only useful metric is if they can apply the knowledge somehow.
To the original point, if someone cheats, they weren't interested in the first place, so why were they there? Likely they were coerced. I just think this effort of "gotcha"ing people that don't give a shit about the subject in the first place might be better spent either showing them why they should care, or working with the people that are actually interested. What could this professor alternatively have done with his time compared to this, which is a headache of dealing with these people that are going to have to, bitterly, retake this class, or worse, leave school altogether? What was accomplished here?
>I was assuming that calibration of difficulty implied some sort of curve.
No, it just means "last year's exam seems to have been a bit too difficult, so I'll make this year's a bit easier" (or vice versa). Curving means that you assign grades based on a student's performance relative to the others in the class rather than on their absolute score on the test.
>but I don't really understand why difficulty would be a useful metric,
If you really don't understand how it's possible for an exam to be too easy or too difficult, then I'm not sure what to say. The exam is intended to differentiate between students who've mastered the material, students who understand the basics, and students who learned nothing. To do that it has to be difficult enough that people who skipped half the classes† can't get full marks, but also easy enough that not everyone fails it. Even these basic boundary conditions on a good exam can be quite tricky to satisfy in some cases.
†Not that I care about people skipping classes per se. I just mean that typically they'll have significant gaps in their knowledge of the material.
To me looking up old exams is normal, it's not even remotely cheating, heck it may even be expected. Saying a website that lists old exams is 'ostensibly' a study aid seems disingenuous. Poisoning said site with wrong information is just making things harder for the students which is the opposite from what a teacher is supposed to be doing.
However what I don't understand is why that even mattered.
Were the students just learning the questions and answers by heart to regurgitate them on the final exam? If they had any understanding at all they should have caught on, but even if they didn't they would simply demonstrate their lack of understanding, it is not dishonest.
Or did they get to fill in the answers unsupervised somewhere? Because if they were left unsupervised with access the web then this is likely just the tip of the iceberg, they could more easily cheat by discussing the questions with each other.
Edit: Reading more carefully it was a take-home exam apparently, which seems to have consisted of multiple-choice questions that are largely the same each year. I can vaguely see how looking up old exams would invalidate it as a test, but if your test is invalidated by normal exam preparation is it the exam's fault or the student's?
For all he knows, the students went to a site with study materials to prep for a test. His policy of disallowing students to refer to prior tests was not even explicit in the syllabus, but behind a link. If this policy were clear upfront - absolutely, reference to outside materials is inappropriate and there should be consequences. But I don’t think it’s reasonable to impose these consequences after the fact based on a reference to a reference.
Based on his post, my assessment is that the prof is lazy and an asshole. Maybe there are other relevant factors that would change my analysis, but I have to think this is the version of the story that puts the prof in the most favorable possible light, so….
For all he knows, the students went to a site with study materials to prep for a test.
Generally previous exams are good for checking that you understood a topic but you can not really study a topic only from past multiple choice tests. If they had studied the topic and gained some understanding, they would have noticed that the answers are wrong. Even just looking at two different past exams would probably reveal that something is off.
I had a similar reaction. I was surprised that this school's policy forbids looking at old exams. I checked my alma mater's policies and they do not. Students openly sought out copies of old exams from upperclassmen or from repositories kept by a few cliques/clubs. Some professors themselves provided a collection of old tests so everyone had easy & equal access.
I guess an important difference is going for engineering, it was trivial for professors to change numbers & details from year to year so old tests could aid study but not provide answers.
>To me looking up old exams is normal, it's not even remotely cheating, heck it may even be expected.
When I was at uni in the early 90's, several of my professors handed out older versions of their tests so that we could see the shape of the thing and prepare. We all appreciated it, too.
Yeah, I feel the same. Where I went to university, printing, scheduling and proctoring exams was a central university function, and it was university policy to publish hard-bound copies of exam papers - including for courses no longer offered - going quite a long way back in the university library (both for students use, and for future academic study). Lecturers / course coordinators have no say on whether this happens.
Students would commonly complete multiple past exams going back years, and anyone struggling with a question could ask a tutor about it in one of the labs (so that they actually learn the underlying principles).
A lecturer re-using an exam question verbatim in that environment would be considered reckless.
It sounds like the rules are different a Sacramento State University - perhaps to facilitate verbatim reuse of questions (which sounds like sacrificing educational quality for lecturer convenience). I do think, however, that this is exposing poor educational practices that reflects negatively on the university and the lecturer as much as on the students.
how would they have performed if they tried to cheat but didn't find the poisoned answers? would they have had to come up with answers on their own? suppose they cheated for 10% of the answers but had to give honest effort on the remaining 90% and just made the cut?
i don't think it's very clear cut or obviously perfectly fair.
> how would they have performed if they tried to cheat but didn't find the poisoned answers?
I don't think this matters. Cheating was against the rules and a cheater demonstrates a failure to adhere to a social contract and voluntarily waives their right to be assessed the same with those who did follow the rules. At least there are avenues for a cheater to have another attempt, even if they have to move to another institution.
Their hypothetical score is irrelevant, as they will be receiving a zero due to cheating. In your hypothetical scenario they are still cheating, in which case their improved score would also become a zero.
Today I had to submit my final grades for my Java class, meaning I had to grade all the final exams for it last night.
During the exam (which I had to administer remotely this time) I made it abundantly clear that they cannot use any kind of AI assistance for their work. No ChatGPT, no Copilot, no Bing AI, no Google Bard, and also no Googling, etc. I repeated this several times and also wrote it in bold font on the top of the test. I really didn't have any way to enforce this, but I was hoping people would be honest.
I'm pretty sure that most of the students were honest on this; the answers I got generally fine, but had grammatical mistakes and were "basically correct but had light factual errors that are common with people new to programming but aren't bad enough to count as 'wrong'". One student, however, who has submitted broken sentences and broken code the entire semester, managed to suddenly have decent writing skills, decent explanations of everything, and his code was clean and concise.
I'm about 95% sure he used ChatGPT to generate answers to the questions. I tried getting ChatGPT (and Bard and Bing AI) to give me a word-for-word copy of what he submitted, but I couldn't. It got somewhat close, but never an exact match.
Technically, it's possible that he just studied very very hard and his code and grammar improved. It's also technically possible that he used Grammarly to make sure his writing was ok (which was technically against the rules but I wouldn't really consider cheating in a Java class), and so I just had to swallow my pride and grade the test assuming he was being honest.
It's kind of upset me all day; I have worked pretty hard trying my best to be available to students if they have questions, and I worked pretty hard to try and make sure that the final exam was a reasonable level of difficulty. I think most of my students were fine, but one bad apple is enough to really ruin my day.
Whilst it's unfortunate this kid might be cheating the system, I don't think it's worth being upset if they got through. The point of these exams is to allow students to study and learn these topics, which it sounds like they did. It sounds like while maybe you failed in gatekeeping the cheaters (an impossible post gpt task) you have succeeded in the real goal which was to help these kids learn and improve.
Oh I largely agree; I'm kind of learning that you have to view teaching as somewhat more of a "statistical success" than anything else.
It's just one of those things that while you're totally right, the one or two bits of failure kind of nag at me. It's extremely easy to take these things personally (especially for a nascent teacher like me) when in reality I should likely view things as transactional.
A good and reasonable sentiment. It's a very human thing for people who are largely successful at something to fixate unproductively on failures--and in this case you can't even conclusively say that it was a "failure".
I've gone back and forth about how I feel about grades as a whole.
I do feel that by having these tests and grades that it does, indeed, sort of become just a transactional way to "vend credentials" instead of focusing on learning. I also think that often homework and tests aren't great measurements; I was bad about doing homework in college the first time around, and it was frustrating to the professors because I was doing horrible grade-wise, but was extremely active during class and generally did fine on the tests. I was mostly learning the material (at least well enough to get an A on the exams), but because I wasn't submitting my assignments, they would be forced to give me a C or a D.
At the same time, I really haven't figured out a better system. I would be open alternative systems, but I think grades and grading and tests do the best job for the "average case".
Maybe I should email some of my old professors and apologize for the headaches I probably gave them...
Today you learned a lesson. You will need to add to your course work how GPT can get things wildly wrong. Just a simple example should not be too terribly hard to find. Then plant the seed 'if it has that wrong what else is it getting wrong? It is a good tool of getting the general idea but one you need to audit. This class is where you are going to learn the basics and know when this thing is wrong when you use it' That thing is not going away any time soon.
I actually did say something more or less exactly like that pretty early into the course.
Something to the effect of "ChatGPT is pretty cool, I like it, but it will just make stuff up sometimes in extremely convincing way. It's a tool that's dangerous to newbies and powerful to professionals, and I'm hoping this class will help you get closer to the latter".
Also, how GPT can get things right, and that if students use GPT (because it's the early days of the graphing calculator all over again but now for code) then the exercise is not "write the code" but "explain the conditions under which this code doesn't work, and why" because if you're taking a programming class it doesn't matter what writes the code, what matters is whether you understand that code. So if you're turning the exercise into a code audit by using GPT, you better damn well be able to explain what problems are left in the code it generated.
> You will need to add to your course work how GPT can get things wildly wrong. Just a simple example should not be too terribly hard to find. Then plant the seed 'if it has that wrong what else is it getting wrong?
People did the same thing for Wikipedia and it was never convincing. And they eventually lost the battle. It's going to require more than that to do the same for GPT.
I wonder if it would be worth it to give some sort of in person feedback to convey (a) you noticed the dramatic improvement (b) you were even worried they might be cheating (c) you decided to give them the benefit of the doubt (d) congratulations on their hard work paying off (e) but if they did cheat they got lucky and if they keep it up it's going to backfire on them one day.
Honestly, unless the grade for that class translates into taking something directly tangible from other students, and a "java" class sounds pretty intro, I wouldn't give any thought to it. Cramming for exams and then immediately forgetting everything you hastily memorized is a-okay even though you didn't learn anything either.
A random college class, to me, isn't the kind of thing where cheating really matters. The stakes are low, it's expected that most students will do well, and everyone gets the same degree at the end. If someone wants to sabotage their own education then fine, and if it doesn't bite them later good for them.
Yeah, I'll admit that the problem is somewhat on my end, but I find it very hard not to take these kinds of things personally. I agree that I shouldn't take them personally, but that's easier said than done.
I'm starting to see this come up in coding interviews.
I had one candidate that struggled MIGHTILY on a coding problem that was easy (kept running into syntax errors) but then pulled out an esoteric solution for the second problem in no time flat.
I can't prove that he was cheating but it very highly suggestive that he is.
I hate it, but I'm starting to think I have to do an AI detection question now. It's not hard, just ask someone to do something impossible. However, I don't like the fact that now I need to be "tricky". I've never believed in making coding challenges hard, I just want to see if you can write code.
I saw something similar in a recent interview. I brought in a candidate who did not meet all of the typical resume check boxes, but who had a long history of success on highly technical problems. These type of candidates can bring very new perspectives to our team, but they’re often a long shot in interviews.
In this case the candidate chose to use a language that I don’t know well and they said they weren’t especially familiar with either. Then they struggled a lot to even talk through the question and it was a grind to work through each line of code. I couldn’t offer much help, and they were very clearly unsure at each step.
But when we clicked “run” on the code, we were both surprised to see that it ran on the first try. Not only that, but it already worked for all of the “but what about…” follow up edge cases. And the candidate didn’t know why… “oh, I guess the language just works that way.”
I don’t know if they were cheating, but the entire thing was far outside any other interview experience I’ve ever had — and fits the pattern of what I’d expect an LLM to produce.
Seems like there's a silver lining here. It reinforces the idea that the answer to the problem is not necessarily the important part. When you can trivially provide a correct answer, it places more emphasis on the skill of being able to actually explain and work through the logic as a way to distinguish yourself from other candidates.
I'm a pretty good practical programmer, good at noticing potential edge cases, writing code that's changeable without being overcomplicated. This based on feedback across years of professional work.
I also have adhd and literally brain damage and have worked professionally in about a dozen different programming languages. I can't write a for loop or declare a static method or w/e in any of them without googling the syntax or using my editor's hints & autocomplete.
They had google access and editor hints. Further, I start every interview with "please, ask any questions, we aren't trying to trick you we just want to see if you can code." And I mean it. If someone has a minor problem with remembering syntax or how to declare it I don't really care and will happily let them know how to do that.
Heck, I've even had interactions where a candidate was like "I think there's a method that does x for this" in a language I was unfamiliar with, so I googled up what x was and shared it with them mid interview. "Oh yeah, looks like this is what does x for your language".
There was also some pretty weird behaviors with the camera/screen that caused them to need to touch it fairly frequently (I'm guessing to take over keyboard control from the AI software they were using).
What they were doing, repeatedly, though the interview was writing
`a[i]` then having to change it to `a.get(i)` because they were working with a `List` in java. I get maybe doing that once or twice, but in the course of the interview they did it every single time they needed to pull something out of the list. (and each time needed to touch their screen and take a few minutes to correct it). This is why I strongly suspected cheating. It extended a question that normally takes 15 minutes to 35 minutes. Then the question I ask that usually takes ~45 minutes they completed in 15 with an optimal solution that relies on a data structure I literally only learned about because the interview hints for the question are like "Hey, java has this data structure that makes everything easier".
Yeah I get it. This is one of those things that's really hard to convincingly describe online because each individual thing is explainable eg I will write a[i] every time and my format-on-save will change it to List.get(a, i) every newline or whatever.
But you were there and I wasn't and if the overall situation seemed suspicious to you that's evidence at least as strong as any specific thing. Intuition can certainly pick up on weird behavior even if it's hard to explain how after the fact.
This may be an unpopular point of view, but I think in situations like this, you should just pass on them. They are applying to tens of jobs and your company is interviewing tens of candidates, so there's no real reason for either of you to continue this process if they struggled on the easy problem and had questionable reasoning on the hard problem. However, if the candidate could very clearly explain the esoteric solution, there is a good chance that it's just similar to something she has done before and you should not be suspicious.
I am assuming that you had a compiler or IDE in this situation providing a useful error message. If you were whiteboard coding and you were telling the candidate "you have a syntax error but I won't tell you where" (like several folks at a former employer of mine were fond of), you may be the problem, not the candidate.
> I am assuming that you had a compiler or IDE in this situation providing a useful error message.
Yup. I prefer to give candidates compilers/IDEs and even access to google if they ask about it. Like I said, I'm not trying to trick them or anything, I just want to know if they can code.
> If you were whiteboard coding and you were telling the candidate "you have a syntax error but I won't tell you where"
I'd hate being on the receiving end of that and would never pull that sort of stunt. The software we use allows both sides to edit the code and I pretty regularly will go in and silently fix syntax errors for a candidate so as not to let a compile/refresh loop get in the way of actually solving the problem.
Like, what does "You have a syntax error" even prove? I'd let you write pure pseudo code so long as it makes sense.
I've been writing code for pay for more than 20 years and will forget basic syntax and keywords for a language I wrote hundreds of lines in yesterday without other code to crib off of, IDE assistance, and/or reference material (google, whatever). Like, I'll forget if it's "else if" or "elsif" or "elseif" or "elif" or whatever. Or I'll mix up how to access hash/list members. How to designate a constructor. Stuff like that.
Universities are going to have to adjust. That's really all there is to it. It's not that hard for an exam, since you can put them in a properly designed classroom with human monitors. Homework? Projects? Much more difficult.
The current system worked in 1960. Not in 2023, and probably not since the late 1990s, when technology started to make our evaluation processes obsolete.
Exams used to be a larger part of school grades. In my area, they were devalued and the reason that was given was that exams favored males over females.
Can I ask what’s the point of testing students in a scenario where they don’t have access to AI tools? As a software developer I use these tools daily, and if I or my colleagues stopped our productivity would suffer - it sounds like the same genre of thinking that leads to having students write out code by hand because using an IDE is “cheating”.
If it would be possible to take a bottom-tier CS student and turn them into a decent programmer using AI tools, the hiring landscape would be /very/ different. These tools aren’t magic, and to me it sounds like the tests are failing at measuring the students competence if they can so easily be gamed by using an AI/old fashioned googling.
Maybe the solution should be to move towards a style of exam/grading that actually measures the competence of the student in a situation closer to what a professional developer will be in, rather than an old fashioned artificial exam setting?
An evaluation for a course should measure how much of the subjects of the course the students learned. It's not a job success simulator, nor a measure of unrelated skills.
This follows the same logic as banning calculators when teaching people basic arithmetic. When something is a replacement for your learning, it should be banned, and when it is an augmentation, it should be encouraged.
You may think it's ridiculous, but many people today can't calculate a 20% tip or add two prices together without a calculator because the constant use of calculators has caused their arithmetic skills to atrophy. This seems like a silly gripe, except a lot of other skills are built around these basic ones: for arithmetic, the ability to estimate the cost or time taken to do something is all based on tricks you learn when you are trying to learn mental math. Society is not worse off for this (enough of us still know how to do these things), but many people are poorer, both intellectually and monetarily, due to a lack of arithmetic skills. And no, this lost knowledge of arithmetic is not replaced by a knowledge of higher math - it tends to come with a fear of it.
The same applies with text-generating AI. In terms of writing, if you don't learn to write dumb essays about books, skills like learning to construct an argument are much harder to pick up. For people writing code, learning to slog through writing and debugging a doubly linked list (something ChatGPT can reliably generate for you today) leads you to later being able to slog through debugging B-trees or lock-free queues (which ChatGPT definitely cannot write for you).
I think there is a very compelling argument along these lines for low-level courses to ban AI tools. However, higher-level courses probably should allow students to add them to their repertoire, where they are an aid and not a crutch. This follows how mathematicians and engineers learn to use calculators and computer algebra tools, which seems to work well.
Earlier this week, I spent several minutes talking one of my co-workers through how I 'did math in my head' to figure out my half of the 20% tip we were leaving for lunch. Even something as rudimentary as moving the decimal place over one and doubling the result seemed like wizardry to them.
They aren't dumb. They've literally just never thought one second past reaching for a calculator. Which is kind of scary, because it means they have no way of sanity checking any numbers they come up with.
It's a good thing that the skills involved in making fire by hand don't transfer to disciplines that matter. The same cannot be said for arithmetic and writing.
I know a person who does interviews for programming positions who asks "what is 20% of 20,000?" The ones who are flummoxed by it are no hires. So are the ones who pull up a calculator app on their phone.
A person who knows how to code without access to AI tools means when they have access to AI tools their productivity would increase but they will also be able to point out mistakes, if any, in the generated code.
A person who does not know how to code without access to AI tools means they will consistently push bugs generated by AI, affecting the teams productivity, if any.
The exam questions are likely simpler than an entire application would be; using AI to write bubble sort is trivial but getting it to understand the full context of your application is much less so.
ChatGPT can solve most LeetCode questions, but I don't think most people here are ready to replace their junior developers with it.
I at least partially share this sentiment for hiring interviews. In that setting, the goal of any kind of coding exercise is to evaluate a persons coding skills in a real-world environment. For something like that, it does not make much sense to hobble the developer by artificially limiting their access to coding resources that are normally employed while on the job.
However, I feel like an academic exam is a bit different. The goal of these exams is to evaluate what students have _learned_. Unless the class was about how to find helpful code examples on SO, it does not make much sense to allow AI/internet-searching during the exam.
Maybe the solution should be to move towards a
style of exam/grading that actually measures the
competence of the student in a situation closer to
what a professional developer will be in, rather than
an old fashioned artificial exam setting?
Most "real world" software development work involves understanding existing code and/or choosing solutions.
AI can help you write a linked list, but can it help you know that you need a linked list vs. a queue or a binary tree? Can it help you to make more architecture-y decisions like deciding whether you need a document database, a traditional SQL RDBMS, or maybe something else?
I'm not in a teaching role but those are the kinds of things I'd like to see examinations cover -- what tool or data structure are you choosing, and why is it the appropriate tool for this specific case?
Or, similarly, can the student understand existing code and fix bugs or make performance improvements?
Bottom line, though: professors should just absolutely accept the reality of AI. Assume all students are using AI tools. Actively mandate or at least encourage their use in order to ensure a level playing field.
> Can I ask what’s the point of testing students in a scenario where they don’t have access to AI tools?
It's not just the issue of "what if they didn't have access to ChatGPT?"
Just as an example: ChatGPT is literally the world's biggest bullshitter. It vacuums up vast stores of "data" (note: NOT "knowledge"), and just pulls out words that it thinks could likely follow from previous words given a context.
Quite often is produces silly, or even dangerously wrong, answers. You need to be pretty well skilled in the art to catch it at its bullshit.
It's just like when you hire assistants to do something critical for you. You have to be able to verify their work, or you have to be able to trust them blindly.
Think of it this way. I use a calculator to do less trivial arithmetic. However, knowing how to multiply means I understand what multiplication is rather than just a number that inexplicably appears on the calculator screen.
I've known engineers who did not understand the analytical tools they were using. They misused them constantly.
This is like asking "what's the point of not allowing students to use Google during their Maths exams" or "why bother with English exams when Grammarly exists".
The point is to gain some knowledge yourself. Obviously you'll use new tools and methods later when you eventually get a job, but the point of most courses is to teach you the concepts, not how to command the IDE. Whether you decide to hand-craft assembly code or become professional a Copilot suggestion approver doesn't really matter.
> Can I ask what’s the point of testing students in a scenario where they don’t have access to AI tools?
It's a question of: Are you testing for understanding or ability? Both have their merits, but often the goal is the former. If it were a project, it is generally the latter.
I'm old: when I went to school we weren't allowed calculators "because there'll be a situation where you need to do basic arithmetic and don't have a calculator". I occasionally laugh at this while firing up the calculator app on my phone. We'd have been better off if they'd taught us sign language rather than arithmetic.
Our education system is going to have to adjust to the new reality. Setting an essay task is now the equivalent of learning the times table - utterly redundant (unless you plan on a career writing essays or doing maths).
The most 'fair' math classes (specifically, I'm thinking of algebra and calculus type courses) I have been in, allowed a good in-between. Basically, you were totally allowed to use a Calculator, even one with a built-in CAS, but you -had- to show the in-between steps if you did so on a test.
This was pretty useful in it's own right, as you at least had a way to check your work, and frankly it was extremely useful for helping me solidify knowledge in those fields. [0]
Of course, AI muddles this, since it can explain the steps for you.
That may be the strawman, but that's not the real reason why people ban calculators when you learn arithmetic. The reason is because learning to manipulate numbers in your head abstracts to other things, like learning to estimate and learning to manipulate equations.
Higher-level math classes usually encourage calculators, and graduate-level math classes will often allow Matlab or Mathematica, as long as you can write a short sentence explaining what the computer algebra system is doing. Where I went to school, this was colloquially called "proof by Steven (Wolfram)."
It is a strawman. But I wonder how much better we'd be able to do those other things if we'd been taught to do them with calculators as a tool instead of being forced to do the basic arithmetic in our heads.
If you look at the natural experiment on that in society right now, much worse.
Many people today do math only with a calculator and correspondingly lack any sort of numerical reasoning skills whatsoever, despite having learned both mental math and numerical reasoning in school. That growth of innumeracy coincides with the rise of calculators and computers who do it for you.
Also, students are taught in more advanced classes to use calculators (and later computer algebra systems) as a tool. This comes after they have learned the basics, not as a replacement for the basics.
Writing (hello ChatGPT) and math are like muscles: use it or lose it. If you can't do the basic versions, you can't do the harder stuff either.
I'm fascinated by linguistics, and once attempted a degree in Deaf Studies, so yes on the sign language (it's not a language, by the way: it's a class of languages, alongside spoken language, which is another class of languages), but the idea that basic arithmetic is worthless is completely alien to me.
Not worthless, obviously, I was maybe being a tad overblown ;)
But I do think that as we move forward with technological tools for this kind of mental task, that we should embrace them in our education system. If we have to teach every generation from first principles then we won't get them comfortable with the actual tech they'll be using in their lives.
thanks for the clarification on sign language :) One of those things on my bucket list that I keep meaning to learn but never get started on.
So the purpose of academia is teach pupils a subject, that they can the use in real life.
When you're solving the same problems in real life you do have access to all those tools that you banned. Because in real life, if you manage to solve a problem it doesn't matter that much how you solved it.
Sure, if you just Google things and always solve your problems by copy pasting. Your solutions will lack depth, and at some point this will catch up to you.
My point is, cheating is primarily cheating on yourself, and it will catch up to you.
I think this is the framing you should have with regards to cheating.
> Technically, it's possible that he just studied very very hard and his code and grammar improved.
As an early procrastinator (rehabilitated perhaps?) I've had a history of doing very well on finals through cramming.
In your student's case it might very well be more likely they are cheating, but those unlikely but possible students who study extra hard to recover their grades for a final deserve the benefit of the doubt.
How unlikely can academic recovery get? I spent an entire semester completely lost in a second level Macroeconomics course...
... I was able to read and write chapter summaries of a semester and a half of content (about 15 chapters, corresponding to the first level and the second level courses) in a single weekend and aced the exam, with the highest marks in the class.
That was the hardest I've ever crammed in my life and I had trouble speaking in the hours that followed the study session.
I later got a recommendation letter from that prof, totally worth it.
Some students cheat. As an adjunct instructor at the university level, I had similar experiences.
You have to both adjust your expectations and make a significant part of your grading use an in-person one-on-one oral exam or project walkthrough.
It is a huge commitment of time - most professors I know seem to have quietly accepted cheating will be be rewarded because they don't have enough time to verify student performance.
It seems we want to turn universities into McDonalds where we push high numbers, but also pretend they are elite institutions that we can charge premium prices for. This is not going to work well in an age of ever increasing automated intelligence...
Well, lest of course the students sit under the watchful eye of AI in the near future
> I made it abundantly clear that they cannot use any kind of AI assistance for their work
You're swimming against the riptide. If I were teaching coding today, I'd allow, even encourage, use of AI, with some ground rules. Students using AI would have to show their work (i.e. the conversation with the machine).
The content of the prompts and in particular follow-up questions, can demonstrate competence. E.g. situations in which the student spotted that something was wrong in the generated code and made an intelligent hint to the machine to fix it.
Students should be prepared for tomorrow's world, not yesterday's.
By that logic wouldn’t it be ok for grade schoolers to use calculators when they are learning basic math? Sure you can teach someone how to get an answer, but have you actually given them an understanding of the material to be built upon?
Kids should be using calculators in kindergarten. You can discover a lot of curious things about the numbers with calculators. Like when you multiply by 5 and keep doing it, the last two digits are always 25, and the hundreds digit bounces between 1 and 6. You can discover that without a calculator, but it's more time consuming.
If we want to ensure kids can do arithmetic without calculators, we can have exams in which there are no calculators.
You can give them homework problems in which they have to fill in all the work steps of multiplication and long division, e.g.:
______
17 | 2091
-__
__
-__
__
__
If you want to ensure that the students know and follow a certain process in obtaining an answer, you can't just test the production of an answer. You have to white box it.
Kids should know time tables to about 12. Beyond that, you can test whether they know the structure of long multiplication and division and such.
Yes, he is unethical. He didn't just catch students cheating. He actively sabotaged the cheaters. He says he placed a (metaphorical) camera to catch the cheaters, but if he didn't pay attention to it they would have failed anyway because he planted wrong answers for them to find. Thinking about it now, he could have just graded them with the wrong answers and left it at that (a different ethical question), but it seems he was determined to "get" them, call them out, and punish.
To be clear, I'm not supporting the cheaters. There are two wrongs here. "Am I the unethical one?" should instead read "am I unethical?" to avoid any discussion about which is worse. Suppose I went on the site and uploaded wrong answers, thereby clearly sabotaging cheaters. Clearly that would have negative consequences for those students and one might ask why would I do that.
Exactly. Whether he intended to or not, he - HIMSELF - put two different, equally authorially-dominant, sets of answers into the wild and expected students to pick the best one in what can only be described as an unauthorized (afaik?) social experiment.
Fuck the students who thought "wait, this seems like...way wrong. I feel like he said the exact opposite in class...? But, I mean, this IS the test that I'm looking at, so I guess I'm just misremembering...", is the apparent sentiment. At least he caught the obvious cheaters, right?
Those latter students would have been taught in high school not to trust sources without some sort of verification. Trusting that a non-verified, random upload to a cheat/study site is accurate shows lack of critical thinking skills, at best.
say what? One is on the official school site, in the lectures and notes from class. Second one is an anonymous upload at the weird unofficial site which does not even have university's branding.
If the students can't tell those two apart, they are in deep trouble and should not be in college.
I'm inclined to agree. If you're actively putting out misinformation, that's always bad. It feels a bit "entrapment"-ey.
Sort of tangential, but I have always thought teachers/professors who put "trick" questions on tests to be sort of assholes. It's fine if it's an extra credit thing and isn't going to take away from the final grade, but when you write a question that literally everyone in the class gets wrong, I think that says a lot more about your communication or teaching ability than the students.
Why is it unethical to be determined to call out and punish cheaters? When I was in college, we had like an academic honesty pledge that made it clear we would get kicked out of school if they caught us cheating. It's not uncommon (or, I think, unethical) to punish cheating in academic programs.
He sabotaged the cheaters. It's a bit different than just catching them and issuing the usual punishment. I think there is room for debate around the distinction, but from an ethical point of view one should avoid gray areas, and this guy actually teaches ethics - to this class.
He did not sabotage them - the usual punishment is at minimum a zero on the relevant assignment/test/etc., and frequently extends to outright failing out of the class. Scoring worse on the test due to their cheating is irrelevant as it'll be reduced to a zero.
From an ethical point of view, he has done these students, the university, and the world at large a huge favor.
This is unethical, but not for the reasons given. It's unethical because it favors the pre-internet status quo of cheating within one's social circle. It was well known at my university that fraternities and sororities kept libraries of old exams. A policy that catches only internet cheaters is a policy that advantages socially connected cheaters over individual cheaters.
Our uni's CS students kept an online catalogue of past exams/answers/cheat-sheets so it was somewhat democratic for it's time :) Tho I guess you had to know it exists to find it first.
> It's unethical because it favors the pre-internet status quo
Don't agree. You're saying that if I can't catch all the cheaters, then I can never call anyone out for cheating. That's a licence to cheat.
Cheating is bad, for the same reason corruption is bad; once it's accepted that it's allowed, then anyone who doesn't cheat/take and give bribes, then everyone who plays by the rules is at a disadvantage. Corruption is a disease that spreads if you don't stamp it out.
Also: cheating on tests is an insult to the intelligence of the examiner. The cheaters are laughing at him. If you don't call out cheaters, then the whole idea of getting a graded result becomes meaningless; you might as well simply give every student that actually paid their fees an 'A', and do without exams.
BTW, I think it's irrelevant that it's an ethics exam. You don't study Ethics to become more ethical. You study it to learn about the philosophy of ethics.
I don't think the honeypot is necessarily unethical, but giving a take-home multiple choice final exam for a philosophy course seems like several levels of bad pedagogy.
Agree. I don't know what a multiple-choice philosophy exam even looks like. I did a Philosophy BA, and all the exams and assignments were essays. The non-exam assignments were all discussed in seminars.
From my experience working at a university (in the foreign language department), I found a number of students who put a large effort into "getting out of work". I was astonished at the number who put more effort into avoiding work than it would have taken to get an A.
If I were in this professor's case, I'd just mark the answers wrong, and in the future upload more wrong answers. The students who use these sorts of online dumps aren't the ones who study and will beg at the end of the semester for some sort of extra-credit. If it were necessary, perhaps have a second gradebook where the number of "exactly the same wrong answer as the bait" were kept.
> From my experience working at a university (in the foreign language department), I found a number of students who put a large effort into "getting out of work". I was astonished at the number who put more effort into avoiding work than it would have taken to get an A.
It's funny when that happens, I used to do this in 7th or 8th grade when we were doing home exercises in touch typing. All the typing tasks were given from a textbook that was the same for everyone in the country, for a few years at least.
Over 2 years of classes, I had probably spent more time scouring the internet for the solution rather than type it myself. The search was however always more entertaining than the exercise and taught me how to use the web. And I still learned how to touch type.
While I've always had a negative outlook on the modern school system due to my own experiences, I fail to see how the answer to the title could be anything but "yes".
I've seen this many times before, where a teacher seems to fail to realize that their students don't just have their own exam to prepare for; they have to prepare for many other exams at the same time, all the while struggling to balance their study time with their responsibilites at home, their social life and possibly their part-time job at the same time.
So when an answer sheet is just readily available online, there aren't many students who wouldn't choose to spend a few hours memorizing the answers so they have a little more breathing room for other (possibly more difficult) exams.
The statements about how this teacher apparently feels oh-so stressful about this situation that he purposefully created himself, all the while dismissing any and all critique from people because they aren't "teachers of any kind" feels very childish and leaves a very bad taste in my mouth.
Ok, let's not inconvenience the students anymore with studying then since they're so busy. We should just award them a degree after 4 years of being in the unversity's register.
My point is that expecting time-pressed students to ignore freely available answer sheets is like expecting a hungry horse to ignore a carrot dangling in front of them.
There is nothing wrong with removing their ability to cheat, but purposefully uploading answer sheets and then getting angry that students made use of them isn't. In fact, it's not just wrong: it's ethically wrong.
I think the "anger" is merited since the students (1) cheated when they were clearly told not to and (2) marked answers that were "obviously wrong" which implies that not even a modicum of effort was invested in demonstrating knowledge.
The "busy" argument is a poor one. We're all busy. Part of gaining an education is learning how to manage your time. As a professor myself, I know for a fact that most students manage their time poorly, yet many students will still pull the "busy" argument when it simply doesn't apply. Rather, just admit to procrastinating. Either way, the outcome is the same (poor performance).
To sum up my sentiments to cheaters... "Play stupid games, win stupid prizes."
Honestly, it's kinda wild to me that the final(!) is just multiple choice questions, and that it's similar enough to the one last year that you can cheat by looking at previous exams? That's just a badly designed course, feels like the students are the ones getting scammed here...
I have done 2 ethics courses during my education at a Dutch 'Hogeschool' (honestly not sure how this maps to US education wikipedia says 'Vocational university'). I did a specific design ethics course and a broader ethics course as part of a philosophy minor, and in both of them you had to write papers or apply the things you learned to a case study. There were some little tests with multiple choice, but they often had additional questions where you had to explain your reasoning.
Maybe there is a language difference here, but I would expect something more involved from a course given by a professor at a university, or is this a course for people who are in high school or something.
Heh, and the professor is polluting the info space too... That is if other parties are taking this fake test out of it's multiple choice format and using the answers as data for other sources. You know, in the same ways bots steal information from recipes and mix them on pages to try to get ad hits. You start building chains of misinformation based on unethical behavior.
I'd argue that the unethical bit is knowingly publishing incorrect information, especially in an educational setting. If there's a chance that someone learned deliberately false information that they will then attempt to apply in a practical setting, then you're effectively sabotaging them. He did say that it was obviously wrong if you had been paying attention, but what if you missed a class or two?
> I ran a binomial analysis and found the likelihood that someone whose answers matched on 19 out of the 45 planted questions had about a 1:100 chance of doing so
This is just bad math. Take a question (A), there are 5 possible answers, using his analysis the probability of a student picking the same answer as the cheating answer is 1 in 5. But let's say the question is hard, and of the 5 possible answers, 2 are highly plausible, so plausible in fact that the students always go for 1 of the 2. Now, if the cheating answer is one of the plausible answers the probability is low, but if it's one of the plausible answers, then it's high. And more specifically, if the cheat answer is correct - what's the probability the student got it right? Well what you should be doing is take the other 55 non-cheating answers, calculate the probaility of correctness and then use that as the probability. The "1/100" threshold is overwhelmingly determine by how he selected the answers on the leaked answer sheet, and you can't say it just average out with such a small sample.
Modelling the whole thing as random choosing is just sloppy maths.
No it's not, he says some of the answers were obviously wrong, but he did the binomial analysis on 45 questions. If you plant 45 answers in a 80 answer test and they're mostly obviously wrong then the students who cheat are going to fail either way, so it can't really be true that they were wrong. And if the answers were very obviously wrong then the probability of choosing the wrong answer for each would be much higher than assuming a 20% chance of matching the fake answer. So the whole 1in 100 is highly questionable.
To put it another way, let's say he planted only wrong answers, and we have an idiot student that always gets the questions wrong. Now the probability of matching the planted answer has clearly jumped, right?
On the flip side, if the planted answers are right, and in general you expect your class to pick correct answers (ie, they're going to pass) then the probability they pick the same as the cheat answer could be as high as 90%
It isn't. "Most of these answers were not just wrong, but obviously wrong", according to the teacher. The chances of picking the worst possible answer is lower than 0.2. If the .01 threshold is fair? It isn't if you assume that someone tried to pass the test by filling out the whole thing at random. If the students tried to pass by choosing the correct answer, >= 19 obviously wrong answers out of 45 seems fair enough.
I'm not saying the conclusion they're cheating is wrong - in fact the threshold is way to generous, but fundamentally the statistical model he's picked just isn't a good fit.
What model would you propose? You don't know the likelihood in advance, do you?
The only thing that could have been added (that I can think of!) is that the likelihood of cheating is not memoryless, as the binomial process assumes, but rather that a cheater would not copy a few answers, but very many. So the chance that a cheater picks the bad answer is > 0.2, the chance that someone who doesn't know the answer = 0.2, and the chance that someone who does < 0.2.
But given the fact that we don't know the correct probabilities, nor the scale of cheating, and not even how often people take a random guess, makes the model a decent starting point, IMO, for getting a first estimate.
After all, the teacher is not researching stochastic models of cheating, but just grading papers.
I'm not sure there's a completely thorough way of proving cheating on a single measure of a relatively short test. If I wanted something more thorough I'd probably compare the mean score on the non-cheat questions vs the score on the cheat questions though. Let's say you provide the correct answers on the "leaked" test. If the score on the leaked questions is significantly higher than the score on the unseen questions then you can confidently say there was some cheating. At that point you'd probably interview the candidates on their tests rather than throw accusations.
Add me to the list of people who see nothing wrong with this. If he didn't encourage or incentivise his students to cheat in any way (and in fact actively warned them against it) then the idea that this was some sort of entrapment is laughable.
The only room for ambiguity I can see is the arbitrary 1% threshold. If there was a student just over the line then it's plausible that they were honest but unlucky here. Given the consequences of being tagged as a cheater both for this exam and beyond I would want much lower odds of a false positive and I'd certainly feel obliged to give a borderline case the benefit of the doubt.
But if it was clearly understood that this kind of behaviour constituted cheating and innocence is a billion to one shot? Throw the book at them. Anything less is unfair to every student who didn't cheat.
Right. He's made this harder for himself than he needs it to be.
The honeypot is fine. If it were me I'd warn the class that I've seeded the internet with bullshit answer keys, and while I'll never be able to prove you used one of them, you'll probably fail if you do. And leave it at that.
Instead he's guessing a threshold, and can't really be sure if someone that insists they didn't cheat is lying or not, with high stakes. It's a bad situation for all involved.
I work in a small university with local students that are aggressively average, if not a bit below that. Especially when it comes to writing a thesis, our expectations are extremely low, and the success rate is abysmal (something like 30% of the students in a year will actually defend their theses). If a student can reach something like, say, 10 people to do a usability test, we are extremely happy and will pass them with a smile on our faces.
This year one of my students claimed to have done a user experience test with 30+ people, all of which came in person to his house during a period of 4 days. This is extremely unlikely, not to say absurdly unrealistic. If he had done it via Zoom I might maybe believe, but in person? Sorry, but no. I asked him what did his parents think of it, and magically they were both travelling that specific week. Then I asked him to scan and send me the signed consent forms for each participant; he promptly said "ok, coming!" then about 8h later I got a bunch of signed scans. Not sure what to do anymore, I guess he'll have his thesis.
> This year one of my students claimed to have done a user experience test with 30+ people, all of which came in person to his house during a period of 4 days. This is extremely unlikely, not to say absurdly unrealistic.
I'm not familiar with conducting a user experience test. Why is this so unbelievable? (Also, the rest of the story is kinda hilarious; there will definitely need to be some standards and process changes in academia.)
depends on the situation but generally most people aren't that enthusiastic to be your UX guinea pigs, and it isn't that much fun to administer the same test so many times in a single day (think of how bored your optometrist sounds as they ask you which lens is better). I worked at a shop that had a target of 3-5 users for any given UX testing because usually it starts to get repetitive after that - you tend to hit the diminishing returns portion of the curve.
So if you assume that the UX tests were short (20-30 minutes) you're still talking ~15 total hours spent on UX testing, or two full work days. That would be a surprising amount of time.
As someone else said, it's conceivable that they ran very short studies on a large number of people in the context of having a party or a couple of small gatherings, but it does seem unlikely overall.
Just to recap: very generously, a college student with money to spend (that might sound like a joke but I consider fortunate circumstances) convincing people that it’s worth their time to sit through a usability test actually going through with it “for the lulz”. The part about a person being willing to do a bunch of work to avoid doing a bunch of work makes sense (typical “mental gymnastics”) but it kinda falls apart at 30+ real people legitimately agreeing. It’s ironic that a single-digit amount is just as useful; that’s even a more believable lie.
(And this real-life scenario where every request is answered with “Coming right up!” and a convincing “successful” completion of the request just cracks me up. My sympathies go out to the people to whom this is a genuine problem.)
You have to get people to actually come to your house voluntarily in the middle of the week to sit down and do some boring stuff in your computer. And it all just clicked with everyone’s schedules in four days. I usually can’t get people to even answer a 5-minute online form, let alone get them to drive to my house. Maybe he’s just popular (which he isn’t, by the way; and a confessed previous cheater.)
> This year one of my students claimed to have done a user experience test with 30+ people, all of which came in person to his house during a period of 4 days.
Is it possible that maybe he thrown a big party (or a series of parties) and asked the folks there to try his stuff? Not sure what is the length and depth of the usability test in question, but if it is a short questionnaire that might work? But I see where your suspicion is coming from.
This surprisingly remind me of the last to birthday parties I organized for myself. Unexpectedly I managed to get much more people together than I expected. Simply by actually trying to ask as many people as possible. I was really happy about this. Both times.
Maybe that student was doing the same. Simply asking a lot of people. And other studends usually do not. I find it really hard to overcome my own shyness in this regard. And I wouldn't consider myself particularly shy. I assume a lot of people simply do not try asking enough people in order to get a decent amount of them to join.
Of course that guy might be simply cheating, but I just wanted to share the story.
In middle school, I put a lot of time into a paper for a history class, at least three times what anyone else did (probably more). I got a B for the class. The teacher said it felt like the paper was copied, because the writing was "too professional" and that middle school students don't write like that. The teacher was wrong. I just happened to have worked really hard because the topic was interesting to me.
Are there rules about investigation? In general, if someone makes a claim, it invites testing.
The consent forms are likely friends, so calling to ask questions about the experience should give you an idea of what to believe.
The outcome can be your decision. Meet with the student and tell them their degree is in jeopardy and use it as a teaching lesson, or prove to them that they can get away with believable lies as long as no one has the courage to catch them. Your colleagues should have some insights, as well.
375 comments
[ 1.7 ms ] story [ 296 ms ] thread> The system is broken
This is truer than it sounds. The very design of Examinations itself is broken in most places. It shouldn't be possible by design to cheat in my opinion.
"Open-book" exams are harder to set, but I think the whole cheating-prevention infrastructure is harder, costlier, and pointless.
It is definitely possible to design assessments on which cheating is difficult. For example, oral examinations or personalized per-student projects or exams. The problem is that this style of personalized assessment fundamentally does not scale past a few dozen students in a classroom.
You want a classroom that small, it's going to cost you. Just instructor salaries for would run each student 10-30k per year, and that's before paying for infrastructure (classrooms, tech, offices) and (admittedly not always useful) administration.
Of course, yes, nothing's perfect. But Open book exams are far better bargain than what the most popular method of exams is as of today.
Cheating was solved with a pretty straightforward approach: don't make it trivial to cheat (re-using test questions, question-bank multiple choice tests, etc) to keep the honest students honest, then trust the students. If someone's caught violating that trust, send them to the business school. (I don't know what they did with cheaters in the business school. I assume promote them)
That may be university-by-university thing. I briefly managed an academic department at an old, highly-ranked university. Professors regularly caught cheaters and there was never any question that they would fail the course. The only question was whether the incident was mild enough that they would be permitted to try again next semester or severe enough to leave the university.
The administration supported us 100% as this was seen as essential to defending the value of the degree conferred on the other students. Without some kind of standards we'd eventually become just another degree mill.
What surprised me most was how many students who never finished the degree (not just cheating -- some just got job offers and decided not to stick around) still listed the degree on their resume/CV. The problem is that many companies now call the school to check as part of their hiring process, and the school would inform them that they had enrolled but had not been granted the degree. In retrospect I wish I had instituted a policy of letting the ex-student know in those cases, because I'm not sure if hiring companies tell them and I wonder how many people think they're just unlucky or something.
> MIT Admissions Dean Resigns After Fake Degrees Come to Light
https://www.thecrimson.com/article/2007/4/26/mit-admissions-...
I routinely got assignments handed-in that students had evidently copied from one-another. More than half of the students were handing in copied work. In at least one case, they hadn't even botherd to change the name at the top. Usually they had the sense to change variable- and function-names, but not always.
As a newbie lecturer, I asked my colleagues what to do. They said: "You can fail them. You'll be accused of racism (most of the students were brown-skinned). They will appeal; you'll then have to sit on exam boards through the summer, which is unpaid for a visting P/T lecturer. There's a good chance the school will overrule you, because these are paying overseas students."
"Or you can tell them that you've noticed the 'sharing' that's been going on; that collaboration and sharing is encouraged, but that they must never do it in marked assignments."
I adopted the latter course of action.
Being a part-time visiting lecturer is a crap job.
If people are selling answers that's a different story, but that's what in person exams are for.
I helped a buddy of mine out on one of the projects by basically writing it for him. I re-wrote my project in a completely different style.
We both got 100%.
If you are going to cheat, do it right and actually show some imagination and creativity.
Like re-code your project in another language and then back again.
Cannot find for the life of me what could be possibly unethical about this. If anything, it’s very, very educational.
- Publishing wrong answers to a previous exam could confuse students who were simply looking up old exams to study.
- Identifying 'cheaters' as anyone who had a less than 1% chance of arriving at the answers randomly. This is wrong for 2 reasons, one is that they aren't answering randomly the other is that even assuming they are answering randomly you'd falsely accuse at least 1 student of wrongdoing on average.
- Not sure if a teacher has a moral duty to make good tests, but if they do then reusing multiple choice questions on a complex topic like ethics isn't ideal.
In any decent institution which does exams, an example exam sheet with questions similar to what you’re going to get, or even one of actual old exams, is usually published for just this purpose.
Sniffing around the sites the express (or commonly agreed upon) purpose of which is enablement of cheating, on the other hand, reveals the intent.
So, nope.
I'm not even sure what a website designed to enable cheating would look like to be honest, but that's because I have no expectation that I'd be able to access the internet during an exam, and because I consider any study of related materials fair game before the exam begins. You'd have to get an answer sheet of the actual exam up front before I start to consider it iffy.
I've heard of some schools where supposedly the students take the honor code very seriously.
> I tell all my students what will happen if I catch them cheating
How do those two statements go together?
Of course this is irrelevant, as the p<0.01 test gives an up to 62%[0] chance that someone in his class of 96 students could be falsely accused of cheating, and is unethical in itself. Apparently one student managed to be right on the threshold, which the professor has chosen to interpret as "the student possibly cheating" rather than "me possibly ruining my student's academic life for no reason by choosing a test with lots of false positives and applying it to a class of 96 students".
[0] 1-(0.99^96), and though the real chance is probably a lot lower if the questions are obviously wrong, it's still a stupid threshold to choose
And what is definitely cheating is to search for past exams while you have the current exam in hand and start copying down answers.
It was an interesting grey spot since most professors were all for using old exams to study, but some weren't (I don't think they liked writing new exams every semester, but I wouldn't either). Some would give threats for giving your exams to this university recognized academic club, but they'd black out names upon receiving them so there wouldn't be any proof that it was you (at least easy proof).
It was an interesting case there but most staff were all for it. Making that old data available to future students, but in a more restricted way seemed fine to me, and it absolutely helped me study, but I wonder what others would think of that situation.
That 40 is reported suspected, by one individual. The same individual claims that only 2/3 of those were admitted at time of writing.
Also, suspicion was by only a 1:100 coincidence probability standard of evidence, and by some imperfect metric. IMHO, that threshold would be too low to "prove" guilt in such a potentially serious matter (negative mark on student record, reputational damage among college social and professional networking peers, and potentially including suspension or expulsion).
...only? In a sample size of 96, what's the value in analysis beyond that level?
No, this seems pretty solid to me. The math to do this is routine freshman statistics stuff, something every practicing scientist knows. The assumptions just require that wrong answers be roughly evenly distributed across the choices (e.g. you could construct a test where everyone who was wrong would be led to choice C and rarely B or D, but that's a little pathological; and regardless it's something that would be evident in the data set and seems not to have been).
I mean, standards vary but I'll bet in most jurisdictions 100:1 odds count as "beyond a reasonable doubt" for jury instructions in criminal trials. At the very least you'd bring the trial, which is what happened here. If I know there's only a 1% chance I'm wrong, it's absolutely valid for me to accuse you of cheating.
IANAL, but I sure hope that 100:1 chance, standing alone, doesn't meet the bar for "beyond a reasonable doubt".
To me, a 1% chance of an accusation being false is definitionally "reasonable doubt".
https://en.wikipedia.org/wiki/Reasonable_doubt
https://en.wikipedia.org/wiki/Blackstone%27s_ratio
Suspected at 1:100, certainly. Quietly confronted and questioned at 1:100, sure. Convicted at 1:100, absolutely not.
I'm not sure this is the best analogy, but imagine that, for every 100 people in society, 1 of them is wrongly convicted, at random. Further imagine that this lottery happens repeatedly, so that even more than 1 in 100 end up getting hit. That sounds like a miserable society in which to live.
Why would you use 0.25 instead of 0.2? I guess it would make sense if you only looked at wrong answers - that is, you wouldn't be asking "what's the probability of your answers matching those on the fake test" but rather "what's the probability of your wrong answers being wrong because you used the fake test". Since you are only looking at wrong answers, your probability is 1 in 4 instead of 1 in 5.
[1] At least, according to this calculator: https://www.socscistatistics.com/tests/binomial/default2.asp...
(Perhaps not in this case but in general this can get you into trouble)
Though personally I'd say that it's quite likely that the p=0.01 threshold falsely flagged a non-cheater, which to me says it is too lax.
But if you ask yourself: why do the students take this course, you can think the reason is to learn something. If they want a piece of paper that proves that, they can't cheat. If you --cynically-- assume that students only take the course to get their diploma with as little effort as possible, then cheating is allowed.
As long as schools want to teach something, they should stay away from online or take-home exams and essays. If they want something else (and usually the motivator here is money), the value of their education will drop.
Sounds funny that an ethics class gets heavy cheating. But it might be a required or elective course that students don't take seriously, or it might be disproportionately Philosophy majors (which major might get more than their share of frosh starting to wield off-the-wall ethical rationalizations).
Good observation - got me thinking. I see lots of suggestions here about requiring ethics training/classes for developers, so it made me wonder-- can ethics be effectively trained into a person predisposed to behave unethically if consequences for the behavior don't exist?
Edited for clarity and moved under the right comment
For example, go back 30 years, and make ordinary software spy on its users, or otherwise act against users' interests, and I think you'd be seen as evil and possibly criminal.
One promising thing, though, is that college students might still be be more open to confronting issues of ethics and morality. (Though often it's only "Revolutionaries till graduation", because of sheltered circumstances or social fashion.) But college is a bit late, and students are already being driven to mercenary behavior by e.g., competition for FAANG jobs (like I used to mainly hear about desperate/ruthless behavior by some pre-meds). Ideally, we teach better values from early age, including by example.
Maybe. But that's not what an ethics class is for; it's for learning about ethics, it's not for learning how to be ethical.
I think that’s reasonable. I would even consider expelling students who did this; specially in college, specially in an ethics course.
At the end of the semester, he explained what was going on. The course was required for the degree (EE), you had to have a C or above to pass, and it was a bear. He said right out, the course was designed to cull the heard.
A few students were outraged. I'm not surprised by the reaction Merriam got. Speaking for myself, I was indifferent to what he had done, though I didn't speak up. I think he should ask the students that he determined hadn't cheated what they think.
That seems unethical too. Students are paying to learn material. Professors are ostensibly there to teach it. Making it extra hard seems to go against that.
Because, if it's the former: great job, Ranger Rick. You definitely used a method that will root out cheaters and give them some (small/limited) incentive not to cheat anymore.
But, if it's the latter, you've failed your students in every respect. It's not even a clean example of the ethics of cheating, because you've tipped the scales in ways that affect multiple variables, instead of just one.
Neither of which is ethical or unethical in the vacuums of consideration that any subject remains neutral in. But if the context is that it's a philosophy class, I would expect the teacher teach me philosophical ethics and let the chaff fall where it may, rather than try to "prove" some nebulous idea of what it means to "know" something, and why one method of being able to 'prove' it is inferior to some other method. Put simply: I'm in this class to learn. If you're giving me the information and then I pass the test, that's your entire responsibility. Whatever third parties are doing - so long as it's not infringing on you - is not relevant. Not to your class and my grades. Sorry you're one of THOSE teachers, but learning isn't a test. It's a lifelong pursuit and you can't force people to pursue what they're A) not interested in or B) deft enough to use digital memory for.
If you're a doctor, and don't know the answer to a question when you need it, you might kill someone. A lawyer has problems there, too. There are great values in keeping people from relying on deferred knowledge, in a great many applications.
I just don't see how philosophical ethics - especially not a mid-level course - benefits from those same lessons/limitations. If a software developer isn't expected to know every language pattern; only how to look up the patterns they need, when they need them, then why would then be disallowed from doing that on a test? A better test would target different metrics than "correct answer" or "incorrect answer". The same thing applies here. You can do software dev without being able to pass an "internal knowledge only" test. You can be productive with ethical philosophy without being able to pass an "internal knowledge only" test.
So yes, both is the goal. What I'm getting at is that both SHOULDN'T be the goal. In a lot of cases. Most, I'd hazard to guess.
But they teach you not to cheat because cheating is bad, on its own terms. The purpose of education is not just to teach facts, but also to teach young people how to be net-positive members of society. (This is why society subsidizes education.) Teaching not to cheat to get ahead falls within that part of the curriculum.
So then, if a professor of an unrelated or tangentially related field takes it upon themselves to make a lesson out of what should be a meta-reinforcement, in a field that they readily admit that they are unprepared to work in or do data analysis for, that is not a reasonable source of ethical education. Especially not when done without prior social experiment approval and safeguards.
You're almost entirely talking about #1, and I totally agree. The vast majority of classes should not forbid looking stuff up because it is an artificial constraint that doesn't reflect the reality outside school.
But I think #2 is also important. I know it's passe, especially in the silicon valley milieu that we swim in here, but I think "not following rules you personally think are stupid is good" is a bad lesson for students to learn.
(FWIW: I thought the opposite when I was a student, but I was a short-sighted idiot when I was a student.)
That is highly debatable. Cheating most often affects the cheater, if anything, because when they need a skill they pretended to have, they'll fail at real-life tasks. But nobody else is affected by cheating.
Except the people standing on the bridge designed by the civil engineer who cheated, or getting radiation therapy from a machine programmed by a cheating software engineer, or being treated by a doctor who cheated in med school.
Cheaters getting degrees from an institution that's supposed to produce high-quality graduates is bad for the value of that degree, which is bad for the institution, anyone else who went there, and anyone who relies on that degree as a signal that the graduate has the skills they're supposed to have.
If only this were true. If you read the history of education, it was generally invented to create compliant factory workers that were adjusted to rigid schedules and strict authorities. (Its not a coincidence that so many schools are named after robber barons. Rockefeller, Carnegie, Vanderbilt, etc) University used to be a bit different, but now that everyone is expected to go and its about vending credentials it's arguable that creating well rounded individuals is at best an occasional bonus.
Once you're there, there's not a lot of reason to go further. "Did he cheat? Yes? Then he bad." Easy peasy, next topic of consideration. Why spend more time on something that has a simple answer?
In this scenario, the students did what would have unequivocally been a "cheat", in generations past. They looked up answers and used them to prove knowledge. That's what every person who ever failed a math test for cheating did.
So, again, even going deeper, this is cheating, plain and simple. No need to wade in any further, when you've already given the benefit of the doubt.
Of course, this not being math, and technology replacing the need for such conventions in math are both important considerations that change the context. But, you have to give it now a third thought before you even get to this point, and of course, here's where things are quite subjective. I bet people would still pretend that knowing math is important for whatever corner of life they inhabit ("I work construction and geometry makes you a pro", "gotta be able to calculate tips", "you'll get killed on contracts with tricky rates"). Most would also quickly concede that if they were just taught how to get to interfaces (apps/websites/etc) that solve those problems, that would be effective as well (even if they couldn't help adding "but knowing is faster/better/convenient/etc"). And so, not only are we dealing with subjectivity and historical bias, but now also a confirmation bias. And all of that is on top of the authority bias that a lot of people START with ("the authority said don't do it; the students did it; they should fail."
So, personally, I don't chalk up to indoctrination/conspiracy/malice anything that can be assumed to be myopia/selfishness/fear/distrust/etc. Humans are capable of the formers, but far more susceptible to the latters. That said, I'm never going to defend the US educational system, so I'm not disagree with you.
Everyone is very quick to suggest brutal punishment for the people who have contradicted the will of the authoritative system, but people aren't really so interested in discussing both why they've done so, and the societal consequences of that disobedience. And even then, not considered the impact on the individual being punished; many people seem to be wishing permanent banishment from the university system for cheating in what is probably a random side unit elective class.
From the comments in this thread you can draw the obvious conclusion: many people here justify cheating because they are cheaters themselves and they don’t want to acknowledge cheating is shameful.
The provided test tries to have it both ways, but that's invalid. If they are supposed to be learning the answers to the questions on the test, they can prove that even if they looked those answers up. But if the test is to not look up answers and then they get a score reflecting their ethics, then the class is meant to make more ethical students; not to have more people understand the answers to the questions on the test.
At best, you could make the case for the ethical test being a component to the class. Give it a rigorous score, just like you do for any other answer on the test (pass/fail seems reductive, in this scenario, but it's reductive for a lot of tests answers, so whatever. bless the teachers who give partial points). But verifying what a student KNOWS does not, in any way, involve what ACTIONS they take.
I have taken some very well constructed tests. They were not takehome and only one was multiple choice (it was a systems class so it became sets of binary answers which became more than multiple choice). I think that ensuring the integrity of answers in necessary for gaging learnings from a test like this but that need, as you've implied, would indicate that the test is the wrong tool
But I think the takeaway is that "critical application" is what constitutes a valuable test. In math, that just means "can you use the procedures of mathematics correctly, resulting in the correct answer?" But in, say, English, memorization has very little value. A valuable test would be in testing whether or not a student can match events in a dramatic story to, say, modern real world events and critically discuss the similarities and differences. And, unfortunately, writing that out clinically and expecting a student to both understand it and be able to do it on command while under pressure of failure is not exactly conducive to positive results. So English tests really shouldn't look anything like Math tests, once you get past the point of memorizing definitions.
The same is true of all disciplines. Tests should be generated (and often updated) based on what is currently useful to the discipline in a real and practical sense, at a fundamental level, and then on a more philosophical or exploratory sense once you get into higher learning.
Plus, cheating or not cheating is an ethical matter, not a pedagogical one. He is not trying to teach anyone not to cheat. The expectation of not cheating comes as a pre-requisite of any formal education. He has the social duty to capture cheaters because if not he is failing the society and system who have given him the responsibility to not only educate, but to certify someone's skills. If the cheaters get out there with a diploma in their hands, certified by him (and his institution), he has failed those who believe in that diploma's validity.
Everything you said about learning being a lifelong pursuit that can't be forced, etc. are personal pursuits which are independent and orthogonal to this teacher's formal responsibilities in the educational system.
Let me ask you people something. Of the college graduates that have been out longer than 2 years, what do you actually remember about non-major classes? My guess is: basically nothing. It's a waste of everyone's time. I don't support cheaters because at the end of the day you agreed to the rules, but I'm not upset that people break dumb rules to save their own time.
But anyway, it’s tricky to set different exams with the exact same difficulty. So the alternatives to some degree of curving are:
(i) accept that the exam may vary significantly in difficulty from one year to the next, or
(ii) use the same exam every year.
There is no perfect option.
To the original point, if someone cheats, they weren't interested in the first place, so why were they there? Likely they were coerced. I just think this effort of "gotcha"ing people that don't give a shit about the subject in the first place might be better spent either showing them why they should care, or working with the people that are actually interested. What could this professor alternatively have done with his time compared to this, which is a headache of dealing with these people that are going to have to, bitterly, retake this class, or worse, leave school altogether? What was accomplished here?
No, it just means "last year's exam seems to have been a bit too difficult, so I'll make this year's a bit easier" (or vice versa). Curving means that you assign grades based on a student's performance relative to the others in the class rather than on their absolute score on the test.
>but I don't really understand why difficulty would be a useful metric,
If you really don't understand how it's possible for an exam to be too easy or too difficult, then I'm not sure what to say. The exam is intended to differentiate between students who've mastered the material, students who understand the basics, and students who learned nothing. To do that it has to be difficult enough that people who skipped half the classes† can't get full marks, but also easy enough that not everyone fails it. Even these basic boundary conditions on a good exam can be quite tricky to satisfy in some cases.
†Not that I care about people skipping classes per se. I just mean that typically they'll have significant gaps in their knowledge of the material.
However what I don't understand is why that even mattered.
Were the students just learning the questions and answers by heart to regurgitate them on the final exam? If they had any understanding at all they should have caught on, but even if they didn't they would simply demonstrate their lack of understanding, it is not dishonest.
Or did they get to fill in the answers unsupervised somewhere? Because if they were left unsupervised with access the web then this is likely just the tip of the iceberg, they could more easily cheat by discussing the questions with each other.
Edit: Reading more carefully it was a take-home exam apparently, which seems to have consisted of multiple-choice questions that are largely the same each year. I can vaguely see how looking up old exams would invalidate it as a test, but if your test is invalidated by normal exam preparation is it the exam's fault or the student's?
Based on his post, my assessment is that the prof is lazy and an asshole. Maybe there are other relevant factors that would change my analysis, but I have to think this is the version of the story that puts the prof in the most favorable possible light, so….
Generally previous exams are good for checking that you understood a topic but you can not really study a topic only from past multiple choice tests. If they had studied the topic and gained some understanding, they would have noticed that the answers are wrong. Even just looking at two different past exams would probably reveal that something is off.
I guess an important difference is going for engineering, it was trivial for professors to change numbers & details from year to year so old tests could aid study but not provide answers.
When I was at uni in the early 90's, several of my professors handed out older versions of their tests so that we could see the shape of the thing and prepare. We all appreciated it, too.
Students would commonly complete multiple past exams going back years, and anyone struggling with a question could ask a tutor about it in one of the labs (so that they actually learn the underlying principles).
A lecturer re-using an exam question verbatim in that environment would be considered reckless.
It sounds like the rules are different a Sacramento State University - perhaps to facilitate verbatim reuse of questions (which sounds like sacrificing educational quality for lecturer convenience). I do think, however, that this is exposing poor educational practices that reflects negatively on the university and the lecturer as much as on the students.
i don't think it's very clear cut or obviously perfectly fair.
I don't think this matters. Cheating was against the rules and a cheater demonstrates a failure to adhere to a social contract and voluntarily waives their right to be assessed the same with those who did follow the rules. At least there are avenues for a cheater to have another attempt, even if they have to move to another institution.
During the exam (which I had to administer remotely this time) I made it abundantly clear that they cannot use any kind of AI assistance for their work. No ChatGPT, no Copilot, no Bing AI, no Google Bard, and also no Googling, etc. I repeated this several times and also wrote it in bold font on the top of the test. I really didn't have any way to enforce this, but I was hoping people would be honest.
I'm pretty sure that most of the students were honest on this; the answers I got generally fine, but had grammatical mistakes and were "basically correct but had light factual errors that are common with people new to programming but aren't bad enough to count as 'wrong'". One student, however, who has submitted broken sentences and broken code the entire semester, managed to suddenly have decent writing skills, decent explanations of everything, and his code was clean and concise.
I'm about 95% sure he used ChatGPT to generate answers to the questions. I tried getting ChatGPT (and Bard and Bing AI) to give me a word-for-word copy of what he submitted, but I couldn't. It got somewhat close, but never an exact match.
Technically, it's possible that he just studied very very hard and his code and grammar improved. It's also technically possible that he used Grammarly to make sure his writing was ok (which was technically against the rules but I wouldn't really consider cheating in a Java class), and so I just had to swallow my pride and grade the test assuming he was being honest.
It's kind of upset me all day; I have worked pretty hard trying my best to be available to students if they have questions, and I worked pretty hard to try and make sure that the final exam was a reasonable level of difficulty. I think most of my students were fine, but one bad apple is enough to really ruin my day.
It's just one of those things that while you're totally right, the one or two bits of failure kind of nag at me. It's extremely easy to take these things personally (especially for a nascent teacher like me) when in reality I should likely view things as transactional.
I do feel that by having these tests and grades that it does, indeed, sort of become just a transactional way to "vend credentials" instead of focusing on learning. I also think that often homework and tests aren't great measurements; I was bad about doing homework in college the first time around, and it was frustrating to the professors because I was doing horrible grade-wise, but was extremely active during class and generally did fine on the tests. I was mostly learning the material (at least well enough to get an A on the exams), but because I wasn't submitting my assignments, they would be forced to give me a C or a D.
At the same time, I really haven't figured out a better system. I would be open alternative systems, but I think grades and grading and tests do the best job for the "average case".
Maybe I should email some of my old professors and apologize for the headaches I probably gave them...
I don’t see this happening unless jobs stop caring about credentials or become more willing accept other forms of credentials.
Of course I’m assuming you aren’t specifically talking about software, which doesn’t have nearly the amount of credentialism as many other fields.
Something to the effect of "ChatGPT is pretty cool, I like it, but it will just make stuff up sometimes in extremely convincing way. It's a tool that's dangerous to newbies and powerful to professionals, and I'm hoping this class will help you get closer to the latter".
People did the same thing for Wikipedia and it was never convincing. And they eventually lost the battle. It's going to require more than that to do the same for GPT.
I think by tonight I'll have cooled off enough to write an appropriate email.
If I got an email half accusing me of cheating I would complain to anyone who would listen.
A random college class, to me, isn't the kind of thing where cheating really matters. The stakes are low, it's expected that most students will do well, and everyone gets the same degree at the end. If someone wants to sabotage their own education then fine, and if it doesn't bite them later good for them.
I had one candidate that struggled MIGHTILY on a coding problem that was easy (kept running into syntax errors) but then pulled out an esoteric solution for the second problem in no time flat.
I can't prove that he was cheating but it very highly suggestive that he is.
I hate it, but I'm starting to think I have to do an AI detection question now. It's not hard, just ask someone to do something impossible. However, I don't like the fact that now I need to be "tricky". I've never believed in making coding challenges hard, I just want to see if you can write code.
In this case the candidate chose to use a language that I don’t know well and they said they weren’t especially familiar with either. Then they struggled a lot to even talk through the question and it was a grind to work through each line of code. I couldn’t offer much help, and they were very clearly unsure at each step.
But when we clicked “run” on the code, we were both surprised to see that it ran on the first try. Not only that, but it already worked for all of the “but what about…” follow up edge cases. And the candidate didn’t know why… “oh, I guess the language just works that way.”
I don’t know if they were cheating, but the entire thing was far outside any other interview experience I’ve ever had — and fits the pattern of what I’d expect an LLM to produce.
I also have adhd and literally brain damage and have worked professionally in about a dozen different programming languages. I can't write a for loop or declare a static method or w/e in any of them without googling the syntax or using my editor's hints & autocomplete.
They had google access and editor hints. Further, I start every interview with "please, ask any questions, we aren't trying to trick you we just want to see if you can code." And I mean it. If someone has a minor problem with remembering syntax or how to declare it I don't really care and will happily let them know how to do that.
Heck, I've even had interactions where a candidate was like "I think there's a method that does x for this" in a language I was unfamiliar with, so I googled up what x was and shared it with them mid interview. "Oh yeah, looks like this is what does x for your language".
There was also some pretty weird behaviors with the camera/screen that caused them to need to touch it fairly frequently (I'm guessing to take over keyboard control from the AI software they were using).
What they were doing, repeatedly, though the interview was writing
`a[i]` then having to change it to `a.get(i)` because they were working with a `List` in java. I get maybe doing that once or twice, but in the course of the interview they did it every single time they needed to pull something out of the list. (and each time needed to touch their screen and take a few minutes to correct it). This is why I strongly suspected cheating. It extended a question that normally takes 15 minutes to 35 minutes. Then the question I ask that usually takes ~45 minutes they completed in 15 with an optimal solution that relies on a data structure I literally only learned about because the interview hints for the question are like "Hey, java has this data structure that makes everything easier".
But you were there and I wasn't and if the overall situation seemed suspicious to you that's evidence at least as strong as any specific thing. Intuition can certainly pick up on weird behavior even if it's hard to explain how after the fact.
I am assuming that you had a compiler or IDE in this situation providing a useful error message. If you were whiteboard coding and you were telling the candidate "you have a syntax error but I won't tell you where" (like several folks at a former employer of mine were fond of), you may be the problem, not the candidate.
> I am assuming that you had a compiler or IDE in this situation providing a useful error message.
Yup. I prefer to give candidates compilers/IDEs and even access to google if they ask about it. Like I said, I'm not trying to trick them or anything, I just want to know if they can code.
> If you were whiteboard coding and you were telling the candidate "you have a syntax error but I won't tell you where"
I'd hate being on the receiving end of that and would never pull that sort of stunt. The software we use allows both sides to edit the code and I pretty regularly will go in and silently fix syntax errors for a candidate so as not to let a compile/refresh loop get in the way of actually solving the problem.
Like, what does "You have a syntax error" even prove? I'd let you write pure pseudo code so long as it makes sense.
I just don't like "mind games" in interviews.
The current system worked in 1960. Not in 2023, and probably not since the late 1990s, when technology started to make our evaluation processes obsolete.
If it would be possible to take a bottom-tier CS student and turn them into a decent programmer using AI tools, the hiring landscape would be /very/ different. These tools aren’t magic, and to me it sounds like the tests are failing at measuring the students competence if they can so easily be gamed by using an AI/old fashioned googling.
Maybe the solution should be to move towards a style of exam/grading that actually measures the competence of the student in a situation closer to what a professional developer will be in, rather than an old fashioned artificial exam setting?
You may think it's ridiculous, but many people today can't calculate a 20% tip or add two prices together without a calculator because the constant use of calculators has caused their arithmetic skills to atrophy. This seems like a silly gripe, except a lot of other skills are built around these basic ones: for arithmetic, the ability to estimate the cost or time taken to do something is all based on tricks you learn when you are trying to learn mental math. Society is not worse off for this (enough of us still know how to do these things), but many people are poorer, both intellectually and monetarily, due to a lack of arithmetic skills. And no, this lost knowledge of arithmetic is not replaced by a knowledge of higher math - it tends to come with a fear of it.
The same applies with text-generating AI. In terms of writing, if you don't learn to write dumb essays about books, skills like learning to construct an argument are much harder to pick up. For people writing code, learning to slog through writing and debugging a doubly linked list (something ChatGPT can reliably generate for you today) leads you to later being able to slog through debugging B-trees or lock-free queues (which ChatGPT definitely cannot write for you).
I think there is a very compelling argument along these lines for low-level courses to ban AI tools. However, higher-level courses probably should allow students to add them to their repertoire, where they are an aid and not a crutch. This follows how mathematicians and engineers learn to use calculators and computer algebra tools, which seems to work well.
Earlier this week, I spent several minutes talking one of my co-workers through how I 'did math in my head' to figure out my half of the 20% tip we were leaving for lunch. Even something as rudimentary as moving the decimal place over one and doubling the result seemed like wizardry to them.
They aren't dumb. They've literally just never thought one second past reaching for a calculator. Which is kind of scary, because it means they have no way of sanity checking any numbers they come up with.
A person who does not know how to code without access to AI tools means they will consistently push bugs generated by AI, affecting the teams productivity, if any.
ChatGPT can solve most LeetCode questions, but I don't think most people here are ready to replace their junior developers with it.
However, I feel like an academic exam is a bit different. The goal of these exams is to evaluate what students have _learned_. Unless the class was about how to find helpful code examples on SO, it does not make much sense to allow AI/internet-searching during the exam.
AI can help you write a linked list, but can it help you know that you need a linked list vs. a queue or a binary tree? Can it help you to make more architecture-y decisions like deciding whether you need a document database, a traditional SQL RDBMS, or maybe something else?
I'm not in a teaching role but those are the kinds of things I'd like to see examinations cover -- what tool or data structure are you choosing, and why is it the appropriate tool for this specific case?
Or, similarly, can the student understand existing code and fix bugs or make performance improvements?
Bottom line, though: professors should just absolutely accept the reality of AI. Assume all students are using AI tools. Actively mandate or at least encourage their use in order to ensure a level playing field.
It's not just the issue of "what if they didn't have access to ChatGPT?"
Just as an example: ChatGPT is literally the world's biggest bullshitter. It vacuums up vast stores of "data" (note: NOT "knowledge"), and just pulls out words that it thinks could likely follow from previous words given a context.
Quite often is produces silly, or even dangerously wrong, answers. You need to be pretty well skilled in the art to catch it at its bullshit.
It's just like when you hire assistants to do something critical for you. You have to be able to verify their work, or you have to be able to trust them blindly.
I've known engineers who did not understand the analytical tools they were using. They misused them constantly.
The point is to gain some knowledge yourself. Obviously you'll use new tools and methods later when you eventually get a job, but the point of most courses is to teach you the concepts, not how to command the IDE. Whether you decide to hand-craft assembly code or become professional a Copilot suggestion approver doesn't really matter.
It's a question of: Are you testing for understanding or ability? Both have their merits, but often the goal is the former. If it were a project, it is generally the latter.
So that they can tell when the AI tools get things wrong. Which is always.
I'm old: when I went to school we weren't allowed calculators "because there'll be a situation where you need to do basic arithmetic and don't have a calculator". I occasionally laugh at this while firing up the calculator app on my phone. We'd have been better off if they'd taught us sign language rather than arithmetic.
Our education system is going to have to adjust to the new reality. Setting an essay task is now the equivalent of learning the times table - utterly redundant (unless you plan on a career writing essays or doing maths).
This was pretty useful in it's own right, as you at least had a way to check your work, and frankly it was extremely useful for helping me solidify knowledge in those fields. [0]
Of course, AI muddles this, since it can explain the steps for you.
Higher-level math classes usually encourage calculators, and graduate-level math classes will often allow Matlab or Mathematica, as long as you can write a short sentence explaining what the computer algebra system is doing. Where I went to school, this was colloquially called "proof by Steven (Wolfram)."
Many people today do math only with a calculator and correspondingly lack any sort of numerical reasoning skills whatsoever, despite having learned both mental math and numerical reasoning in school. That growth of innumeracy coincides with the rise of calculators and computers who do it for you.
Also, students are taught in more advanced classes to use calculators (and later computer algebra systems) as a tool. This comes after they have learned the basics, not as a replacement for the basics.
Writing (hello ChatGPT) and math are like muscles: use it or lose it. If you can't do the basic versions, you can't do the harder stuff either.
But I do think that as we move forward with technological tools for this kind of mental task, that we should embrace them in our education system. If we have to teach every generation from first principles then we won't get them comfortable with the actual tech they'll be using in their lives.
thanks for the clarification on sign language :) One of those things on my bucket list that I keep meaning to learn but never get started on.
When you're solving the same problems in real life you do have access to all those tools that you banned. Because in real life, if you manage to solve a problem it doesn't matter that much how you solved it.
Sure, if you just Google things and always solve your problems by copy pasting. Your solutions will lack depth, and at some point this will catch up to you.
My point is, cheating is primarily cheating on yourself, and it will catch up to you.
I think this is the framing you should have with regards to cheating.
As an early procrastinator (rehabilitated perhaps?) I've had a history of doing very well on finals through cramming.
In your student's case it might very well be more likely they are cheating, but those unlikely but possible students who study extra hard to recover their grades for a final deserve the benefit of the doubt.
How unlikely can academic recovery get? I spent an entire semester completely lost in a second level Macroeconomics course...
... I was able to read and write chapter summaries of a semester and a half of content (about 15 chapters, corresponding to the first level and the second level courses) in a single weekend and aced the exam, with the highest marks in the class.
That was the hardest I've ever crammed in my life and I had trouble speaking in the hours that followed the study session.
I later got a recommendation letter from that prof, totally worth it.
You have to both adjust your expectations and make a significant part of your grading use an in-person one-on-one oral exam or project walkthrough.
It is a huge commitment of time - most professors I know seem to have quietly accepted cheating will be be rewarded because they don't have enough time to verify student performance.
Well, lest of course the students sit under the watchful eye of AI in the near future
You're swimming against the riptide. If I were teaching coding today, I'd allow, even encourage, use of AI, with some ground rules. Students using AI would have to show their work (i.e. the conversation with the machine).
The content of the prompts and in particular follow-up questions, can demonstrate competence. E.g. situations in which the student spotted that something was wrong in the generated code and made an intelligent hint to the machine to fix it.
Students should be prepared for tomorrow's world, not yesterday's.
If we want to ensure kids can do arithmetic without calculators, we can have exams in which there are no calculators.
You can give them homework problems in which they have to fill in all the work steps of multiplication and long division, e.g.:
If you want to ensure that the students know and follow a certain process in obtaining an answer, you can't just test the production of an answer. You have to white box it.Kids should know time tables to about 12. Beyond that, you can test whether they know the structure of long multiplication and division and such.
Which is exactly what the parent was trying to do with a an exam that forbade the use of AI.
To be clear, I'm not supporting the cheaters. There are two wrongs here. "Am I the unethical one?" should instead read "am I unethical?" to avoid any discussion about which is worse. Suppose I went on the site and uploaded wrong answers, thereby clearly sabotaging cheaters. Clearly that would have negative consequences for those students and one might ask why would I do that.
Fuck the students who thought "wait, this seems like...way wrong. I feel like he said the exact opposite in class...? But, I mean, this IS the test that I'm looking at, so I guess I'm just misremembering...", is the apparent sentiment. At least he caught the obvious cheaters, right?
say what? One is on the official school site, in the lectures and notes from class. Second one is an anonymous upload at the weird unofficial site which does not even have university's branding.
If the students can't tell those two apart, they are in deep trouble and should not be in college.
Sort of tangential, but I have always thought teachers/professors who put "trick" questions on tests to be sort of assholes. It's fine if it's an extra credit thing and isn't going to take away from the final grade, but when you write a question that literally everyone in the class gets wrong, I think that says a lot more about your communication or teaching ability than the students.
From an ethical point of view, he has done these students, the university, and the world at large a huge favor.
Don't agree. You're saying that if I can't catch all the cheaters, then I can never call anyone out for cheating. That's a licence to cheat.
Cheating is bad, for the same reason corruption is bad; once it's accepted that it's allowed, then anyone who doesn't cheat/take and give bribes, then everyone who plays by the rules is at a disadvantage. Corruption is a disease that spreads if you don't stamp it out.
Also: cheating on tests is an insult to the intelligence of the examiner. The cheaters are laughing at him. If you don't call out cheaters, then the whole idea of getting a graded result becomes meaningless; you might as well simply give every student that actually paid their fees an 'A', and do without exams.
BTW, I think it's irrelevant that it's an ethics exam. You don't study Ethics to become more ethical. You study it to learn about the philosophy of ethics.
If I were in this professor's case, I'd just mark the answers wrong, and in the future upload more wrong answers. The students who use these sorts of online dumps aren't the ones who study and will beg at the end of the semester for some sort of extra-credit. If it were necessary, perhaps have a second gradebook where the number of "exactly the same wrong answer as the bait" were kept.
It's funny when that happens, I used to do this in 7th or 8th grade when we were doing home exercises in touch typing. All the typing tasks were given from a textbook that was the same for everyone in the country, for a few years at least. Over 2 years of classes, I had probably spent more time scouring the internet for the solution rather than type it myself. The search was however always more entertaining than the exercise and taught me how to use the web. And I still learned how to touch type.
I've seen this many times before, where a teacher seems to fail to realize that their students don't just have their own exam to prepare for; they have to prepare for many other exams at the same time, all the while struggling to balance their study time with their responsibilites at home, their social life and possibly their part-time job at the same time.
So when an answer sheet is just readily available online, there aren't many students who wouldn't choose to spend a few hours memorizing the answers so they have a little more breathing room for other (possibly more difficult) exams.
The statements about how this teacher apparently feels oh-so stressful about this situation that he purposefully created himself, all the while dismissing any and all critique from people because they aren't "teachers of any kind" feels very childish and leaves a very bad taste in my mouth.
There is nothing wrong with removing their ability to cheat, but purposefully uploading answer sheets and then getting angry that students made use of them isn't. In fact, it's not just wrong: it's ethically wrong.
The "busy" argument is a poor one. We're all busy. Part of gaining an education is learning how to manage your time. As a professor myself, I know for a fact that most students manage their time poorly, yet many students will still pull the "busy" argument when it simply doesn't apply. Rather, just admit to procrastinating. Either way, the outcome is the same (poor performance).
To sum up my sentiments to cheaters... "Play stupid games, win stupid prizes."
I have done 2 ethics courses during my education at a Dutch 'Hogeschool' (honestly not sure how this maps to US education wikipedia says 'Vocational university'). I did a specific design ethics course and a broader ethics course as part of a philosophy minor, and in both of them you had to write papers or apply the things you learned to a case study. There were some little tests with multiple choice, but they often had additional questions where you had to explain your reasoning.
Maybe there is a language difference here, but I would expect something more involved from a course given by a professor at a university, or is this a course for people who are in high school or something.
This is just bad math. Take a question (A), there are 5 possible answers, using his analysis the probability of a student picking the same answer as the cheating answer is 1 in 5. But let's say the question is hard, and of the 5 possible answers, 2 are highly plausible, so plausible in fact that the students always go for 1 of the 2. Now, if the cheating answer is one of the plausible answers the probability is low, but if it's one of the plausible answers, then it's high. And more specifically, if the cheat answer is correct - what's the probability the student got it right? Well what you should be doing is take the other 55 non-cheating answers, calculate the probaility of correctness and then use that as the probability. The "1/100" threshold is overwhelmingly determine by how he selected the answers on the leaked answer sheet, and you can't say it just average out with such a small sample.
Modelling the whole thing as random choosing is just sloppy maths.
Already accounted for per the article.
To put it another way, let's say he planted only wrong answers, and we have an idiot student that always gets the questions wrong. Now the probability of matching the planted answer has clearly jumped, right?
On the flip side, if the planted answers are right, and in general you expect your class to pick correct answers (ie, they're going to pass) then the probability they pick the same as the cheat answer could be as high as 90%
The only thing that could have been added (that I can think of!) is that the likelihood of cheating is not memoryless, as the binomial process assumes, but rather that a cheater would not copy a few answers, but very many. So the chance that a cheater picks the bad answer is > 0.2, the chance that someone who doesn't know the answer = 0.2, and the chance that someone who does < 0.2.
But given the fact that we don't know the correct probabilities, nor the scale of cheating, and not even how often people take a random guess, makes the model a decent starting point, IMO, for getting a first estimate.
After all, the teacher is not researching stochastic models of cheating, but just grading papers.
The only room for ambiguity I can see is the arbitrary 1% threshold. If there was a student just over the line then it's plausible that they were honest but unlucky here. Given the consequences of being tagged as a cheater both for this exam and beyond I would want much lower odds of a false positive and I'd certainly feel obliged to give a borderline case the benefit of the doubt.
But if it was clearly understood that this kind of behaviour constituted cheating and innocence is a billion to one shot? Throw the book at them. Anything less is unfair to every student who didn't cheat.
The honeypot is fine. If it were me I'd warn the class that I've seeded the internet with bullshit answer keys, and while I'll never be able to prove you used one of them, you'll probably fail if you do. And leave it at that.
Instead he's guessing a threshold, and can't really be sure if someone that insists they didn't cheat is lying or not, with high stakes. It's a bad situation for all involved.
This year one of my students claimed to have done a user experience test with 30+ people, all of which came in person to his house during a period of 4 days. This is extremely unlikely, not to say absurdly unrealistic. If he had done it via Zoom I might maybe believe, but in person? Sorry, but no. I asked him what did his parents think of it, and magically they were both travelling that specific week. Then I asked him to scan and send me the signed consent forms for each participant; he promptly said "ok, coming!" then about 8h later I got a bunch of signed scans. Not sure what to do anymore, I guess he'll have his thesis.
I'm not familiar with conducting a user experience test. Why is this so unbelievable? (Also, the rest of the story is kinda hilarious; there will definitely need to be some standards and process changes in academia.)
So if you assume that the UX tests were short (20-30 minutes) you're still talking ~15 total hours spent on UX testing, or two full work days. That would be a surprising amount of time.
As someone else said, it's conceivable that they ran very short studies on a large number of people in the context of having a party or a couple of small gatherings, but it does seem unlikely overall.
(And this real-life scenario where every request is answered with “Coming right up!” and a convincing “successful” completion of the request just cracks me up. My sympathies go out to the people to whom this is a genuine problem.)
Thanks for explaining!
Is it possible that maybe he thrown a big party (or a series of parties) and asked the folks there to try his stuff? Not sure what is the length and depth of the usability test in question, but if it is a short questionnaire that might work? But I see where your suspicion is coming from.
Maybe that student was doing the same. Simply asking a lot of people. And other studends usually do not. I find it really hard to overcome my own shyness in this regard. And I wouldn't consider myself particularly shy. I assume a lot of people simply do not try asking enough people in order to get a decent amount of them to join.
Of course that guy might be simply cheating, but I just wanted to share the story.
Scan the signatures and reverse image search them. Or look one or two of the people up on facebook and ask them whether they did the study.
The consent forms are likely friends, so calling to ask questions about the experience should give you an idea of what to believe.
The outcome can be your decision. Meet with the student and tell them their degree is in jeopardy and use it as a teaching lesson, or prove to them that they can get away with believable lies as long as no one has the courage to catch them. Your colleagues should have some insights, as well.