Wouldn't it be an option to basically chain the "events" in the local storage queue with crytographic hashes (kind a like a block chain). You can then fairly easily verify modifications somewhere in the list of events.
It wouldn't make it entirely impossible, but much more complicated.
A hacker does yes, but can a kid recreate the hash quickly without the teacher noticing? If some script appear on the net to automate that, Khan academy could react by changing their hash frequently.
It's not even that much of a cheat, really. I'm surprised they decided to make a full blog post on how they fight such a simple trick (incorrectly, even).
It won't work that much anyway, a simple chrome extension that injects code can overturn their whole thing.
How far can they fight cheating anyway? I remember making a javascript snippet in Chrome devtools that would bruteforce all the crypto questions (because they had predictable patterns, like all sentences involved "Anna" or some similar name). That seems way harder to fight.. will it deserve another blog post?
I must admit also, I was reading this and wondering how their fix actually "solved" the problem, and even more confused that they thought it was a novel enough solution to actually blog about it. In reality, this blog post reveals further ignorance of the actual issue.
Sure, but presumably, if the student does that they learned quite a bit about how browsers work… It may not be related to the course they are taking, but at least, they learned something thanks to Khan Academy, and will prompt them to learn even more.
I'd love to see one created, just to light a fire under khan academy so they fix it properly. If going offline is so uncommon in their eyes, why not store the hint on the server?
Probably a dumb question but why not just let people cheat? I'm pretty sure we all heard in grade school that "cheating only harms yourself." Those who actually want to learn the material will stick it out and do the work.
I agree. Unless Khan Academy has recently started handing out degrees or certificates that people use to get ahead in something (as opposed to just learning stuff), I'm not sure I see the point of this.
It's kinda funny that this blog post, essentially describing how to cheat on Khan Academy, soon is going to be one of the top results on Google when students search for "Khan Academy cheat".
Perhaps this blog post is a decoy, and they implemented a better prevention mechanism. I don't feel like "giving away" this possibility harms their efforts, as it will only make those intent on cheating more paranoid. >:]
Strange to see them trying to fix this with technology at all. Cheating is a social problem.
Universities survive just fine giving out take-home tests and trusting students not to cheat. Honor codes work, even when there is an actual useful reason to cheat.
But on a site like Khan Academy, where the end goal is to learn something useful, all the user is doing is cheating themself out of the thing they signed up to get. Let them go nuts. If they want to go through a course, but go out of their way to avoid actually learning anything, that shouldn't cause the site owner any grief whatsoever.
Individually, cheating is an user experience and design problem. Did you ever play an engaging, interesting game, used a cheat code for more resources/lives and game instantly stops being interesting even if you don't use cheat codes any more? This happened to me multiple times. Experiences of others is similar.
"go out of their way to avoid actually learning anything, that shouldn't cause the site owner any grief whatsoever."
Obviously, Khan Academy staff has a different take on this problem and that's great. Online education should strive to cater to cheaters, too.
As a working university teacher, I have to disagree with this somewhat. Cheating happens of a massive scale, I'd say at least 1/3 of all coursework I have graded involved some degree of cheating, and I've seen cases where clearly everybody in the course had copied from one person (who had an ingenious but false solution). Whenever I discuss this with former students, they think 1/3 is too low. When I give out coursework, it shows on sites like rent-acoder.com. One meets people who make a living out of writing dissertations for rich students.
The universities accept this tacitly for two reasons.
1. They need the income from the students. If they'd fire every student caught cheating, they'd go bankrupt pretty quickly.
2. It's an extreme amount of administrative work for the professors to deal with. In my experience, I need to invest a day's worth of work per cheating student, all things considered. I really have better things to do in life.
To put it pithily: the students pretend not to cheat, and the universities pretend not to notice.
This is nothing new, students have been cheating since time immemorial, the world keeps on spinning ... there are lot's of stories about cheating in what may be the oldest exam system in the world, the Chinese Imperial examination. (As an aside, if you want an alround fascinating story that involves the Imperial examination, check out the history of the "Taiping Rebellion".)
the university is indirectly cheating when they publish statistics about the grades their students get.
Not really as long as they don't guarantee that grade X guarantees a specific skill level. Clearly there is a usable correlation between grades and skill, but it's not super reliable, whence coding tests, interviews, requirements for n years experience, and so on.
I agree with you. Cheating was and probably still is very huge on my campus. Honor systems don't work and many of my professors didn't care.
Very few people talk about what it feels like not to cheat and why students do it. So here goes:
I made it through college without cheating. My university actually managed to scare me from doing so with freshmen orientation. I focused on studying and making sure I took advantage of walk-in hours and student teachers. For a while I thought everyone was doing this. Then, as weeks passed in each semester the students got more and more brazen. During my years of school I was outright asked to share my test (sometimes during a test), I was offered payment and I was made to feel ostracized when I didn’t “help”. It sucked.
The worst part came when I would meet other honors students and find out how they got their high scores. I was surprised at how many of them cheated, to what degree they would cheat and how little they knew about their course work. This is when I found that there was no incentive not to cheat. Honors students got scholarships and were often given some very good internship opportunities.
Why would anyone not cheat? No one cares if you do. You’ll have an easier time. It doesn’t hurt your career. It also does wonders for your social life. My final two years I pretended to have low scores on tests so as not to be asked to help cheat. I kept my grades a secret and would often not look at graded papers in class. I stopped talking about any of my classes with anyone that was in my major.
I know what it is like to cheat and not to cheat: I cheated like crazy
in high-school in my weak subjects, but didn't cheat at all at
university because I was the best student in the year so copying from
others would have reduced my grades.
Why would anyone not cheat?
You forgot the most important reason: to learn. In some sense cheating
is a self-punishing crime, because the cheaters forgo the learning
opportunities that courseworks and exams offer. Indeed the only reason
universities have to have graded courseworks and exams is to put an
incentive structure in place that focusses student attention. In a way
grades are an anti-procrastination framework.
I technically cheated a lot in college and HS, but not in my weak subjects. It was an economy of time. Person A would do some parts of the assignments and I would do other parts. Combine the work and you've got twice as much done with 3/4 of the effort (I would still spend a little time deriving the work instead of wholesale copying). By the rules, I should've been thrown out of most of my courses (except CS and EE where I loved the work too much to let someone else take it from me) but I wound up with marks that put me at the top of the class. I learned enough doing just enough of the assignments to show up and get good marks on the tests without cheating.
I honestly don't think what I did should've been considered cheating, and I would do it that way again today- in fact, don't our jobs work just about the same way? But I'm all in favor of strict rules around cheating on testing or final projects. Those are where you demonstrate what you personally have learned, I don't think it should matter how you got to that knowledge, but you should have it.
It sounds like your university needs fixing. I went to a run of the mill State school in the US, and it was made abundantly clear that cheating on an exam would be followed by expulsion.
It was pretty rare that I heard of anybody doing so.
I'd recommend going ahead and having those kids expelled, or at least starting the process and taking it as far as you can go (before the university has to step in and make a point of stopping it). Change has to begin somewhere, so why not with you?
Edited to add: I don't really buy the economic argument either. A top school has a pretty long list of kids wanting to get in. They could toss out half the current students each year and never run in to an issue filling back up the next year. If anything, it would only serve to improve their reputation as an "elite" school where you really do have to pull your weight to make it through.
By the time you're out on the street, they have your money. If tossing out a few cheaters makes the class size a bit smaller for the rest of the semester, that also can't be anything but a net good.
I was talking about coursework and dissertations. Cheating on exams is harder, although not impossible (e.g. bathroom breaks).
it was made abundantly clear
We also make this abundantly clear. But when they get caught (which is already rare), we issue warnings, send them on courses where they learn how properly to quote and so on. 10 strikes and you are out ...
so why not with you?
Because life is short because I don't like tilting at windmills, because I have better things to do, and because I believe in specialisation. I didn't sign up to be a police man, and I have no skills in this direction. Our department actually used to pay somebody to scan rent-a-coder.com and we also used turnitin.com. The former was abandoned and the latter was reduced to a student service (i.e. they could check their coursework before submission)! We were never really given a reason why. One has one's suspicions ...
having those kids expelled
What in your opinion is the cost, financially, in terms of time and reputation, of dealing with all the law-suits that the aggrieved and high-paying parents of the expelled students are sure to launch? (Aka "my precious child a common cheater? Never, we raised her well, it must be your fault!")
A top school [...] could toss out half the current students
LOL. In 1837, Hong Xiuquan failed the imperial examination and was denied his dream to enter the Qing dynasty's civil service. This lead to the Taiping Rebellion, one of the bloodiest conflicts in human history. I really don't want to know what would happen if you fire 1/2 of Harvard students in the time of helicopter parents and tiger mothers, Lord have mercy!
"my precious child a common cheater? Never, we raised her well, it must be your fault!"
Clearly I've been living abroad too long. I had completely forgotten about this aspect of the American psyche.
You're right. No amount of having done something wrong will ever be enough to make something an American's fault. I remember golfing with a guy and hooking a ball into somebody's yard, saying something along the lines of "glad I didn't break that window." He then went on to explain his view that the golf course should pay for things like that, since it would clearly not be his fault if he hit a ball through somebody's window.
Expel that guy's kid, and I can see what you mean. Good luck fighting the good fight!
I don't understand why universities don't do the obvious and simply require the students to defend their work in a viva voce and a written final. When I studied physics (Exeter Uni., 1974-1977) there really was no possibility to cheat because written examinations were open note, theses and project reports had to be defended orally, and the class of one's degree was determined entirely by the finals (oral and written).
require the students to defend their work in a viva voce and a written final.
Time & effort, primarily.
Also equality is a problem. Academics are human too, and tend to have preferences regarding looks, sex, nationalities etc that are hard to ignore. I had female students rub my legs in presentations, wear super short mini-shirts, see-thru top with massive clevage etc. Preventing this is a major advantage of anonymous grading. These days you want to protect yourself against harassment lawsuits. I feel very uncomfortable with non-anonymous examiations even though I agree that from a pedagogic point-of-view they preferable, not just to deal with cheating, but also because they give a much better view of the students strengths and weaknesses, leading to much better feedback.
If your students are behaving like that then run of the mill cheating is the least of your problems. And I can't see how a thesis can be properly defended except in real time, we aren't talking about marking an exam here but getting the student to make it clear that he or she really does understand the work presented (in my case over a hundred pages of experimental design, execution, results, and interpretation) and can explain it and defend it against criticism.
Some universities still require PhD theses to be defended in public, for instance Oslo:
==== begin quote
The PhD degree is conferred on the basis of:
completion of the programme's educational component
the doctoral thesis
the doctoral examination
The doctoral examination consists of the trial lecture and the public defence of the thesis, also known as the disputation.
> But on a site like Khan Academy, where the end goal is to learn something useful, all the user is doing is cheating themself out of the thing they signed up to get. Let them go nuts. If they want to go through a course, but go out of their way to avoid actually learning anything, that shouldn't cause the site owner any grief whatsoever.
Yes, especially since Khan Academy can't totally prevent cheating. For example, one can always create two accounts, one that is used as a draft to get hints and the second one that will enter the correct answer after the hints have been showed to the first account.
> They have spent ten years at school being told that there is one answer – it’s at the back…and don’t look, and don’t copy, because that’s cheating. I mean, outside schools, that’s called collaboration.
I don't think cheating is the big picture problem. I think it's how we think of are tests. Imagine a test designed in such a way that you couldn't cheat - or even that collaboration was ok - what would it look like? Could it be an open note/open book test?
The way I give tests to my students is open book/open note but not open neighbor. I do this because I'm realistic - memorization is not comprehension and whatever they did memorize will be lost 5 minutes after class ends (they have N other subjects that they need to memorize material for - even though I teach a core class they have much more important things to worry about). My tests are long and short answer but not hard - so if you have been comatose for the lectures you won't do well (I have had people fail my tests...). Some teachers have the one notecard rule and think that they are clever tricking the students to memorize the material - they aren't. You may retain some more information - but it's still the core of the problem memorization != comprehension.
Yes I understand some fields do require memorization - such as history or the medical profession. But people don't need to memorize the bubble sort or the binary search algorithms - it's been done a million times and if anyone ever needed it for their career a quick google search would reveal this site [2]. Yes they need to understand it - but they don't need to memorize it.
The problem is that localStorage never gets cleared on its own. So you could have a situation where a hint persists from the last user using the computer (or last session in general), effectively sabotaging the current user's answer(s). An edge case, but possible. SessionStorage would be more suited here.
To my understanding they do not store the hints. They maintain a request log to enable retransmission of requests made during a offline period.
So the hint-thing should not be a thing. On the other hand: The request log requires a user identification method as part of the requests to avoid crediting a "hint-taken" request to the next user in your scenario.
This solution seems to overlook the fact that once you've served something, you lose control of it. If the hint's been served, there's probably another way to find it - even just watching the bytes come over the wire.
I may be missing the point - but why not serve up the hint on a separate request? That way if you're offline, you don't get the hint, and if you're online, they can log that you requested it on the server side.
I think you ARE missing the point. Watching the bytes come over the wire isn't something that they are trying to protect. Just easy cheating in the classroom. They probably received some complains from teachers.
They wanted to keep it working offline, and hints should still be available for students in areas with spotty internet.
It's not Khan Academy responsibility to ensure students don't cheat after all. I mean for all they know the students could have other browsers/computers open.
The post title is a bit misleading, then. The way I see it, it's more like a bug they fixed (since people could legitimately lose their connection and have their hint count wrong) rather than a real no-cheat policy.
Definitely agree, the hints should not get served to the client unless they are requested in the first place. This would obviate much of the "fix" they proposed, and it would seem to me that this would have been the first thought in designing the system anyway.
I'm a professional and I search the web, look in books, and ask other people ... I also learn new stuff every day by doing so. It's pretty stupid that it's considering cheating in school.
Do you want schools to turn our kids into encyclopedias!? I think it's more important that we learn them to share, work together, communicate, find information (source criticism), explore and learn new stuff.
The problem they were having wasn't Googling for the answer, it was pressing the "Hint" button right next to the question. That doesn't require any of the skills you mentioned, so it's probably not very helpful for assessment. It may well be helpful to teach the concept though, which is probably why it's there in the first place.
48 comments
[ 3.7 ms ] story [ 78.3 ms ] threadIt wouldn't make it entirely impossible, but much more complicated.
The blockchain gets round this with consensus.
Wouldn't that be the 'right' way of doing this?
It's not even that much of a cheat, really. I'm surprised they decided to make a full blog post on how they fight such a simple trick (incorrectly, even).
It won't work that much anyway, a simple chrome extension that injects code can overturn their whole thing.
How far can they fight cheating anyway? I remember making a javascript snippet in Chrome devtools that would bruteforce all the crypto questions (because they had predictable patterns, like all sentences involved "Anna" or some similar name). That seems way harder to fight.. will it deserve another blog post?
KA implementing these technical means makes the product more useful to those schools.
Universities survive just fine giving out take-home tests and trusting students not to cheat. Honor codes work, even when there is an actual useful reason to cheat.
But on a site like Khan Academy, where the end goal is to learn something useful, all the user is doing is cheating themself out of the thing they signed up to get. Let them go nuts. If they want to go through a course, but go out of their way to avoid actually learning anything, that shouldn't cause the site owner any grief whatsoever.
"go out of their way to avoid actually learning anything, that shouldn't cause the site owner any grief whatsoever."
Obviously, Khan Academy staff has a different take on this problem and that's great. Online education should strive to cater to cheaters, too.
The universities accept this tacitly for two reasons.
1. They need the income from the students. If they'd fire every student caught cheating, they'd go bankrupt pretty quickly.
2. It's an extreme amount of administrative work for the professors to deal with. In my experience, I need to invest a day's worth of work per cheating student, all things considered. I really have better things to do in life.
To put it pithily: the students pretend not to cheat, and the universities pretend not to notice.
This is nothing new, students have been cheating since time immemorial, the world keeps on spinning ... there are lot's of stories about cheating in what may be the oldest exam system in the world, the Chinese Imperial examination. (As an aside, if you want an alround fascinating story that involves the Imperial examination, check out the history of the "Taiping Rebellion".)
As you say, the incentive and resource for the university to do something about this just isn't there.
Very few people talk about what it feels like not to cheat and why students do it. So here goes:
I made it through college without cheating. My university actually managed to scare me from doing so with freshmen orientation. I focused on studying and making sure I took advantage of walk-in hours and student teachers. For a while I thought everyone was doing this. Then, as weeks passed in each semester the students got more and more brazen. During my years of school I was outright asked to share my test (sometimes during a test), I was offered payment and I was made to feel ostracized when I didn’t “help”. It sucked.
The worst part came when I would meet other honors students and find out how they got their high scores. I was surprised at how many of them cheated, to what degree they would cheat and how little they knew about their course work. This is when I found that there was no incentive not to cheat. Honors students got scholarships and were often given some very good internship opportunities.
Why would anyone not cheat? No one cares if you do. You’ll have an easier time. It doesn’t hurt your career. It also does wonders for your social life. My final two years I pretended to have low scores on tests so as not to be asked to help cheat. I kept my grades a secret and would often not look at graded papers in class. I stopped talking about any of my classes with anyone that was in my major.
I honestly don't think what I did should've been considered cheating, and I would do it that way again today- in fact, don't our jobs work just about the same way? But I'm all in favor of strict rules around cheating on testing or final projects. Those are where you demonstrate what you personally have learned, I don't think it should matter how you got to that knowledge, but you should have it.
It was pretty rare that I heard of anybody doing so.
I'd recommend going ahead and having those kids expelled, or at least starting the process and taking it as far as you can go (before the university has to step in and make a point of stopping it). Change has to begin somewhere, so why not with you?
Edited to add: I don't really buy the economic argument either. A top school has a pretty long list of kids wanting to get in. They could toss out half the current students each year and never run in to an issue filling back up the next year. If anything, it would only serve to improve their reputation as an "elite" school where you really do have to pull your weight to make it through.
By the time you're out on the street, they have your money. If tossing out a few cheaters makes the class size a bit smaller for the rest of the semester, that also can't be anything but a net good.
Clearly I've been living abroad too long. I had completely forgotten about this aspect of the American psyche.
You're right. No amount of having done something wrong will ever be enough to make something an American's fault. I remember golfing with a guy and hooking a ball into somebody's yard, saying something along the lines of "glad I didn't break that window." He then went on to explain his view that the golf course should pay for things like that, since it would clearly not be his fault if he hit a ball through somebody's window.
Expel that guy's kid, and I can see what you mean. Good luck fighting the good fight!
Also equality is a problem. Academics are human too, and tend to have preferences regarding looks, sex, nationalities etc that are hard to ignore. I had female students rub my legs in presentations, wear super short mini-shirts, see-thru top with massive clevage etc. Preventing this is a major advantage of anonymous grading. These days you want to protect yourself against harassment lawsuits. I feel very uncomfortable with non-anonymous examiations even though I agree that from a pedagogic point-of-view they preferable, not just to deal with cheating, but also because they give a much better view of the students strengths and weaknesses, leading to much better feedback.
Some universities still require PhD theses to be defended in public, for instance Oslo: ==== begin quote
The PhD degree is conferred on the basis of:
completion of the programme's educational component
the doctoral thesis
the doctoral examination
The doctoral examination consists of the trial lecture and the public defence of the thesis, also known as the disputation.
=== end quote
Yes, especially since Khan Academy can't totally prevent cheating. For example, one can always create two accounts, one that is used as a draft to get hints and the second one that will enter the correct answer after the hints have been showed to the first account.
I want to quote Sir Ken Robinson [1]:
> They have spent ten years at school being told that there is one answer – it’s at the back…and don’t look, and don’t copy, because that’s cheating. I mean, outside schools, that’s called collaboration.
I don't think cheating is the big picture problem. I think it's how we think of are tests. Imagine a test designed in such a way that you couldn't cheat - or even that collaboration was ok - what would it look like? Could it be an open note/open book test?
The way I give tests to my students is open book/open note but not open neighbor. I do this because I'm realistic - memorization is not comprehension and whatever they did memorize will be lost 5 minutes after class ends (they have N other subjects that they need to memorize material for - even though I teach a core class they have much more important things to worry about). My tests are long and short answer but not hard - so if you have been comatose for the lectures you won't do well (I have had people fail my tests...). Some teachers have the one notecard rule and think that they are clever tricking the students to memorize the material - they aren't. You may retain some more information - but it's still the core of the problem memorization != comprehension.
Yes I understand some fields do require memorization - such as history or the medical profession. But people don't need to memorize the bubble sort or the binary search algorithms - it's been done a million times and if anyone ever needed it for their career a quick google search would reveal this site [2]. Yes they need to understand it - but they don't need to memorize it.
[1] http://edtechnow.net/2012/01/20/sir-ken-robinson/
[2] http://rosettacode.org/wiki/Binary_search
It's quite effective in its regard, unless you search for answer with a phone or something.
So the hint-thing should not be a thing. On the other hand: The request log requires a user identification method as part of the requests to avoid crediting a "hint-taken" request to the next user in your scenario.
I may be missing the point - but why not serve up the hint on a separate request? That way if you're offline, you don't get the hint, and if you're online, they can log that you requested it on the server side.
They wanted to keep it working offline, and hints should still be available for students in areas with spotty internet.
It's not Khan Academy responsibility to ensure students don't cheat after all. I mean for all they know the students could have other browsers/computers open.
Do you want schools to turn our kids into encyclopedias!? I think it's more important that we learn them to share, work together, communicate, find information (source criticism), explore and learn new stuff.