This is the correct take. To contrast the Terance Tao piece from earlier (https://news.ycombinator.com/item?id=46017972), AI research tools are increasingly useful if you're a competent researcher that can judge the output and detect BS. You can't, however, become a Terence Tao by asking AI to solve your homework.
So, in learning environments we might not have an option but to open the floodgates to AI use, but abandon most testing techniques that are not, more or less, pen and paper, in-person. Use AI as much as you want, but know that as a student you'll be answering tests armed only with your brain.
I do pity English teachers that have relied on essays to grade proficiency for hundreds of years. STEM fields has an easier way through this.
> AI research tools are increasingly useful if you're a competent researcher that can judge the output and detect BS.
This assumes we even need more Terence Taos by the time these kids are old enough. AI has gone from being completely useless to solving challening math problems in less than 5 years. That trajectory doesn't give me much hope that education will matter at all in a few years.
One of my students recently came to me with an interesting dilemma. His sister had written (without AI tools) an essay for another class, and her teacher told her that an "AI detection tool" had classified it as having been written by AI with "100% confidence". He was going to give her a zero on the assignment.
Putting aside the ludicrous confidence score, the student's question was: how could his sister convince the teacher she had actually written the essay herself? My only suggestion was for her to ask the teacher to sit down with her and have a 30-60 minute oral discussion on the essay so she could demonstrate she in fact knew the material. It's a dilemma that an increasing number of honest students will face, unfortunately.
My high school history teacher gave me an F on my term paper. I asked him why, and he said it was "too good" for a high school student. The next day I dumped on his desk all the cited books, which were obscure and in my dad's extensive collection. He capitulated, but disliked me ever since.
The new trick being used by some professors in college classes is to mandate a specific document editor with a history view. If the document has unusual copy/paste patterns or was written in unusual haste then they may flag it. That being said, they support use of ai in the class and have confidence the student is not able to one shot the assignment with ai.
I suspect this is going in the wrong direction. Telling a sandboxed AI to have a long conversation with a student to ensure they actually know what they're talking about, while giving minimal hints away, seems like the scalable solution. Let students tackle the material however they will, knowing that they will walk into class the next day and be automatically grilled on it, unaided. There's no reason a similar student couldn't have their essay fed into the AI and then asked questions about what they meant on it.
Once this becomes routine the class can become e.g. 10 minutes conversation on yesterday's topic, 40 minutes lecturing and live exercises again. Which is really just reinventing the "daily quiz" approach, but again the thing we are trying to optimize for is compliance.
Always stunned by how much teachers can accuse without proof and invert the "innocent until proven guilty".
Honestly, students should have a course in "how the justice system works" (or at least should work). So should the teachers.
Student unions and similar entities should exist and be ready to intervene to help students in such situations.
This is nothing new, AI will just make this happen more often, revealing how stupid so many teachers are. But when someone spent thousands for a tool, which purports to be reliable, and is so quick to use, how can an average person resist it? The teacher is as lazy as the cheaters they intend to catch.
I wrote a paper about building web applications in 10th grade a long time ago. When class was out the teacher asked me to stay for a minute after everybody left. He asked in disbelief, “did you really write that paper?”
I could see why he didn’t, so I wasn’t offended or defensive and started to tell him the steps required to build web apps and explained it in a manner he could understand using analogies. Towards the end of our conversation he could see I both knew about the topic and was enthusiastic about it. I think he was still a bit shocked that I wrote that paper, but he could tell from the way I talked about it that it was authentic.
It will be interesting to see how these situations evolve as AI gets even better. I suspect assessment will be more manual and in-person.
It's not that hard to prove that you did the work and not an AI. Show your work. Explain to the teacher why you wrote what you did, why that particular approach to the narrative appealed to you and you chose that as the basis for your work. Show an outline on which the paper was based. Show rough drafts. Explain how you revised the work, where you found your references, and why you retained some sources in the paper and not others.
To wit, show the teacher that YOU did the work and not someone else. If the teacher is not willing to do this with every student they accuse of malfeasance, they need to find another job. They're lazy as hell and suck at teaching.
> My only suggestion was for her to ask the teacher to sit down with her and have a 30-60 minute oral discussion on the essay so she could demonstrate she in fact knew the material.
This sounds like, a good solution? It’s the exception case, so shouldn’t be constant (false positives), although I suppose this fails if everyone cheats and everyone wants to claim innocence.
I wouldn't mind seeing education return to its roots of being about learning instead of credentialization. In an age where having a degree is increasingly meaningless in part due to many places simply becoming thinly veiled diploma treadmills (which are somehow nonetheless accredited), this is probably more important than ever. This is doubly so if the AI impact extremists end up being correct.
So why is the issue you described an issue? Because it's about a grade. And the reason that's relevant is because that credential will then be used to determine where she can to to university which, in turn, is a credential that will determine her breadth of options for starting her career, and so on. But why is this all done by credentials instead of simple demonstrations of skill? What somebody scored in a high school writing class should matter far less than the output somebody is capable of producing when given a prompt and an hour in a closed setting. This is how you used to apply to colleges. Here [1], for instance, is Harvard's exam from 1869. If you pass it, you're in. Simple as that.
Obviously this creates a problem of institutions starting to 'teach the test', but with sufficiently broad testing I don't see this as a problem. If a writing class can teach somebody to write a compelling essay based on an arbitrary prompt, then that was simply a good writing class! As an aside this would also add a major selling point to all of the top universities that offer free educational courses online. Right now I think 'normal' people are mostly disinterested in those because of the lack of widely accepted credentials, which is just so backwards - people are actively seeking to maximize credentials over maximizing learning.
This is one of the very few places I think big tech in the US has done a great job. Coding interviews can be justifiably critiqued in many ways, but it's still a much better system than raw credentialization.
Yeah, this, but also as an adult; When you are a non-native speaker and you use AI to make things more concise and correct. The detector will go off. People may find some wording "AI-ish" (even though I replaced em-dashes with commas and told it to "avoid American waiter-like enthusiasm"). My reaction is: Ok. you want my original? Which is much harder to read and uses 2x the amount of words? Fine.
I mean, what is the problem? It's my report! I know all the ins and outs, I take full responsibility for it. I'm the one taking this to the board of directors who will grill me on all the details. I'm up for it. So why is this so "not done"? Why do you assume I let the AI do the "thinking"? I'm appalled by your lack of trust in me.
I've had the same problem online for years, when I translate something people presume I am using Google Translate (even though in one case said language isn't on Google Translate — I checked!)... Or got the answer off Wikipedia.
One of the funniest things was being accused of plagiarising Wikipedia, when I'd actually written most of the Wikipedia article on said subject. The irony... Wikipedia doesn't just use unpaid labour, it ends up undermining the people who wrote it.
The oral discussion does not scale well in large classes. The solution is to stop using essays for evaluation, relying on (supervised) examinations instead.
Of course, there will be complaints from many students. However, as a prof for decades, I can say that some will prefer an exam-based solution. This includes the students who are working their way through university and don't have much time for busy-work, along with students who write their essays themselves and get lower grades than those who do not.
The real problem here is (in this case) lazy teachers. These kind of tools should only be used to flag potential AI generation. If the teacher read the essay and thought it reflected standard work for this student, then all would be fine. Instead they are just running the tool first to be lazy and taking the tool as gospel.
This reminds me of when GPS routing devices first came onto the scene. Lots of people drove right into a lake or ocean because the device said keep going straight. (because of poorly classified multi-modal routing data)
That's an interesting point. It seems it makes cheaper to provide knowledge but more expensive to have individual assessments.
I think AI got me some brain rot as I concern to finish stuff on time and I can't bare to spend brain energy on that (and spend on it anyway because AI sucks)
This stuff is getting more pervasive too. I'm working on my Master's degree right now and any code I submit, I make sure it has spelling mistakes and make it god awful because I don't want to get flagged by some 3rd party utility that checks if it was AI generated or copied from someone else.
My son recently told me his teacher used him as an example for the class as someone who wrote a good piece himself. Teacher accused all the other students of using AI.
He also told me that he had in fact used AI, but asked AI multiple times to simplify the text, and he had entered the simplified version. He liked the first version best, but was aware his teacher would consider it written by AI.
Years ago in the 90's my brother wrote a short story for a fourth grade assignment and the teacher accused him of plagiarism because how could a 9 year old come up with such a vivid and elaborate story so he received a zero. I forget the details but my mother went to the school making a big stink and eventually had the zero changed to an A.
It creates another layer of arbitrary gate-keeping. I experience this in interviews too. If I need to think about the low latency response I can give to derive djikstras algo verbatim, I get accused of reading notes on another monitor, not someone who studied for success.
It is beginning to become an awful situation where these companies are selling tools that undermine the student. Education is suppose to be the great equalizer in society, not another toggle or tool for oppression.
This couldn’t have happened at a better time. When I was young my parents found a schooling system that had minimal homework so I could play around and live my life. I’ve moved to a country with a lot less flexibility. Now when my kids will soon be going to school, compulsory homework will be obsolete.
Zero homework grades will be ideal. Looking forward to this.
In my CS undergrad I had Doug Lea as a professor, really fantastic professor (best teacher I have ever had, bar none). He had a really novel way to handle homework hand ins, you had to demo the project. So you got him to sit down with you, you ran the code, he would ask you to put some inputs in (that were highly likely to be edge cases to break it). Once that was sufficient, he would ask you how you did different things, and to walk him through your code. Then when you were done he told you to email the code to him, and he would grade it. I am not sure how much of this was an anti-cheating device, but it required that you knew the code you wrote and why you did it for the project.
I think that AI has the possibility of weakening some aspects of education but I agree with Karpathy here. In class work, in person defenses of work, verbal tests. These were corner stones of education for thousands of years and have been cut out over the last 50 years or so outside of a few niche cases (Thesis defense) and it might be a good thing that these come back.
Maybe as a society we can take some of the productivity gains from AI and funnel them into moving teaching away from scantrons and formulaic essays. I want to be optimistic.
The TL:DR of every "AI vs Schools, what should teachers do?" article boils down to exactly this: Talk with the students 1-1. You can fake an essay, you can't fake a conversation about the topic at hand.
15+ years ago I was doing a CS undergrad (or Bachelors? not sure how it translates) at the local uni in a small EU Country and this approach was the standard across all subjects as part of 'lab work'. There were people there to do that, not the prof himself, but approach was exactly the same. And after a few months they had a really good picture on what level everyone is ect.
On the other hand, I had a neighbour ask me if he can make his 1 month apprenticeship when he finished his 3rd year of CS High School (eg ~18 years old, 3 of 4 years of 'CS trade school') 6 months ago or so. I was totally gobsmacked by his lack of basic understanding of how computers work, I am confident that he did not confidentially know the difference between a file and a folder. But he was very confident in the AI slop he produced. I had a grand plan of giving him tasks that would show him the pitfalls of AI -> no need for that, he blindly copied whatever AI gave him (he did not figure out Claude Code exsists), even when the results were very visibly bad - even from afar. I tried explaining stuff to him to no avail. I know this is a sample size of 1, but damn, I did not expect it to be that bad.
Strongly agree. I was involved with several CS lectures in the past ~10 years that did not require a final exam, and we always did a 1:1 session between student and tutor in which the tutor asked the student detailed questions about their past exercise sheet solutions. Over the years, I estimate that I conducted about 100 of such 1:1s. It was always obvious when the students did not write the code themselves. They couldn't really explain their design process, they didn't encounter the edge cases themselves during testing, and you couldn't discuss possible improvements with them.
This doesn't adress the point that AI can replace going to school. AI can be your perfect personal tutor to help you learn thing 1:1. Needing to have a teacher and prove to them that you know what they teached will become a legacy concept. That we have an issue of AI cheating at school is in my eyes a temporary issue.
It's a fair question, but there's maybe a bit of US defaultism baked in? If I look
back at my exams in school
they were mostly closed-book written + oral examination, nothing would really need to change.
A much bigger question is what to teach assuming we get models much more powerful than those we have today. I'm still confident there's an irreducible hard core in most subjects that's well worth knowing/training, but it might take some soul searching.
"You have to assume that any work done outside classroom has used AI."
That is just such a wildly cynical point of view, and it is incredibly depressing. There is a whole huge cohort of kids out there who genuinely want to learn and want to do the work, and feel like using AI is cheating. These are the kids who, ironically, AI will help the most, because they're the ones who will understand the fundamentals being taught in K-12.
I would hope that any "solution" to the growing use of AI-as-a-crutch can take this cohort of kids into consideration, so their development isn't held back just to stop the less-ethical student from, well, being less ethical.
It seems like a good path forward is to somewhat try to replicate the idea of "once you can do it yourself, feel free to use it going forward" (knowing how various calculator operations work before you let it do it for you).
I'm curious if we instead gave students an AI tool, but one that would intentionally throw in wrong things that the student had to catch. Instead of the student using LLMs, they would have one paid for by the school.
This is more brainstorming then a well thought-out idea, but I generally think "opposing AI" is doomed to fail. If we follow a montessori approach, kids are naturally inclined to want to learn thing, if students are trying to lie/cheat, we've already failed them by turning off their natural curiosity for something else.
> The students remain motivated to learn how to solve problems without AI because they know they will be evaluated without it in class later.
Learning how to prepare for in-class tests and writing exercises is a very particular skillset which I haven't really exercised a lot since I graduated.
Never mind teaching the humanities, for which I think this is a genuine crisis, in class programming exams are basically the same thing as leetcode job interviews, and we all know what a bad proxy those are for "real" development work.
I’ve been following this approach since last school year. I focus on in-class work and home-time is for reading and memorization. My classmates still think classrooms are for lecturing, but it's coming. The paper-and-pen era is back to school!
I think legacy schooling just needs to be reworked. Kids should be doing way more projects that demonstrate the integration of knowledge and skills, rather than focusing so much energy on testing and memorization. There's probably a small core of things that really must be fully integrated and memorized, but for everything else you should just give kids harder projects which they're expected to solve by leveraging all the tools at their disposal. Focus on teaching kids how to become high-agency beings with good epistemics and a strong math core. Give them experiments and tools to play around and actually understand how things work. Bring back real chemistry labs and let kids blow stuff up.
The key issue with schools is that they crush your soul and turn you into a low-agency consumer of information within a strict hierarchy of mind-numbing rules, rather than helping you develop your curiosity hunter muscles to go out and explore. In an ideal world, we would have curated gardens of knowledge and information which the kids are encouraged to go out and explore. If they find some weird topic outside the garden that's of interest to them, figure out a way to integrate it.
I don't particularly blame the teachers for the failings of school though, since most of them have their hands tied by strict requirements from faceless bureaucrats.
But testing and paper assessments are cheap and feasible for mass education. There are only so many workshop projects you can have before you run out of budget.
Increase spending on schools by an order of magnitude and it would be possible.
All of schooling breaks down to costs and society’s willingness and desire to invest in child nutrition, education, and training.
We simply do not even have the wherewithal to have the conversation about it, without getting blackholed by cultural minefields and assumptions of child rearing, parental responsibility, morality and religion.
People have been saying that we "focus too much on memorization" for as long as I have been alive. To be honest, I don't really think that is true, if anything we don't focus enough on memorization nowadays since people leave school without knowing basic things about the world whether in science or in history. Knowing things allows one to make connections and see things in a different way that you simply cannot get if you rely on the internet or LLMs or whatever to look everything up.
I just went and had a flutter at being a high school math teacher. I went in saying 'I never used math to create until my honours year, I want different for my students'.
I soon changed my mind; I think those of us who become expert have often have really rich memories of a project where we learnt so much, but we just don't remember episodically all the accumulated learning that happened in boring classrooms to enable the project-induced higher order synthesis.
Having had some experience teaching and designing labs and evaluating students in my opinion there is basically no problem that can't be solved with more instructor work.
The problem is that the structure pushes for teaching productivity which basically directly opposes good pedagogy at this point in the optimization.
Some specifics:
1. Multiple choice sucks. It's obvious that written response better evaluates students and oral is even better. But multiple choice is graded instantly by a computer. Written response needs TAs. Oral is such a time sink and needs so many TAs and lots of space if you want to run them in parallel.
1.5 Similarly having students do things on computers is nice because you don't have to print things and even errors in the question can be fixed live and you can ask students to refresh the page. But if the chatbots let them cheat too easily on computers doing hand written assesments sucks cause you have to go arrange for printing and scanning.
2. Designing labs is a clear LLM tradeoff. Autograded labs with testbenches and fill in the middle style completetions or API completetions are incredibly easy to grade. You just pull the commit before some specific deadline and run some scripts.
You can do 200 students in the background when doing other work its so easy. But the problem is that LLMS are so good at fill in the middle and making testbenches pass.
I've actually tried some more open ended labs before and its actually very impressive how creative students are. They are obviously not LLMs there is this diversity in thought and simplicity of code that you do not get with ChatGPT.
But it is ridiculously time consuming to pull people's code and try to run open ended testbenches that they have created.
3. Having students do class presentations is great for evaluating them. But you can only do like 6 or 7 presentations in a 1 hr block. You will need to spend like a week even in a relatively small class.
4. What I will say LLMs are fun for are having students do open ended projects faster with faster iterations. You can scope creep them if you expect expect to use AI coding.
I know a teacher who basically only does open questions but since everything is digital nowadays students just use tools like Cluely [0] that run on the background and provide answers.
Since the testing tool they use does notice and register 'paste'-events they've resorted to simply assigning 0 points to every answer that was pasted.
A few of us have been telling her to move to in-class testing etc. but like you also notice everything in the school organization pushes for teaching productivity so this does require convincing management / school board etc. which is a slow(er) process.
I think part of the reason AI is having such a negative effect on schools in particular is because of how many education processes are reliant on an archaic, broken way of "learning." So much of it is focused upon memorization and regurgitation of information (which AI is unmatched at doing).
School is packed with inefficiency and busywork that is completely divorced from the way people learn on their own. In fact, it's pretty safe to say you could learn something about 10x by typing it into an AI chat bot and having it tailor the experience to you.
> So much of it is focused upon memorization and regurgitation of information, which AI is unmatched at doing.
This applies both to education, and to what people need to know to do work. Knowing all the written stuff is less valuable. Automated tools can been able to look it up since the Google era. Now they can work with what they look up.
There was a time when programmers poured over Fundamental Algorithms. No one does that today. When needed, you find existing code that does that stuff. Probably better than you could write. Who codes a hash table today?
Teachers worry about AI because they do not just care about memorization. Before AI, being able to write cohesive essays about a subject is a good proxy to prove your understanding beyond simple memorization. Now it's gone.
A lazy, irresponsible teacher who only cares about memorization will just grade students via in-class multi choices tests exclusively and call it a day. They don't need to worry about AI at all.
The way you learn is totally different from the way a novice learns; they don't have a vast memorised store of knowledge, let alone the connected structure over that memorised knowledge. When you learn something, it gets incoporated thanks to these foundations.
I did a lot of my blog and book writing before these AI tools, but now I show my readers images of handwritten notes and drafts (more out of interest than demonstrating proof of work).
Here is my proposal for AI in schools: raise the bar dramatically. Rather than trying to prevent kids from using AI, just raise the expectations of what they should accomplish with it. They should be setting really lofty goals rather than just doing the same work with less effort.
This is great for a “capstone project” at the end of a degree. But along the way, you have to master sub tasks and small skills in order to build on them later to accomplish lofty goals. So you need to learn the basics first. But AI is really good at helping you cheat on the basics without learning. So we still need to get them to the point of being able to use AI intelligently
I made a tool for this! It's an essay writing platform that tracks the edits and keystrokes rather than the final output, so its AI detection accuracy is _much_ higher than other tools:
https://collie.ink/
As a teacher, I try to keep an open mind, but consistently I can find out in 5 minutes of talking to a student if they understand the material. I might just go all in for the oral exams.
This is exactly why I'm focusing on job readiness and remediation rather than the education system. I think working all this out is simply too complex for a system with a lot of vested interest and that doesn't really understand how AI is evolving. There's an arms race between students, teachers, and institutions that hire the students.
It's simply too complex to fix. I think we'll see increased investment by corporates who do keep hiring on remediating the gaps in their workforce.
Most elite institutions will probably increase their efforts spent on interviewing including work trials. I think we're already seeing this with many of the elite institutions talking about judgment, emotional intelligence critical thinking as more important skills.
My worry is that hiring turns into a test of likeability rather than meritocracy (everyone is a personality hire when cognition is done by the machines)
Source: I'm trying to build a startup (Socratify) a bridge for upskilling from a flawed education system to the workforce for early stage professionals
How about just dispense with the AI nonsense in education and go to totally in-person, closed-book, manually-written, proctored exams? No homework, no assignments, no projects. Just pure mind-to-paper writing in a bare room under the eye of an examiner. Those that want to will learn and will produce intelligent work regardless of setting.
this is a very American issue. In my entire student career in Italy, home assignments were never graded. Maybe you had a project or two through university, but otherwise I got all my grades for onsite tests.
When I was a kid in Italy in the late 80s early 90s the elentary school teachers also did "interrogazione" at the blackboard. That is individual kids would go up to the blackboard and get asked questions front of the rest of the class.
Not sure if those two were just old school (they'd occasionally hit us/ pull ears too) but damn was I ahead of all the other kids when I came back to the USA
Also the two teachers had the same class of kids for all of elementary school, teaching 1st through 5th grades sequentially
So they got to know the kids quite well
103 comments
[ 4.5 ms ] story [ 78.7 ms ] threadI'm not minimizing Karpathy in any way, but this is obviously the right way to do this.
So, in learning environments we might not have an option but to open the floodgates to AI use, but abandon most testing techniques that are not, more or less, pen and paper, in-person. Use AI as much as you want, but know that as a student you'll be answering tests armed only with your brain.
I do pity English teachers that have relied on essays to grade proficiency for hundreds of years. STEM fields has an easier way through this.
This assumes we even need more Terence Taos by the time these kids are old enough. AI has gone from being completely useless to solving challening math problems in less than 5 years. That trajectory doesn't give me much hope that education will matter at all in a few years.
Putting aside the ludicrous confidence score, the student's question was: how could his sister convince the teacher she had actually written the essay herself? My only suggestion was for her to ask the teacher to sit down with her and have a 30-60 minute oral discussion on the essay so she could demonstrate she in fact knew the material. It's a dilemma that an increasing number of honest students will face, unfortunately.
Once this becomes routine the class can become e.g. 10 minutes conversation on yesterday's topic, 40 minutes lecturing and live exercises again. Which is really just reinventing the "daily quiz" approach, but again the thing we are trying to optimize for is compliance.
Honestly, students should have a course in "how the justice system works" (or at least should work). So should the teachers.
Student unions and similar entities should exist and be ready to intervene to help students in such situations.
This is nothing new, AI will just make this happen more often, revealing how stupid so many teachers are. But when someone spent thousands for a tool, which purports to be reliable, and is so quick to use, how can an average person resist it? The teacher is as lazy as the cheaters they intend to catch.
I could see why he didn’t, so I wasn’t offended or defensive and started to tell him the steps required to build web apps and explained it in a manner he could understand using analogies. Towards the end of our conversation he could see I both knew about the topic and was enthusiastic about it. I think he was still a bit shocked that I wrote that paper, but he could tell from the way I talked about it that it was authentic.
It will be interesting to see how these situations evolve as AI gets even better. I suspect assessment will be more manual and in-person.
To wit, show the teacher that YOU did the work and not someone else. If the teacher is not willing to do this with every student they accuse of malfeasance, they need to find another job. They're lazy as hell and suck at teaching.
This sounds like, a good solution? It’s the exception case, so shouldn’t be constant (false positives), although I suppose this fails if everyone cheats and everyone wants to claim innocence.
So why is the issue you described an issue? Because it's about a grade. And the reason that's relevant is because that credential will then be used to determine where she can to to university which, in turn, is a credential that will determine her breadth of options for starting her career, and so on. But why is this all done by credentials instead of simple demonstrations of skill? What somebody scored in a high school writing class should matter far less than the output somebody is capable of producing when given a prompt and an hour in a closed setting. This is how you used to apply to colleges. Here [1], for instance, is Harvard's exam from 1869. If you pass it, you're in. Simple as that.
Obviously this creates a problem of institutions starting to 'teach the test', but with sufficiently broad testing I don't see this as a problem. If a writing class can teach somebody to write a compelling essay based on an arbitrary prompt, then that was simply a good writing class! As an aside this would also add a major selling point to all of the top universities that offer free educational courses online. Right now I think 'normal' people are mostly disinterested in those because of the lack of widely accepted credentials, which is just so backwards - people are actively seeking to maximize credentials over maximizing learning.
This is one of the very few places I think big tech in the US has done a great job. Coding interviews can be justifiably critiqued in many ways, but it's still a much better system than raw credentialization.
[1] - https://graphics8.nytimes.com/packages/pdf/education/harvard...
I mean, what is the problem? It's my report! I know all the ins and outs, I take full responsibility for it. I'm the one taking this to the board of directors who will grill me on all the details. I'm up for it. So why is this so "not done"? Why do you assume I let the AI do the "thinking"? I'm appalled by your lack of trust in me.
One of the funniest things was being accused of plagiarising Wikipedia, when I'd actually written most of the Wikipedia article on said subject. The irony... Wikipedia doesn't just use unpaid labour, it ends up undermining the people who wrote it.
It turned out he ran it through a plagiarism detector and multiple lines of code where identical to lines in their database.
It was very silly because there’s a lot of boiler plate code in win32 projects
Of course, there will be complaints from many students. However, as a prof for decades, I can say that some will prefer an exam-based solution. This includes the students who are working their way through university and don't have much time for busy-work, along with students who write their essays themselves and get lower grades than those who do not.
but if u talk like this boss i had, then obv ur a human, kthx
Great incentives. /s
She can't because she didn't write the essay herself, obviously.
This reminds me of when GPS routing devices first came onto the scene. Lots of people drove right into a lake or ocean because the device said keep going straight. (because of poorly classified multi-modal routing data)
The great thing about AI is that with a bit of imagination it can be used to amplify teachers too.
In this case, yes, you need to do a viva voce to convince the teacher (though I suspect they should be able to get fairly confident in 10-15 minutes).
But you could also have students convince an AI (probably in a proctored space?) if you need to scale this approach out.
I think AI got me some brain rot as I concern to finish stuff on time and I can't bare to spend brain energy on that (and spend on it anyway because AI sucks)
He also told me that he had in fact used AI, but asked AI multiple times to simplify the text, and he had entered the simplified version. He liked the first version best, but was aware his teacher would consider it written by AI.
Guess the teachers have already lost...
It is beginning to become an awful situation where these companies are selling tools that undermine the student. Education is suppose to be the great equalizer in society, not another toggle or tool for oppression.
Zero homework grades will be ideal. Looking forward to this.
I think that AI has the possibility of weakening some aspects of education but I agree with Karpathy here. In class work, in person defenses of work, verbal tests. These were corner stones of education for thousands of years and have been cut out over the last 50 years or so outside of a few niche cases (Thesis defense) and it might be a good thing that these come back.
On the other hand, I had a neighbour ask me if he can make his 1 month apprenticeship when he finished his 3rd year of CS High School (eg ~18 years old, 3 of 4 years of 'CS trade school') 6 months ago or so. I was totally gobsmacked by his lack of basic understanding of how computers work, I am confident that he did not confidentially know the difference between a file and a folder. But he was very confident in the AI slop he produced. I had a grand plan of giving him tasks that would show him the pitfalls of AI -> no need for that, he blindly copied whatever AI gave him (he did not figure out Claude Code exsists), even when the results were very visibly bad - even from afar. I tried explaining stuff to him to no avail. I know this is a sample size of 1, but damn, I did not expect it to be that bad.
A much bigger question is what to teach assuming we get models much more powerful than those we have today. I'm still confident there's an irreducible hard core in most subjects that's well worth knowing/training, but it might take some soul searching.
That is just such a wildly cynical point of view, and it is incredibly depressing. There is a whole huge cohort of kids out there who genuinely want to learn and want to do the work, and feel like using AI is cheating. These are the kids who, ironically, AI will help the most, because they're the ones who will understand the fundamentals being taught in K-12.
I would hope that any "solution" to the growing use of AI-as-a-crutch can take this cohort of kids into consideration, so their development isn't held back just to stop the less-ethical student from, well, being less ethical.
I'm curious if we instead gave students an AI tool, but one that would intentionally throw in wrong things that the student had to catch. Instead of the student using LLMs, they would have one paid for by the school.
This is more brainstorming then a well thought-out idea, but I generally think "opposing AI" is doomed to fail. If we follow a montessori approach, kids are naturally inclined to want to learn thing, if students are trying to lie/cheat, we've already failed them by turning off their natural curiosity for something else.
Learning how to prepare for in-class tests and writing exercises is a very particular skillset which I haven't really exercised a lot since I graduated.
Never mind teaching the humanities, for which I think this is a genuine crisis, in class programming exams are basically the same thing as leetcode job interviews, and we all know what a bad proxy those are for "real" development work.
The key issue with schools is that they crush your soul and turn you into a low-agency consumer of information within a strict hierarchy of mind-numbing rules, rather than helping you develop your curiosity hunter muscles to go out and explore. In an ideal world, we would have curated gardens of knowledge and information which the kids are encouraged to go out and explore. If they find some weird topic outside the garden that's of interest to them, figure out a way to integrate it.
I don't particularly blame the teachers for the failings of school though, since most of them have their hands tied by strict requirements from faceless bureaucrats.
All of schooling breaks down to costs and society’s willingness and desire to invest in child nutrition, education, and training.
We simply do not even have the wherewithal to have the conversation about it, without getting blackholed by cultural minefields and assumptions of child rearing, parental responsibility, morality and religion.
I soon changed my mind; I think those of us who become expert have often have really rich memories of a project where we learnt so much, but we just don't remember episodically all the accumulated learning that happened in boring classrooms to enable the project-induced higher order synthesis.
The problem is that the structure pushes for teaching productivity which basically directly opposes good pedagogy at this point in the optimization.
Some specifics:
1. Multiple choice sucks. It's obvious that written response better evaluates students and oral is even better. But multiple choice is graded instantly by a computer. Written response needs TAs. Oral is such a time sink and needs so many TAs and lots of space if you want to run them in parallel.
1.5 Similarly having students do things on computers is nice because you don't have to print things and even errors in the question can be fixed live and you can ask students to refresh the page. But if the chatbots let them cheat too easily on computers doing hand written assesments sucks cause you have to go arrange for printing and scanning.
2. Designing labs is a clear LLM tradeoff. Autograded labs with testbenches and fill in the middle style completetions or API completetions are incredibly easy to grade. You just pull the commit before some specific deadline and run some scripts.
You can do 200 students in the background when doing other work its so easy. But the problem is that LLMS are so good at fill in the middle and making testbenches pass.
I've actually tried some more open ended labs before and its actually very impressive how creative students are. They are obviously not LLMs there is this diversity in thought and simplicity of code that you do not get with ChatGPT.
But it is ridiculously time consuming to pull people's code and try to run open ended testbenches that they have created.
3. Having students do class presentations is great for evaluating them. But you can only do like 6 or 7 presentations in a 1 hr block. You will need to spend like a week even in a relatively small class.
4. What I will say LLMs are fun for are having students do open ended projects faster with faster iterations. You can scope creep them if you expect expect to use AI coding.
Since the testing tool they use does notice and register 'paste'-events they've resorted to simply assigning 0 points to every answer that was pasted.
A few of us have been telling her to move to in-class testing etc. but like you also notice everything in the school organization pushes for teaching productivity so this does require convincing management / school board etc. which is a slow(er) process.
[0] https://cluely.com/
School is packed with inefficiency and busywork that is completely divorced from the way people learn on their own. In fact, it's pretty safe to say you could learn something about 10x by typing it into an AI chat bot and having it tailor the experience to you.
This applies both to education, and to what people need to know to do work. Knowing all the written stuff is less valuable. Automated tools can been able to look it up since the Google era. Now they can work with what they look up.
There was a time when programmers poured over Fundamental Algorithms. No one does that today. When needed, you find existing code that does that stuff. Probably better than you could write. Who codes a hash table today?
> focused upon memorization and regurgitation
This is what is easy to test in-class.
Teachers worry about AI because they do not just care about memorization. Before AI, being able to write cohesive essays about a subject is a good proxy to prove your understanding beyond simple memorization. Now it's gone.
A lazy, irresponsible teacher who only cares about memorization will just grade students via in-class multi choices tests exclusively and call it a day. They don't need to worry about AI at all.
But the foundations start with memorisation.
It's simply too complex to fix. I think we'll see increased investment by corporates who do keep hiring on remediating the gaps in their workforce.
Most elite institutions will probably increase their efforts spent on interviewing including work trials. I think we're already seeing this with many of the elite institutions talking about judgment, emotional intelligence critical thinking as more important skills.
My worry is that hiring turns into a test of likeability rather than meritocracy (everyone is a personality hire when cognition is done by the machines)
Source: I'm trying to build a startup (Socratify) a bridge for upskilling from a flawed education system to the workforce for early stage professionals
Not sure if those two were just old school (they'd occasionally hit us/ pull ears too) but damn was I ahead of all the other kids when I came back to the USA
Also the two teachers had the same class of kids for all of elementary school, teaching 1st through 5th grades sequentially So they got to know the kids quite well