I used to make my classes 60-80% project work, 40-80% quizzes all online.
I now do 50% project work, 50% in person quizzes, pencil on paper on page of notes.
I'm increasingly going to paper-driven workflows as well, becoming an expert with the department printer, printing computer science papers for students to read and annotate in class, etc.
Ironically, the traditional bureaucratic lag in university might actually help: we still have a lot of infrastructure for this sort of thing, and university degrees may actually signal competence-beyond-ai-prompting in the future.
I think the oral exam is probably a great way to ascertain a student's ability, but let's be real, undergraduate class sizes numbers in the hundreds, for almost every first year class. I don't think it's possible to administer that. I think I would have loved to see that in my later years at university though, we still just did things by written exam + course work.
One consequence of LLM fraud at scale making remote/online tests & document submission worthless is it might act as a giant revitalizing boost for the bricks-and-mortars school systems. Suddenly having real teachers and students in room together has value again, for credibility and authenticity alone.
LLMs are also making having a public repo code portfolio be much more worthless as a sign of legitimacy
If AI can do the work, maybe the test should be more focused on what AI can’t do? This is like anyone still doing a traditional coding interview with leetcode problems just because they haven’t yet done the work to figure out what to test for in a world where Claude Code exists.
What's interesting is that as I understand, folks are using things like Google Docs for papers, and that it's (apparently) straight forward to do analysis on a Google Doc to see, well, the life of the document. How it was typed in, how fast, what was pasted and cut back out.
My understanding is that the Google Doc is not a word processing document, it's an event recording of a word processor. So, in theory, you could just "play back" watching the document being typed in and built to "see" how it was done.
I only mention this because given the AIs, I'm sure even with a typewriter, it's more efficient to have the AI do the work, and then just "type it in" to the typewriter, which kind of invalidates the entire purpose of it in the first place.
The typing in part is inevitable. May as well have a "perfect first draft" to type it in from in the first place.
And we won't mention the old retro interfaces that let you plug in a IBM Selectric as a printer for your computer. (My favorite was a bunch of solenoids mounted above the keys -- functional, but, boy, what a hack.)
TaaS -- Typing as a service. Send us your Markdown file and receive a typed up, double spaced copy via express shipping the next day!
This would take about 1 day for some student to realize you can instruct one of the LLMs to operate the computer screen for you and have it type and fake edit a document for you. The tip would spread among the cheaters and the metric would become harder to judge by itself.
I had a typewriter growing up and I remember thinking it was the coolest thing. I was amazed by it and tried writing several stories. Eventually my dad bought me a crappy old computer that was only really good for writing, and that was cool too. I loved that thing. It was small too, with an integrated monitor and keyboard, so it didn't take over the whole desk where I still used pencil and paper often
Imagine being able to do some writing without notifications going off every few seconds, and where you're not always one click away from a search engine and some website scientifically designed to drag your attention down a rabbit hole and keep it there
If students cheat they hurt only themselves. Make sure they understand the consequences for cheating (missing out on learning) and that's about all you can do.
I’m confused about too many things being measured at once. Is Phelps banning AI to ensure her students are fit to pass terminal examination? And doing so to ensure that her class has a good pass rate, proving she is a good teacher and can keep her job? What if her cohort are particularly dumb? Is she incentivized to make it easy to pass her classes to get that A you paid so much for? Or hard or make that A worth something?
My mentor, a PhD in classics, told me it was never about outcomes and only about improvement. I suppose that answers my question. If your AI gets you an A at the start of the course and an A at the end, then, in the sense that you have not succeeded over anything, you have failed.
at my university in math exams we were only allowed to use 1 specific model of calculator, and most of the exams were answered symbolically anyways, so the calculator usually was not helpful anyways.
I'm confused at these comments because it makes me think the commenter didn't go to college. All my classes were basically open everything except for a literal laptop. Calculators were not useful.
When I did my Computer Science degree the vast majority of courses were 50% final, 30% midterm - even programming exams were hand written, proctored by TAs in class or in the gymnasium - assignments/labs/projects were a small part of your grade but if you didn’t do them the likelihood you’d pass the term exams was pretty darn low.
When I did tertiary studies in programming there wasn't AI but we did our programming exams in pencil and paper. The "beneficial" prep we had and I had since high school was using punch cards. And 24h turnaround time for compiles. That really makes you think. And you learn how to desk check even thousand line programs. Intense focus, structuring for readability (to catch typos) and simplicity (catch logic errors) helped enormously. Was not unusual to change hundred lines of code and submit knowing that it wouldn't compile but will throw up the other errors I couldn't find. Our exams would give us 4-6 attempts for clean compile AND correct output. The only space where I experience same challenge now (40+ yrs later) is embedded code. Desktops and web stuff have LSPs and dynamic reloads and interpreted code (not a thing for me when learning) with instant feedback.
Lots of skills from those old days that have been lost/ignored in the pretence of productivity.
My networking final in high school was probably my favorite test taking experience - there was a small written portion on eg subnets, but the bulk of our grade was setting up a physical network, testing it, and leaving the room. Our teacher sabotaged three parts of our network - could be hardware, router misconfiguration, etc. when we came in we had iirc 20-ish minutes to diagnose and fix it.
The best was when she barely unscrewed one of this big DIN connectors so at quick glance it looked fine, but wasn’t fully connected.
Yeah none of the problems with AI in education are new; some schools (or news articles) are just panicking because they gave their students laptops (and/or made them a mandatory part) and now the genie is out of the bottle.
But there were already heaps of problems with tech in education before AI.
My CS projects were often pretty free-form so in theory I could've just used AI - today, anyway. But a big part of the grade was a face to face interview where you actually had to talk about the code you wrote. Anyone lifting along with other people who didn't actually do any work would fall through then.
That was common when I studied computer science in the nineties as well. Hand written exams mostly.
Writing papers is a useful skill to have. And many students aren't very good at that. I taught some classes during my Ph. D. and supervised some students with their master thesis and PhD thesis work. Many students get their degrees without that really getting addressed. At least Computer science degrees in the Netherlands just spend very little time on writing skills. You get students with high school levels of English and Dutch and that's it.
I learned to write properly only when I started my Ph. D. My supervisor made me do it right before he allowed me to submit papers for publication.
AI might actually be good for education long term. It will result in a more personalized approach, which I think is good. There are plenty of ways to test students that are more engaging and interesting for both teachers and students than some of the old ways. You can't fake knowledge when you do a verbal test. Or test people with a good old written exam.
And of course for teachers, you can automate a lot of the verification work. This can be a lot of work.
In Spain, the whole university system was like that until like 15ish years ago. Exams were king, in most courses they were worth 80%-90%, and of course always in person.
Then we did a university reform, partly with the excuse of aligning with the rest of the EU within the Bologna process (and I say "excuse" because that's what it was, because the politicians introduced some things with that pretense that weren't like that in the rest of the EU at all, and it was perfectly possible to comply with Bologna without doing them) and partly to copy the US/UK ways. And one of the pillars of that reform was continuous assessment, and evaluating coursework.
As a consequence of this, first of all working class students were royally screwed. Because suddenly it wasn't OK to just organize yourself to prepare the exam, you had to attend lots of sessions to earn points, which put students who work at a disadvantage. And second, passing by cheating became possible, even before LLMs. People tend to forget that before everyone got access to ChatGPT, some people had access to experts (family members, or even paying someone to do the work).
Now that this kind of cheating has been democratized and everyone can do it instead of just the most privileged with access to experts or money to pay them, people act all outraged. Although pretty much nothing is being done, except for using snake oil detectors, or sometimes increasing difficulty of assignments to make them LLM-proof (with which you screw the students who actually want to learn without LLMs).
They spent years indoctrinating us (professors) in training courses on how the old exam-based ways were wrong (the "Napoleonic" model, they called it... none of them seems to entertain the thought that maybe if it had been working essentially unchanged since Napoleon it wasn't that bad, and you need solid reasons to change it beyond "this is old so let's change") and the new ways were the bee's knees. Like in the Milgram experiment, it's difficult for people to back down and acknowledge that they have been wrong, even when the solution is obvious.
My local college used to have a test center you would physically go into. Now they no longer let teachers send their students there (idk if its just for programming or what) for whatever reason. I know they've wanted to cut down on wasted paper for years.
If my college is doing this, I cannot imagine how many others are also impeding on their entire goal: education.
I’m guessing we’re similar vintage. My CS classes were like this as well.
The only exception is that when I got into grad level classes we did have some big programming projects. But most of that programming happened on sparc stations, and it was actually just easier and more productive to sit at the machine in person with its nice big (at the time) display with all the other folks doing programming projects. Those machines had the standard dev toolchains provisioned that weren’t easy (at the time) to do on a dorm room Mac or windows computer.
I really think a lot of the ways we can reduce reliance on AI for thinking is to just set up systems where it’s not an inviting or rewarding option.
When I was in university we had the "Honor System"—no proctors, but you had to sign a statement on every exam and assignment that you did not cheat. And if you did cheat, one of your fellow students could report you to the Honor Board. Basically using the prisoners' dilemma to enforce honesty. And the Honor Board were always threatening to bring proctors back if cheating continued.
But yeah, everything was hand-written. On sheets of paper with pencil. I even had to write x86 assembly out by hand for my CPU architecture class. Of course, laptops were available back then but not cellphones and certainly not LLMs, so cheating by electronic means probably presents a stickier wicket now than it did back then.
You mean an AI proof grading system. Grades have very little to do with actual learning and are there solely for signalling. It would be better if Universities shifted their focus to learning and eliminated grades/exams all together. They should seek to stop trying to be the gatekeepers for white collar work and instead focus on learning and research.
Not only that, the tabletification of education has actually been shown (via standardized testing) to make our kids dumber. This is the first time a new generation has scored worse than their parents. Technology has its place, we need to pick and choose where.
Which strikes me as a terrible way to teach and test programming skills. If you're teaching to program without so much as syntax highlighting, you're not preparing your students for anything that even remotely resembles the industry they aspire to work in.
Honestly, these days universities should probably find a way to incorporate AI into their teaching, rather than fight it. Anything else is betting that AI will not stick around, which strikes me as a hopelessly naïve bet. Especially for software development.
I don't pretend to have all the answers, I don't know how to teach systems thinking in a appropriate way either. But I'm pretty sure typewriters isn't it, unless your students are hoping to get hired by Ada Lovelace, it's just not going to be relevant.
Seems like anybody could just study 6 hours a day for the last month before the final, last 2 weeks before the mid term and use AI for everything else.
Might be an unpopular opinion in this thread, but college was made worthless for most degrees as soon as the internet got popular and silly performative shit like this is the death knell. College is about learning how to work in an industry. I'd predict an uptick in trade schools and other hands-on work like medicine, and a continuing downturn in so-called formal education for anything white-collar, programming included. Students are customers. Businesses are going to use AI going forward. No reason to waste time on this.
Why are people promoting the idea that exams are not written or given in person anymore? I graduated relatively recently and maybe had 1 take home exam during my entire education. Every other exam was proctored in person and written. The professor who made the take home exam also made it much more difficult than a normal exam so I would not really say it was easier than a normal in person test.
Things have changed drastically since COVID-19, at least in the US. Tons of schools and universities shifted to online systems, and never abandoned the systems they built up when it was time to go back to school.
I graduated in 2020, so I've only gotten to see the changes secondhand through friends and family who are teachers, and through my sibling who graduated a few years after me. But the difference is staggering.
This will only work until somebody figures out how to connect an AI to the typewriter which will have some sort of MIC, and the person will start dictating into it with AI-assisted revisions. Once the dictation is over, the AI-enabled typewriter will be instructed to type the work out.
Testing and instruction should be modified to account for AI. If a student uses an Agentic AI for work, learning, research, then when test time comes, the student should be required to stand in the front of the class and teach the class what they have learned, i.e. "Teach Back" all they learned to the entire class student body and teacher. The entire class, instructor included, will also be required to participate in a Q&A session to make sure that student's learning is not just made up of memorization, e.g. restate the information learned but using different words, different scenarios, etc.
I like open note exams (and perhaps open book exams, as you need to know the book well to know which page to look at) - it forces you to condense the material to the salient points and operationalise it to solve what would be more challenging problems than a simple recall exam.
When I see 'cheat sheets' - designed to be hidden on the back of calculators or whatever - then I see true application of human ingenuity and intellect.
Reading all these comments, I feel like US universities are a joke.
I had to do all the exams in person. 100% of the grade was decided at the exam. Millions of people graduated this way and they are fine. No students were harmed in the process.
So you didn't have to do any course work? No collaboration? No labs? I'm not aware of any University that doesn't have coursework outside of online diploma mills.
I agree fully. Not sure what they are on about, with "no labs??" in the replies.
You still do all the same things, and they are graded, but this doesn't affect your final grade. Instead, you need to pass a threshold to enter the exam, which is then graded.
The US isn't so amazing at this, it simply can be done better. Recognizing where you can improve and from whom you can learn is a great first step to ACTUAL improvement.
for people with medium to high anxiety problems and/or with ADHD, exams are an absolute nightmare that is not representative at all of workplace performance or ability to do good work.
My school couldn't afford typewriters in the 1980's and early 1990's.
We wrote assignments by hand using a pencil or pen.
Is that really complicated?
When I got to college and everything had to be typed I still wrote everything by hand on paper and edited with an eraser and a red pen to reorganize some sentences or paragraphs. Then I would go to the computer lab and type it in and print it out.
I like this. Related, this semester I've been using handwritten quizzes in class. A simple change that's been one of the best things as it changed students' expectations of class prep. Kind of do the readings and sort of prep and you can coast in class. But if you need to write out quiz answers you're forced to know the material better as well as maintain the ability to express yourself.
I also use low-point bonus questions to test general knowledge (huge variation on subjects I thought everyone knew).
104 comments
[ 5.3 ms ] story [ 85.2 ms ] threadI now do 50% project work, 50% in person quizzes, pencil on paper on page of notes.
I'm increasingly going to paper-driven workflows as well, becoming an expert with the department printer, printing computer science papers for students to read and annotate in class, etc.
Ironically, the traditional bureaucratic lag in university might actually help: we still have a lot of infrastructure for this sort of thing, and university degrees may actually signal competence-beyond-ai-prompting in the future.
We'll see.
My tests are almost 100% in person. Project work included, you can hand something in, but I'm going over line by line and ask what you did there.
I can do this, because while my school hasn't updated the tests yet, my classes are small and I can do all of them in-person.
Not sure anyone even attempted to cheat in that scenario. And the conversations were usually great, although very stressful for us cramming types
LLMs are also making having a public repo code portfolio be much more worthless as a sign of legitimacy
My understanding is that the Google Doc is not a word processing document, it's an event recording of a word processor. So, in theory, you could just "play back" watching the document being typed in and built to "see" how it was done.
I only mention this because given the AIs, I'm sure even with a typewriter, it's more efficient to have the AI do the work, and then just "type it in" to the typewriter, which kind of invalidates the entire purpose of it in the first place.
The typing in part is inevitable. May as well have a "perfect first draft" to type it in from in the first place.
And we won't mention the old retro interfaces that let you plug in a IBM Selectric as a printer for your computer. (My favorite was a bunch of solenoids mounted above the keys -- functional, but, boy, what a hack.)
TaaS -- Typing as a service. Send us your Markdown file and receive a typed up, double spaced copy via express shipping the next day!
Imagine being able to do some writing without notifications going off every few seconds, and where you're not always one click away from a search engine and some website scientifically designed to drag your attention down a rabbit hole and keep it there
My mentor, a PhD in classics, told me it was never about outcomes and only about improvement. I suppose that answers my question. If your AI gets you an A at the start of the course and an A at the end, then, in the sense that you have not succeeded over anything, you have failed.
We already had AI proof education.
Lots of skills from those old days that have been lost/ignored in the pretence of productivity.
The best was when she barely unscrewed one of this big DIN connectors so at quick glance it looked fine, but wasn’t fully connected.
But there were already heaps of problems with tech in education before AI.
My CS projects were often pretty free-form so in theory I could've just used AI - today, anyway. But a big part of the grade was a face to face interview where you actually had to talk about the code you wrote. Anyone lifting along with other people who didn't actually do any work would fall through then.
Writing papers is a useful skill to have. And many students aren't very good at that. I taught some classes during my Ph. D. and supervised some students with their master thesis and PhD thesis work. Many students get their degrees without that really getting addressed. At least Computer science degrees in the Netherlands just spend very little time on writing skills. You get students with high school levels of English and Dutch and that's it.
I learned to write properly only when I started my Ph. D. My supervisor made me do it right before he allowed me to submit papers for publication.
AI might actually be good for education long term. It will result in a more personalized approach, which I think is good. There are plenty of ways to test students that are more engaging and interesting for both teachers and students than some of the old ways. You can't fake knowledge when you do a verbal test. Or test people with a good old written exam.
And of course for teachers, you can automate a lot of the verification work. This can be a lot of work.
Then we did a university reform, partly with the excuse of aligning with the rest of the EU within the Bologna process (and I say "excuse" because that's what it was, because the politicians introduced some things with that pretense that weren't like that in the rest of the EU at all, and it was perfectly possible to comply with Bologna without doing them) and partly to copy the US/UK ways. And one of the pillars of that reform was continuous assessment, and evaluating coursework.
As a consequence of this, first of all working class students were royally screwed. Because suddenly it wasn't OK to just organize yourself to prepare the exam, you had to attend lots of sessions to earn points, which put students who work at a disadvantage. And second, passing by cheating became possible, even before LLMs. People tend to forget that before everyone got access to ChatGPT, some people had access to experts (family members, or even paying someone to do the work).
Now that this kind of cheating has been democratized and everyone can do it instead of just the most privileged with access to experts or money to pay them, people act all outraged. Although pretty much nothing is being done, except for using snake oil detectors, or sometimes increasing difficulty of assignments to make them LLM-proof (with which you screw the students who actually want to learn without LLMs).
They spent years indoctrinating us (professors) in training courses on how the old exam-based ways were wrong (the "Napoleonic" model, they called it... none of them seems to entertain the thought that maybe if it had been working essentially unchanged since Napoleon it wasn't that bad, and you need solid reasons to change it beyond "this is old so let's change") and the new ways were the bee's knees. Like in the Milgram experiment, it's difficult for people to back down and acknowledge that they have been wrong, even when the solution is obvious.
Apparently you learn to double check your work!
If my college is doing this, I cannot imagine how many others are also impeding on their entire goal: education.
The only exception is that when I got into grad level classes we did have some big programming projects. But most of that programming happened on sparc stations, and it was actually just easier and more productive to sit at the machine in person with its nice big (at the time) display with all the other folks doing programming projects. Those machines had the standard dev toolchains provisioned that weren’t easy (at the time) to do on a dorm room Mac or windows computer.
I really think a lot of the ways we can reduce reliance on AI for thinking is to just set up systems where it’s not an inviting or rewarding option.
But yeah, everything was hand-written. On sheets of paper with pencil. I even had to write x86 assembly out by hand for my CPU architecture class. Of course, laptops were available back then but not cellphones and certainly not LLMs, so cheating by electronic means probably presents a stickier wicket now than it did back then.
I'm sure they had some kind of submit your code as assignment and using testing as a way to grade the assignments.
Which strikes me as a terrible way to teach and test programming skills. If you're teaching to program without so much as syntax highlighting, you're not preparing your students for anything that even remotely resembles the industry they aspire to work in.
Honestly, these days universities should probably find a way to incorporate AI into their teaching, rather than fight it. Anything else is betting that AI will not stick around, which strikes me as a hopelessly naïve bet. Especially for software development.
I don't pretend to have all the answers, I don't know how to teach systems thinking in a appropriate way either. But I'm pretty sure typewriters isn't it, unless your students are hoping to get hired by Ada Lovelace, it's just not going to be relevant.
I graduated in 2020, so I've only gotten to see the changes secondhand through friends and family who are teachers, and through my sibling who graduated a few years after me. But the difference is staggering.
If you're not interested in learning the course content, then what are you doing there? Pretty expensive waste of time.
I very fondly recall many of the course I did at university. The exams were a helpful motivating factor even for the interesting courses.
Testing and instruction should be modified to account for AI. If a student uses an Agentic AI for work, learning, research, then when test time comes, the student should be required to stand in the front of the class and teach the class what they have learned, i.e. "Teach Back" all they learned to the entire class student body and teacher. The entire class, instructor included, will also be required to participate in a Q&A session to make sure that student's learning is not just made up of memorization, e.g. restate the information learned but using different words, different scenarios, etc.
When I see 'cheat sheets' - designed to be hidden on the back of calculators or whatever - then I see true application of human ingenuity and intellect.
One of my best college professors would review such essays in-person, one-on-one twice each semester.
I had to do all the exams in person. 100% of the grade was decided at the exam. Millions of people graduated this way and they are fine. No students were harmed in the process.
What a narrow set of skills to send into your economy.
You still do all the same things, and they are graded, but this doesn't affect your final grade. Instead, you need to pass a threshold to enter the exam, which is then graded.
The US isn't so amazing at this, it simply can be done better. Recognizing where you can improve and from whom you can learn is a great first step to ACTUAL improvement.
We wrote assignments by hand using a pencil or pen.
Is that really complicated?
When I got to college and everything had to be typed I still wrote everything by hand on paper and edited with an eraser and a red pen to reorganize some sentences or paragraphs. Then I would go to the computer lab and type it in and print it out.
I also use low-point bonus questions to test general knowledge (huge variation on subjects I thought everyone knew).