... or Blue Book exams? We did tons of those, way before ChatGPT was a twinkle in anyone's eye. Do they not do those anymore? Or is this just for remote learning or to supplement grading things like term papers?
That's what replaced oral exams. The color of writing material's binding is irrelevant.
Oral exams offer a fluidity not possible in written forms and the rationale for a written exam -- whether by the stroke of fingers to key or hand to paper bound in Blue, it is now muted by the ability to video record the oral exam for later grading.
Thus the performance of the exam can be recorded; allowing for a leisurely assessment. With the performance itself capable of a far more robust introspection of one's unaided knowledge.
Oral exams were tough if you were nervous. You are facing the teacher feeling like they are getting pissed at your lack of preparedness. On the other hand, if you record them yourself, who is to say you don't have a teleprompter just behind the camera? You'd have to make people come on campus into a testing center to ensure fidelity. Its probably a lot more expensive to build out AV facilities for each and every student at a massive college taking an oral exam than it is to just have a 1foot folding desk and a few sheets of paper.
Yeah, I'm actually in favor of oral exams regardless, I'm just not getting how they address ChatGPT cheating any better than in-person written exams, unless we're mainly talking about online courses.
They still do those. However in the years before remote learning and increasingly after, schools started buying services like blackboard or canvas. These are academic websites that manage homework assignments, deadlines, store the syllabus, announcements, etc for each class where the teacher bothers to use the system (most of them at least have a syllabus if nothing else posted). They also allow assignment submissions, even timed testing environments where the browser is locked down. They also link with turnitin to check submissions for plagiarism. Courses that have grown reliant on these third party tools will have to simply turn the clock back to 2010 or so and go back to in person assignments. I wouldn't be surprised if it got to the point of otherwise fully remote classes booking a room on campus 3x a semester to conduct exams, because a paper exam seems so much simpler than fighting an arms race with chatgpt and electronic submissions.
This might not actually hurt companies like blackboard or canvas that much, because universities are like these big cows where once they have an institutional license they will probably maintain it forever, even if they are also buying the competing product (typical to pay for microsoft business and google cloud services both even though they are entirely redundant for example).
> I wouldn't be surprised if it got to the point of otherwise fully remote classes booking a room on campus 3x a semester to conduct exams
This is exactly what the otherwise-fully-remote courses that I took did, about a decade ago. The school had arrangements all over the country (indeed, all over the world, IIRC) for testing locations, often at other universities (or at their own primary campus or a handful of satellite campuses), so as long as you lived within once-a-semester commute distance of one of those locations so you could do the final exam (I don't think they did a mid-semester exam at those, but it's been a while, maybe they did) you could take the courses, no problem. They had a proctor or two in the room to make sure no-one cheated and to help with basic issues that might come up, and the proctor collected your work at the end, to submit it.
Yeah, that sounds great, but the article completely fails to offer a solution to the reason universities have largely dropped oral exams in the first place, which is that they're expensive. Oral exams need much more time from competent evaluators than written exams do, and those evaluators need to be paid for their time. The vast majority of universities these days are going to prioritize cost-reduction over quality of education if they can get away with it, and they probably can.
Is it not enough to point out the ills of a system. Why gate it behind the need for a solution?
Expensive is your unsupported concern. The Greeks figured this out. Why assume this needs more expense than professor to do more than grade the performance that is carried out amongst what is some combination of peer and mentor?
Maybe a computer model is used to grade the performance. Maybe the mentor is a computer model.
gut the brusar and fire half the administrators. We can replace 80% of the school administration with AI anyway. that will free up enough money and PAY professors a living wage.
Chemistry 101 is likely a better example. Intro to chemistry is a required to class for almost all STE (not M, but M often takes it too for a physical science degree), along with anything that touches on nutrition, medical...
> Chemistry, as the central science, is relevant to many other disciplines and is therefore part of the core curriculum of numerous degree programs on campus. By the time they graduate, 55 percent of undergraduates have taken a chemistry class at UW-Madison. As enrollment grew over the years, department and campus leaders noticed that students were often not able to schedule needed courses. This meant it took them longer to earn a degree.
There are about 10,000 under grads for each year at UW Madison. Chem 103 gets about 5000 students per year.
While that roughly matches the Psych 101 with 2000... this shows that "here's a real number."
Maybe there’s value to classes outside of what employers want Especially for survey-level classes that aren’t specific to a specialization or major.
I think alternatively we need employers to stop asking for degree’s altogether so the enrollees are people who are there to learn for themselves not for employers. It’d reduce the crazy demand and cost around education.
1. Some people want to study psychology (or other classes) for pure intellectual reasons. They’re curious about the world. Shouldn’t that be enough?
2. “Survey” courses have 2 values through helping people learn new topics (1) gain broad understanding of the world, and (2) discover fields they may wish to major or study deeper in if they haven’t decided yet.
My caveat here is that I suspect (unsubstantiated) that some topics become filler by universities.
3. There are lots of jobs people do with a psychology major, besides a psychologist/researcher, but I don’t know how many of them require that deep knowledge explicitly.
Many jobs are probably better performed if you learn some psychology (eg a few classes worth) - marketing, non-therapeutic counseling (coaches, teaching, etc), HR, recruiting, doctors, nurses, organizational management, etc.
I want to reiterate point 1 - there’s value in education beyond employment, but it’s plausible that employers should care less. I doubt most employers expect candidates to take survey level courses, but requiring a degree of any sort implicitly assumes they have taken those courses.
Good point. The solution that comes to mind is grading exams with AI assistance. No idea what that'd look like, somehow the professor would have to supervise the agents.
I'm sure that is coming. Aside from research, teaching students is just explaining concepts and guiding them through any questions. This is already possible it's just not very reliable yet.
Have you taken an oral exam? You're making some assumptions that do not hold, in my experience.
For example, oral exams are still very common in Poland and Polish universities. They typically takes less time to take, for a few reasons (the number of questions, the speed of writing versus oral production of answers, the ability of the professor to ask probing and follow-up questions instead of having to give the same battery of questions to each student in a blind and generic manner, even when it is clear that the student has answered the question in another answer or another way). The goal is for the professor to establish with confidence the degree to which the student understands the material.
Now, yes, it takes more time for the professor to administer the exam than it would take to administer a written exam, but note that grading exams is also very time consuming. You can delegate to grad students, but the same can be said of oral exams, so there's little differentiation there. The oral exam combines the administration of the exam with the grading, whereas written exams separate the two.
I have, yes. My university experience ran the gamut, in that I had both classes in a massive lecture hall of hundreds of students where most never spoke a word to the professor and all exams were done via scantron, as well as small graduate courses of around 4 students where the entire class was oral exams and one-on-one demonstrations and group discussions. The smallest course was only available at all because of a particularly passionate professor who insisted on holding it, and the class was never available during semesters where he wasn't available, because nobody else was willing to put in the work to personally administer the class. Notably, all those courses cost about the same amount of money per credit hour from my perspective, so the university was making orders of magnitude more money on the large class (where I can't see oral exams being feasible without significant investment) than they were making off the small class.
> You can delegate to grad students, but the same can be said of oral exams
It's much harder to delegate an oral exam than a written one. As you've mentioned, oral exams combine administration with grading, which means the grad students who are delegated to need to be able to administer the exam (ie, effectively probe the knowledge of each examinee) as well as evaluate their responses. In a written exam, the grad student only needs to evaluate the responses, usually against a set of predetermined criteria since the structure of the questions is set in stone.
None of what universities are moving towards is better, but I fail to see how it isn't cheaper.
Another factor is in the USA, at the undergraduate level, there is a strong incentive for "professors" (more assistant and adjunct than tenured) to grade in a manner which is totally rationalizable and consistent between students. Administrators call students "customers" who are buying a degree; if a student doesn't like their grade they will complain about it, and if the teacher hasn't explicitly formalized their grading process they will be in hot water. So for example asking different probing questions of different students might lead to accusations of arbitrary and capricious grading. It encourages checklist-style rubrics for grading. Obviously one could formalize a decision tree for more loosely structured oral exams but that's significant effort.
A rubric is as easy to apply to oral exams as a written one and that is generally all that administrators would like to see. Any interpretive exam is going to run into accusations of arbitrary and capricious grading.
I'm not disputing your point at all, but an interesting thing that happened at my school was that we weren't called "customers" but rather the president of the school would refer to students as "commodities". For a long time they'd treated us as such but hearing this was the blatant example of it.
At my university in Germany the trade-off is at approx. 50 students in a course when written becomes much more efficient. I can take max around 12 oral exams a day mostly it is rather 8.
The point is that the written exam is largely outsourced to PhD students here. The professor only looks at the model solution before and the grade distribution afterwards in extreme cases. Also you can parallelize the grading so that even for 300 students the grading is done in a week. Typically the professor spends half a day otherwise it would be 2 month...
I only do oral exams for my course and I am very happy how well it works. I normally ask students to grade themselves afterwards and I am typically spot on (I have written down the grade before).
That's the bottleneck though. It's not on the students end. It's on the professor/instructor/grad student end. It's not uncommon at big universities to have classes with hundreds or even thousands of students. Even if the oral exam can be completed in half an hour and there's 10 professor/instructors/grad students that can evaluate them that's STILL 50 hours each in a class of 1000 students. With a written exam it's one two or three hour writing period, then marking. Some of the huge classes will do multiple choice only on Scantron sheets so marking is literally done in minutes. With the courses I've marked it usually takes much less than half an hour to mark an exam on average even if it took the student 3 hours to write.
I did an oral final exam for my theory of computation class. From the student perspective, it was nicer than an in person final as I knew I didn't have to have perfect prose in my answers (and that it only took 30 mins rather than 2 hours!). But the professor said he'd never do it again because 30 minutes * 30ish students was too much work.
I cannot imagine a return to oral exams in the current academic setting with huge class sizes. The only place I could them being useful is in tiny honors/grad classes where the professor already has a close familiarity with each of the students.
By auto-grading the transcript as if it were an open ended written exam with no answer key, ignoring the confidence of the candidate and any nonverbalities that would be significant in an oral exam.
Said by the same kind of people who won't come into the office anymore for 8 hours ...
Simply coordinating meeting with 30 individuals within a single week is really difficult--think about how long it takes a manager to deal with annual reviews for everybody.
> But the professor said he'd never do it again because 30 minutes * 30ish students was too much work.
The old German Diplom system (replaced by BSc/MSc about 15 years ago) was based on oral exams. The way this worked there was that there were much fewer exams. For example, in Computer Science I had my first oral exams two years into the degree, and there were only five of them: one in Mathematics, one in each of Theoretical, Practical and Applied, Computer Science, and one for the second subject. Each of these oral exam was 30 minutes but covered the content of four semesters. There was a second round of such exams at the end of the degree and that was it.
To be admitted to an oral exam, one needed to pass the courses that the exam was going to be about. Lecturer were free to set the passing requirements. It was often just something like a coursework threshold of 60%.
> the professor said he'd never do it again because 30 minutes * 30ish students was too much work.
So he prefers spending that time grading written exams? Or does he just dump the burden on a platoon of grad students? (And if you're going to do that, why not have the grad students administer the oral exam?)
Oral exams combine administration and grading, so it can actually be less time consuming and less effort to give them. You can also vary questions based on whether previous answers already answer later questions, probing as you go along. The point of exams isn't supposed to be a bureaucratic checking off of boxes. It's supposed to be an assessment of familiarity with the taught material.
I wish my college experience had oral exams. Even for technical subjects - this was actually quite common in the Soviet Union, and to my understanding still somewhat common in Russia today. Given good questions and follow ups, they force you to have a complete picture of the topic in your head and see the connection between different ideas within the subject. I recently had an interview with a particuarly talented interviewer and although I didn’t get the job, his probing led me to spend the next week filling in gaps in my understanding and I feel more equipped as a result.
Absolutely No. Anyone who has done job interviews can testify that this is a terrible idea. In person verbal communication is much more prone to miscommunication than written word where you know exactly what was written. Plus oral exams leave out room for a lot of subjectivity and bias.
And yet I've never head of a hiring process that is only in writing.
I agree that it's more subjective, but with verbal communication (where you can have back and forth), it's easier to AVOID miscommunication.
Both oral and written have their pros/cons. Perhaps the best approach is to rely on written for scoring and oral for "sanity checks" (i.e. is the person just faking it).
God forbid college educated professionals know a minimum about communication and speaking. Aren't you/we supposed to be society's elite?
You're not supposed to "miscommunicate" by the time you graduate.
College students in the 1910s could read Greek and Latin. Before beginning college. And there were no fancy apps to help them. Everything was by the book and hard memorization. But it would be asking too much of a graduate in 2023 to be fluent in English?
A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.
>God forbid college educated professionals know a minimum about communication and speaking.
You're not supposed to "miscommunicate" by the time you graduate.
I think in the era of remote work one might assumes that all communication is done via slack/email and no real time verbal communication is needed.
The core of the problem lies with your expectation that just being fluent in English is sufficient to not have miscommunication. Communication is a lot more involved and something as trivial as not having a good night’s sleep can take your attention away from what others are saying. Written communication helps because you can read something back quickly if you skipped over something or were not paying attention.
Specialization is the only way we can currently make progress as a species, and is the only way we will be able to for the foreseeable future.
If everyone only had a passing or surface level knowledge of every field, they could not add to our collective understanding of any field.
To actually advance human understanding in a field, an individual needs to catch up on everything that has been learned about it up to this point. For some fields, that is hundreds of years of research and knowledge gained. That's not practical to learn for more than one or two fields - we simply don't live long enough.
If you want the person who makes the next alloy that makes planes another 20% lighter, you're going to need someone who has dedicated most of their life to just learning material sciences and is putting all their effort into that topic. It isn't going to be done by someone who had one quarter of a semester in college on it. (With rare exceptions)
>In person verbal communication is much more prone to miscommunication than written word where you know exactly what was written.
Me thinks you are mistaken.
In case of verbal miscommunication, you can always give the other party a second chance and ask them to repeat or rephrase their statement or you can rephrase yourself if you felt your originals statement was confusing.
In case of written communication there is no second chance to rephrase your statement. What's written is written, and that goes, regardless if it was well understood or not.
Not if you're under time pressure, which many written exams are. If you have to write fast, you're maximizing the chance of mistakes and there's little opportunities to go back, correct and rewrite.
I was talking to my brother (who is a professor at at a uni) about this topic. I had the opinion that oral exams were mostly dropped because of cost reasons. However, he gave a different reason: Lawsuits. Hard to get sued over a wrong multi choice answer. Easy to get sued over an oral exam.
Proctored exams are the long-used solution for cheating, when licensing is involved. There's a whole industry behind in person exams (Pearson, etc). I find this article extremely suspect, in that it's not mentioned once. In person, proctored, exams, seems infinitely more likely, and useful, than switching to oral exams.
Proctored exams, in RF shielded rooms, where you have to go through metal detectors (to prevent bringing computers into the room), are going to become the norm.
I think they already are in some country. In my European country there's a human proctor and the exam is also video recorded to make sure that the human proctor is actually doing his job and not turning a blind eye. Also cheating is rampant through these wireless earphones that students mount on their eardrums and end up having to go to the hospital to have them removed.
Universities should return to oral exams for a different reason, which is that it gives students interviewing practice and familiarity. Written exams do test your knowledge of a subject, but they aren't really how knowledge is assessed outside of school. For obvious reasons -- they're easy to cheat!
People go through all of university, ace their exams, then go to get hired in their first job and faceplant the interview in spite of having the qualification.
This is no surprise, they haven't ever needed to practice that skill!
It is only a matter of time before the current education system faces irrelevance. While they are concerned about cheating, it is in fact their own institution that is at risk.
Mostly it is the AI hallucination problem that stands in the way.
Furthermore, if AI continues to consume the world, who is interested in investing years into building up lifelong debt to only risk that everything you learned is now obsolete or irrelevant upon completion.
I asked a history professor friend how he combats this? He says his essay assignments are very targeted toward material covered in the class and in the class notes. He thinks the course and note material is unique enough to prevent people from using AI to cheat.
How long is it going to take universities and companies to face the reality that we should not be assessing people's capabilities in a vacuum where they can't access the tools that they will be using in the real world.
If knowledge can be easily acquired in 5 seconds, why are you testing people on whether or not they have memorized it? What good is 5 seconds? If the difference between an accredited engineer and a regular person is supposed to be a 5 second Google, then what the hell is the point?
We need to modernize assessment. We should have done it decades ago when cheating became rampant. Problem is that universities have zero incentive to fix this problem. Human resource teams don't either. The only people that really care are the people that have to work with everyone who gets hired despite having zero actual skills. Unfortunately, they have one of the quietest voices in this chain.
> If knowledge can be easily acquired in 5 seconds, why are you testing people on whether or not they have memorized it?
If you don't know much, you don't even know the right questions to ask your AI assistant, or the right tasks to have it perform. You'll also be less-able to check, and to apply, its output. The more you know, the more you can get out of your assistant.
At least so far, these tools are more like a bicycle than an internal combustion engine: they help almost anyone go faster and farther, but the people who are fit will still be able to go the fastest and farthest. Substitute "knowing stuff" for "fitness" and that's what we're looking at with "AI" (again, so far—if we achieve actual AGI so a huge set of complex tasks don't need a human involved at all, that may be another matter)
That's why people still need to know stuff they could find out in five seconds.
> If you don't know much, you don't even know the right questions to ask your AI assistant, or the right tasks to have it perform
It sounds like you are agreeing with me here. We should not be making life-changing assessments of people's skills in unrealistic situations. In the real world, nobody gives you the question. Nobody gives you a breakdown of the tasks needed. And yet that is exactly how we assess people in these sorts of exams.
I agree with you on the AI part though. I like the bicycle analogy.
I think one of the important aspect of intelligence is how fast you can comprehend raw data / unseen information. In my observation in university lab, being able to google for answers and fix bugs is the rarest perk in computer science department. I see people just stuck in the search result page althought looking exactly at the answer. This aspect can only be assessed when the target is interacting with the environment (search engine).
Exam is oudated. Human can never get rid of this mindset. Why do we need exams in the education system? Especially we are entering the era of stron AI that knows and understand everything. Education should now turn to teach people how to better use the tool named science and technology. Teach people how to come up with innovative ideas to solve all kind of problems by their tools in hand. Should not waste time in remembering unnecessary stuff, especially some always emphasize how important it is to be creative/ AI can not have human's creativity.
And how are you going to score students? Like autism with IQ160 that cannot talk but building Transformer model since elementary school something like that? Should you fail him/her just because doesn't like to talk and getting nervous in front of a bunch of examiners? And you sure you can score consistently against all students?
Of course human have been thinking about making a list of questions then ask a subset of it can somehow make the exam process more standardized. So you never think that you would end up asked the wrong question? And how it is standardized when you ask different people a difference question?
And guess what, those communicative sport guy or girls can always score higher than those introvert or even autistic self taught programmers. People just tend to judge others by the oral skills, not actual intellectual skills.
This appears to pose a significant challenge for individuals who are not native English speakers, as it may further complicate their ability to convey information clearly, especially during oral examinations where accents and language barriers may come into play. Do these exams take into account and address these concerns? (As someone who has not taken this type of exam, I'm curious if these issues are acknowledged and addressed appropriately.)
Oral exams are very common in Italian universities (many courses have both a written and an oral exam!), I was surprised when I moved to Germany for my masters and met students from various countries who had never taken one!
I agree, it's a great way to assess students. I've been through them too, they are nerve wracking, but end up being a lot better than written.
But the lawyers, man, they'd hate it.
There's too much variability. Some students would get 'harder' questions than others. Some would get 'grilled' more in depth. Some would be asked to turn around and face a whiteboard more. Some students would say this is because I'm part of a certain category. And of those students, a lot would unfortunately be in the right. Because professors are just as biased as the rest of us, if not more so.
So, if you'd want to do this, the lawyers would say you'd need to have at least one person of every imaginable category in the room giving questions. And it would need to be an open exam (Pence rule), and they'd like for the exam to be recorded too, video and audio. And you've got to use a rubric with standard questions, so we can make sure there's no bias. Which means you have to sequester those that completed from those that haven't, phones included. And the proctors have to be behind a screen. And a few dozen waivers, just to round things off.
As others here have said, the expense would be huge. At least a handful of people, A/V techs, a revolving door of students, translators, the works. All just to end up asking students the exact same questions anyway.
91 comments
[ 3.2 ms ] story [ 158 ms ] threadOral exams offer a fluidity not possible in written forms and the rationale for a written exam -- whether by the stroke of fingers to key or hand to paper bound in Blue, it is now muted by the ability to video record the oral exam for later grading.
Thus the performance of the exam can be recorded; allowing for a leisurely assessment. With the performance itself capable of a far more robust introspection of one's unaided knowledge.
This might not actually hurt companies like blackboard or canvas that much, because universities are like these big cows where once they have an institutional license they will probably maintain it forever, even if they are also buying the competing product (typical to pay for microsoft business and google cloud services both even though they are entirely redundant for example).
This is exactly what the otherwise-fully-remote courses that I took did, about a decade ago. The school had arrangements all over the country (indeed, all over the world, IIRC) for testing locations, often at other universities (or at their own primary campus or a handful of satellite campuses), so as long as you lived within once-a-semester commute distance of one of those locations so you could do the final exam (I don't think they did a mid-semester exam at those, but it's been a while, maybe they did) you could take the courses, no problem. They had a proctor or two in the room to make sure no-one cheated and to help with basic issues that might come up, and the proctor collected your work at the end, to submit it.
Problem solved.
Expensive is your unsupported concern. The Greeks figured this out. Why assume this needs more expense than professor to do more than grade the performance that is carried out amongst what is some combination of peer and mentor?
Maybe a computer model is used to grade the performance. Maybe the mentor is a computer model.
the Greeks didn't have Psych 101 classes with 2000 students
People want to learn, but what they really want is a degree. No one is going to go two more years into debt for pedagogical purity
https://chem.wisc.edu/chemistry-building-project-overview/
> Chemistry, as the central science, is relevant to many other disciplines and is therefore part of the core curriculum of numerous degree programs on campus. By the time they graduate, 55 percent of undergraduates have taken a chemistry class at UW-Madison. As enrollment grew over the years, department and campus leaders noticed that students were often not able to schedule needed courses. This meant it took them longer to earn a degree.
There are about 10,000 under grads for each year at UW Madison. Chem 103 gets about 5000 students per year.
While that roughly matches the Psych 101 with 2000... this shows that "here's a real number."
Q1: Are there thousands of employers out there demanding students who have passed Psych 101 with flying colours?
Q2: If not, why not try to stop this?
I think alternatively we need employers to stop asking for degree’s altogether so the enrollees are people who are there to learn for themselves not for employers. It’d reduce the crazy demand and cost around education.
(Full disclosure: I don't understand the US college system at all [sorry!])
In what circumstances would a student be already at college and wanting/needing to "survey" psychology by doing Psych 101?
More broadly, who wants to major in psychology .. and why? Do employers need more professional psychologists?
2. “Survey” courses have 2 values through helping people learn new topics (1) gain broad understanding of the world, and (2) discover fields they may wish to major or study deeper in if they haven’t decided yet.
My caveat here is that I suspect (unsubstantiated) that some topics become filler by universities.
3. There are lots of jobs people do with a psychology major, besides a psychologist/researcher, but I don’t know how many of them require that deep knowledge explicitly.
Many jobs are probably better performed if you learn some psychology (eg a few classes worth) - marketing, non-therapeutic counseling (coaches, teaching, etc), HR, recruiting, doctors, nurses, organizational management, etc.
I want to reiterate point 1 - there’s value in education beyond employment, but it’s plausible that employers should care less. I doubt most employers expect candidates to take survey level courses, but requiring a degree of any sort implicitly assumes they have taken those courses.
OK, so what proportion of college students choose their classes for those reasons?
Thinking back to my time at college, there wasn't a whole lot of purely intellectual motivation (or genuine curiosity) going around.
The overwhelming majority were there a) to have a good time, and/or b) because they saw it as a route to a well-paying job once it was completed.
For example, oral exams are still very common in Poland and Polish universities. They typically takes less time to take, for a few reasons (the number of questions, the speed of writing versus oral production of answers, the ability of the professor to ask probing and follow-up questions instead of having to give the same battery of questions to each student in a blind and generic manner, even when it is clear that the student has answered the question in another answer or another way). The goal is for the professor to establish with confidence the degree to which the student understands the material.
Now, yes, it takes more time for the professor to administer the exam than it would take to administer a written exam, but note that grading exams is also very time consuming. You can delegate to grad students, but the same can be said of oral exams, so there's little differentiation there. The oral exam combines the administration of the exam with the grading, whereas written exams separate the two.
I have, yes. My university experience ran the gamut, in that I had both classes in a massive lecture hall of hundreds of students where most never spoke a word to the professor and all exams were done via scantron, as well as small graduate courses of around 4 students where the entire class was oral exams and one-on-one demonstrations and group discussions. The smallest course was only available at all because of a particularly passionate professor who insisted on holding it, and the class was never available during semesters where he wasn't available, because nobody else was willing to put in the work to personally administer the class. Notably, all those courses cost about the same amount of money per credit hour from my perspective, so the university was making orders of magnitude more money on the large class (where I can't see oral exams being feasible without significant investment) than they were making off the small class.
> You can delegate to grad students, but the same can be said of oral exams
It's much harder to delegate an oral exam than a written one. As you've mentioned, oral exams combine administration with grading, which means the grad students who are delegated to need to be able to administer the exam (ie, effectively probe the knowledge of each examinee) as well as evaluate their responses. In a written exam, the grad student only needs to evaluate the responses, usually against a set of predetermined criteria since the structure of the questions is set in stone.
None of what universities are moving towards is better, but I fail to see how it isn't cheaper.
But what if AI conducts it?
The point is that the written exam is largely outsourced to PhD students here. The professor only looks at the model solution before and the grade distribution afterwards in extreme cases. Also you can parallelize the grading so that even for 300 students the grading is done in a week. Typically the professor spends half a day otherwise it would be 2 month...
I only do oral exams for my course and I am very happy how well it works. I normally ask students to grade themselves afterwards and I am typically spot on (I have written down the grade before).
It's unfortunate because my mind becomes completely useless in those setting.
I cannot imagine a return to oral exams in the current academic setting with huge class sizes. The only place I could them being useful is in tiny honors/grad classes where the professor already has a close familiarity with each of the students.
imagine 15 hours of work being too much work.
Simply coordinating meeting with 30 individuals within a single week is really difficult--think about how long it takes a manager to deal with annual reviews for everybody.
The old German Diplom system (replaced by BSc/MSc about 15 years ago) was based on oral exams. The way this worked there was that there were much fewer exams. For example, in Computer Science I had my first oral exams two years into the degree, and there were only five of them: one in Mathematics, one in each of Theoretical, Practical and Applied, Computer Science, and one for the second subject. Each of these oral exam was 30 minutes but covered the content of four semesters. There was a second round of such exams at the end of the degree and that was it.
To be admitted to an oral exam, one needed to pass the courses that the exam was going to be about. Lecturer were free to set the passing requirements. It was often just something like a coursework threshold of 60%.
So he prefers spending that time grading written exams? Or does he just dump the burden on a platoon of grad students? (And if you're going to do that, why not have the grad students administer the oral exam?)
Oral exams combine administration and grading, so it can actually be less time consuming and less effort to give them. You can also vary questions based on whether previous answers already answer later questions, probing as you go along. The point of exams isn't supposed to be a bureaucratic checking off of boxes. It's supposed to be an assessment of familiarity with the taught material.
This could hurt remote learning, but if you are remote there are plenty of ways one can cheat on their exams.
I agree that it's more subjective, but with verbal communication (where you can have back and forth), it's easier to AVOID miscommunication.
Both oral and written have their pros/cons. Perhaps the best approach is to rely on written for scoring and oral for "sanity checks" (i.e. is the person just faking it).
You're not supposed to "miscommunicate" by the time you graduate.
College students in the 1910s could read Greek and Latin. Before beginning college. And there were no fancy apps to help them. Everything was by the book and hard memorization. But it would be asking too much of a graduate in 2023 to be fluent in English?
A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.
I think in the era of remote work one might assumes that all communication is done via slack/email and no real time verbal communication is needed.
The core of the problem lies with your expectation that just being fluent in English is sufficient to not have miscommunication. Communication is a lot more involved and something as trivial as not having a good night’s sleep can take your attention away from what others are saying. Written communication helps because you can read something back quickly if you skipped over something or were not paying attention.
Specialization is the only way we can currently make progress as a species, and is the only way we will be able to for the foreseeable future.
If everyone only had a passing or surface level knowledge of every field, they could not add to our collective understanding of any field.
To actually advance human understanding in a field, an individual needs to catch up on everything that has been learned about it up to this point. For some fields, that is hundreds of years of research and knowledge gained. That's not practical to learn for more than one or two fields - we simply don't live long enough.
If you want the person who makes the next alloy that makes planes another 20% lighter, you're going to need someone who has dedicated most of their life to just learning material sciences and is putting all their effort into that topic. It isn't going to be done by someone who had one quarter of a semester in college on it. (With rare exceptions)
Me thinks you are mistaken.
In case of verbal miscommunication, you can always give the other party a second chance and ask them to repeat or rephrase their statement or you can rephrase yourself if you felt your originals statement was confusing.
In case of written communication there is no second chance to rephrase your statement. What's written is written, and that goes, regardless if it was well understood or not.
The student has to prove to the machine their capacity for intelligence on subject mater.
If anything their advantages over oral exams become higher.
I'd have to ask for a typewriter.
AI won't get them right, but sadly probably more than half of the humans won't as well.
Another thing is that huge part of the knowledge passed in schools can't serve as a basis for any sensible inference.
People go through all of university, ace their exams, then go to get hired in their first job and faceplant the interview in spite of having the qualification.
This is no surprise, they haven't ever needed to practice that skill!
Mostly it is the AI hallucination problem that stands in the way.
Furthermore, if AI continues to consume the world, who is interested in investing years into building up lifelong debt to only risk that everything you learned is now obsolete or irrelevant upon completion.
If knowledge can be easily acquired in 5 seconds, why are you testing people on whether or not they have memorized it? What good is 5 seconds? If the difference between an accredited engineer and a regular person is supposed to be a 5 second Google, then what the hell is the point?
We need to modernize assessment. We should have done it decades ago when cheating became rampant. Problem is that universities have zero incentive to fix this problem. Human resource teams don't either. The only people that really care are the people that have to work with everyone who gets hired despite having zero actual skills. Unfortunately, they have one of the quietest voices in this chain.
If you don't know much, you don't even know the right questions to ask your AI assistant, or the right tasks to have it perform. You'll also be less-able to check, and to apply, its output. The more you know, the more you can get out of your assistant.
At least so far, these tools are more like a bicycle than an internal combustion engine: they help almost anyone go faster and farther, but the people who are fit will still be able to go the fastest and farthest. Substitute "knowing stuff" for "fitness" and that's what we're looking at with "AI" (again, so far—if we achieve actual AGI so a huge set of complex tasks don't need a human involved at all, that may be another matter)
That's why people still need to know stuff they could find out in five seconds.
It sounds like you are agreeing with me here. We should not be making life-changing assessments of people's skills in unrealistic situations. In the real world, nobody gives you the question. Nobody gives you a breakdown of the tasks needed. And yet that is exactly how we assess people in these sorts of exams.
I agree with you on the AI part though. I like the bicycle analogy.
And how are you going to score students? Like autism with IQ160 that cannot talk but building Transformer model since elementary school something like that? Should you fail him/her just because doesn't like to talk and getting nervous in front of a bunch of examiners? And you sure you can score consistently against all students?
Of course human have been thinking about making a list of questions then ask a subset of it can somehow make the exam process more standardized. So you never think that you would end up asked the wrong question? And how it is standardized when you ask different people a difference question?
And guess what, those communicative sport guy or girls can always score higher than those introvert or even autistic self taught programmers. People just tend to judge others by the oral skills, not actual intellectual skills.
To be able to check that the student learned something.
I agree, it's a great way to assess students. I've been through them too, they are nerve wracking, but end up being a lot better than written.
But the lawyers, man, they'd hate it.
There's too much variability. Some students would get 'harder' questions than others. Some would get 'grilled' more in depth. Some would be asked to turn around and face a whiteboard more. Some students would say this is because I'm part of a certain category. And of those students, a lot would unfortunately be in the right. Because professors are just as biased as the rest of us, if not more so.
So, if you'd want to do this, the lawyers would say you'd need to have at least one person of every imaginable category in the room giving questions. And it would need to be an open exam (Pence rule), and they'd like for the exam to be recorded too, video and audio. And you've got to use a rubric with standard questions, so we can make sure there's no bias. Which means you have to sequester those that completed from those that haven't, phones included. And the proctors have to be behind a screen. And a few dozen waivers, just to round things off.
As others here have said, the expense would be huge. At least a handful of people, A/V techs, a revolving door of students, translators, the works. All just to end up asking students the exact same questions anyway.