HN Explorer

Ask HN: As a teacher, should I ban ChatGPT-generated assessments?

2 points by hacb ↗ HN
Since I started teaching 2 years ago, LLMs exploded and ChatGPT has become a thing. My position on LLM-assisted assessments has always been the same: I don't care. As a professional, you'll probably use those tools a lot (GitHub Copilot, etc.). So better start learning how they work and how to use them properly.

But I must admit that correcting and marking projects that have been mostly written by LLMs is kinda sad, as none of the codebases I saw was interesting to read. They are all just average, following the same patterns and idioms. I also see a lot of basic errors and misunderstandings, indicating that students simply copy-pasted stuff without even trying to understand what they are doing, or generating.

So what's the solution? As those assessments are do-at-home ones, the only thing I can think of for students to get more involved in the project is to make them a lot harder. As all the boilerplate code is now handled by ChatGPT and co, they will have to focus on the core of the project that can't be easily done by LLMs: projects requiring multiple source files, or longer OS-specific code with more constraints.

Grades are not here to evaluate the assessment per-se: grades are here to monitor learning and progress. When the assessments are done without any learning behind, then aren't the grades useless?

What do you think? Am I thinking wrong about it?

9 comments

[ 1.6 ms ] story [ 26.7 ms ] thread
jasiek-net
The point of school / university is to teach you how to think and program so it's different from commercial work (where you want to speed up things). So yes, I would force my students to write code by hand without any shortcuts, there will be time to use copilot or gpt ;)
hacb
I agree with you, but in my case I'm teaching programming, so doing table-top assignments is a no-go for me. I wrote too many C programs with a paper in a pen to know how bad it is. :)

That's why those assessments are tailored to be done at home, and then they send me an archive or a Git repo link.

bbarnett
I think the best you can do here, is to grade based upon what is in front of you.

If the code doesn't work, it means they (as you said), didn't understand what they copy and pasted. And as the whole point is to demonstrate learning, understanding, it is valid to mark such things as failures of that.

And this matches real world expectations too.

If there is any way you can tilt assignments to better demonstrate understanding, that's a win.

Hmm.

You could try to break up responses. By that I mean, have a dozen short coding exercises, but then tie them all together.

EG In the end, one bit of code to call the api/functions of the rest?

It might help break AI responses a bit.

hacb
Having multiple exercises that in the end get tied together is a pretty good idea! This would also highlight inconsistencies between different pieces of code that may indicate that the code has been generated and/or students are not understanding what they are doing.
MrVandemar
Writing 'C' programs on paper with a pen is actually a good way of learning for a certain percentage of students. Writing with a pen is a different cognitive activity than typing, and can aid in reinforcing knowledge.

So, one way to approach it would be to set the students to solve a task, say, one of the sorting algorithms. They can do this however they like.

Then in class you get them to write the same algorithm for a slightly different application thus (a) re-inforcing the algorithm, (b) demonstrating more than one application, and (c) pinpointing people who copied and pasted code without understanding and internalising the code.

Our 'C' teacher used to do this all the time (even before the internet) and when we got into exams, we did extremely well on them because we were copying out algorithms we knew off by heart and applying them to the exam questions.

[deleted]
(comment deleted)
ggm
Ask them to critique the code. Own up or fail.
[deleted]
(comment deleted)
kai_ferr
ChatGPT is super handy for us, students, but I can recall studying without it pretty clearly. Sometimes, I struggled with complex projects and ended up asking my teachers for help. But what really bugs me is when students don't even bother to check what they've generated. Back in my day, I had to go over tons of information by myself, but it helped me to remember stuff. If you just let an AI generate your work and don't even take a second look at it, what kind of learning is that, right?