As long as we keep having to ask, "ChatGPT answered this questions but was it right?" it is a nonstarter. It is currently a novelty, not a tool. To be a tool it needs to be reliable.
In most subjects I have found ChatGPT to be fairly reliable. Are there specific scenarios that you have found concerning?
I think perfect reliability is not absolutely necessary. None of the human teachers that I have learned from were perfectly reliable even in their area of expertise.
Thankfully, skepticism seems to be an innate human trait.
> In most subjects I have found ChatGPT to be fairly reliable.
Not the OP, but I am very curious what subjects you have experimented with and whether you used any special prompt techniques.
My experience has been quite on the contrary: on the few subjects I have some understanding of, ChatGPT appears to do fine for basic questions and attempts to summarize concepts at a high-level, but any attempt at drilling in to specifics seem to cause it to hallucinate quite a bit, particularly with regards to concrete facts or citations or anything of that sort.
Most of my questioning have been practical questions relating to computer science / programming and language learning (English / Spanish). I have asked a lot of other general knowledge questions, but nothing exceptionally deep.
Can you give more detail as to the hallucinations that you found?
Prompt: Cite three important academic journal articles about cybersecurity, including DOI numbers
Response:
1. Anderson, R., & Moore, T. (2006). The Economics of Information Security. Science, 314(5799), 610-613. DOI: 10.1126/science.1130992
2. Schneier, B. (2015). Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World. WW Norton & Company. DOI: 10.1080/13600834.2015.1093541
3. Fung, B., & Graham, M. (2019). AI and cybersecurity: Safety, privacy, and security in the age of AI. IEEE Security & Privacy, 17(3), 10-14. DOI: 10.1109/MSEC.2019.2904199
#1 is good. #2 is a book (the DOI points to nothing). I can't find any evidence #3 exists. I don't consider this a difficult question to ask of cybersecurity students, let alone teachers.
I guess I see ChatGPT as a very broadly knowledgeable, yet fallible person, not as a database. It would be a very rare person who could come up with the data you asked for off the top of their heads. Granted, it would be much better if it could tell you "I don't know", rather than hallucinate facts.
As far as it being able to explain base concepts in a conversational manner and helping bridge gaps in my own understanding, it has been very helpful to me.
I'm sure there are certain subjects where this will work better than others, but I would guess especially for most K-12 classwork, ChatGPT even with its current warts would be a net gain for many students in helping to increase their understanding just by talking with it.
ChatGPT does not get full credit on the homework I create, even when the topic is some standard-ish algorithm with significant presence in the training data. Or maybe I'm just a bad prompt engineer.
MS in computer science. The specific question I was surprised about was a O(log (m+n)) algorithm for finding the median of the union of two sorted arrays. This is in-scope for elite leetcoders and similar. The code ChatGPT generated had some issues resulting in out of bounds array accesses. It also wasn't clear if the algorithm was the desired log(m+n) one or a simpler but inferior log(m) + log(n) one. Was the model was confused and blended parts of different approaches? Is that statement too anthropomorphic for a LLM?
I’m still on the waitlist but I hear hallucinations have been largely solved already by connecting it to the internet with ChatGPT Plugins and telling it to verify its sources.
OK. Let’s keep the context of this conversation in mind. As is typical for these sorts of comments, you slowly move the goalposts from “non-starter”, to “my arbitrary idea of ‘good’, to “superhuman”.
At my day job we get a pretty good idea of the shortcomings of a typical teacher. We are also getting a very good, measurable, idea of what LLMs are capable of in this area. There’s skeptical, and there’s being in denial.
I've been using ChatGPT for language learning by asking it to explain the differences between words. The answers align with what the dictionary says but are far more elaborate and better nuanced.
It wouldn't be possible to get equivalent explanations from native speakers unless you pay them to teach.
Same. I’ve been using it quite a bit for work, but I’ve also been just playing/experimenting with it and many of my questions have come from knowing a tiny bit of Japanese and watching anime. It’s great for explaining differences between words or phrases that translate similarly.
Why ChatGPT? It can be other AI solutions. AI in general is going to change education. It’s going to make world-class education available to everyone. It’s going to rewire our brains.
>> In the past, Donahoe would set her students to writing assignments in which they had to make an argument for something—and grade them on the text they turned in. This semester, she asked her students to use ChatGPT to generate an argument and then had them annotate it according to how effective they thought the argument was for a specific audience. Then they turned in a rewrite based on their criticism.
That won't work. The student can just ask ChatGPT to annotate the argument it generated itself previously (or in the same session).
Basically, as long as you're asking your students to turn in a written assignment, they can use ChatGPT to write the assignment. It doesn't really matter what you ask them to write, and how much of it you ask them to generate by ChatGPT or by the student's hand, the student can choose to have the whole thing generated by ChatGPT and there's nothing you can do about it.
So what's the problem? If ChatGPT can do all that stuff then there's no reason to teach students how to do it. Same reason we don't teach assembly in programming classes anymore, since compilers can do that for us.
Instead, students can now move on to more complex things. For example, maybe the teacher can have their 2nd graders use ChatGPT to reproduce a research paper on quantum computing and submit their results to a peer-reviewed scientific journal.
Not just a research paper, anything! The 2nd graders -hell, the 1st graders!- can use ChatGPT to submit articles to wikipedia. They can use it to submit articles to news outlets. They can use it to generate recipes to submit to recipe aggregators. The possibilities are truly endless!
In fact, the people working in those publications and tasked to review all the submissions, don't have to do that anymore. They can just feed the submissions to ChatGPT, and let ChatGPT generate a review for them.
See, there is no read to write anything anymore, and there's no need to read anything anymore, either. We can just let ChatGPT do all that stuff for us!
Of course, a few years down the line, there won't be any reason to submit, or publish, articles or anything else anymore, either, since there won't be anyone left who can actually read, besides ChatGPT, and that would be pointless if it will be ChatGPT that's produced all the writing already. But, I'm sure that all this will sort itself out in the long run.
27 comments
[ 0.26 ms ] story [ 66.8 ms ] threadI think perfect reliability is not absolutely necessary. None of the human teachers that I have learned from were perfectly reliable even in their area of expertise.
Thankfully, skepticism seems to be an innate human trait.
Not the OP, but I am very curious what subjects you have experimented with and whether you used any special prompt techniques.
My experience has been quite on the contrary: on the few subjects I have some understanding of, ChatGPT appears to do fine for basic questions and attempts to summarize concepts at a high-level, but any attempt at drilling in to specifics seem to cause it to hallucinate quite a bit, particularly with regards to concrete facts or citations or anything of that sort.
Can you give more detail as to the hallucinations that you found?
Response:
1. Anderson, R., & Moore, T. (2006). The Economics of Information Security. Science, 314(5799), 610-613. DOI: 10.1126/science.1130992
2. Schneier, B. (2015). Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World. WW Norton & Company. DOI: 10.1080/13600834.2015.1093541
3. Fung, B., & Graham, M. (2019). AI and cybersecurity: Safety, privacy, and security in the age of AI. IEEE Security & Privacy, 17(3), 10-14. DOI: 10.1109/MSEC.2019.2904199
#1 is good. #2 is a book (the DOI points to nothing). I can't find any evidence #3 exists. I don't consider this a difficult question to ask of cybersecurity students, let alone teachers.
Apparently GPT-4 is much more accurate and natural in its responses.
As far as it being able to explain base concepts in a conversational manner and helping bridge gaps in my own understanding, it has been very helpful to me.
I'm sure there are certain subjects where this will work better than others, but I would guess especially for most K-12 classwork, ChatGPT even with its current warts would be a net gain for many students in helping to increase their understanding just by talking with it.
https://www.reuters.com/technology/australian-mayor-readies-...
EDIT: I commented that is often wrong when talking about RF analog circuits or motor control, but this is a rabbit hole. It's like a bad textbook.
At my day job we get a pretty good idea of the shortcomings of a typical teacher. We are also getting a very good, measurable, idea of what LLMs are capable of in this area. There’s skeptical, and there’s being in denial.
It wouldn't be possible to get equivalent explanations from native speakers unless you pay them to teach.
That won't work. The student can just ask ChatGPT to annotate the argument it generated itself previously (or in the same session).
Basically, as long as you're asking your students to turn in a written assignment, they can use ChatGPT to write the assignment. It doesn't really matter what you ask them to write, and how much of it you ask them to generate by ChatGPT or by the student's hand, the student can choose to have the whole thing generated by ChatGPT and there's nothing you can do about it.
Instead, students can now move on to more complex things. For example, maybe the teacher can have their 2nd graders use ChatGPT to reproduce a research paper on quantum computing and submit their results to a peer-reviewed scientific journal.
In fact, the people working in those publications and tasked to review all the submissions, don't have to do that anymore. They can just feed the submissions to ChatGPT, and let ChatGPT generate a review for them.
See, there is no read to write anything anymore, and there's no need to read anything anymore, either. We can just let ChatGPT do all that stuff for us!
Of course, a few years down the line, there won't be any reason to submit, or publish, articles or anything else anymore, either, since there won't be anyone left who can actually read, besides ChatGPT, and that would be pointless if it will be ChatGPT that's produced all the writing already. But, I'm sure that all this will sort itself out in the long run.