Ask HN: How has AI changed your learning methods?
While doing some research on a topic, I was wondering about the impact of AI on traditional study methods.
Any real-world experiences from students or teachers? Is there any research on the impact of AI on learning outcomes?
46 comments
[ 2.9 ms ] story [ 91.3 ms ] threadHaving learned before and after ChatGPT, the workload of students has remained the same, but some can obviously be done with AI, and most students do this, or have tried it.
I believe as a result, the efficacy boost has been offset by the lower amount of time people spend studying - just like most historical studying technologies. For every 1 student who uses AI to learn, there is 2-3 who prefer just to use it to cheat. But it works brilliantly for every type of student for basic explanations, run downs of historic authors or positions, etc. But this is pretty much just wikipedia stuff re-arranged to your learning level. It's helpful, but not augmenting.
Also lately I've been taking photos of stuff like coffee grinders and making it guess what it is. It's surprisingly very accurate and you can use it to explore the thought processes of why someone might pick a particular set.
The danger I see here, is that if you ask an LLM to explain the thought processes, it will never say “I don’t know”. It will instead describe some thought processes associated with coffee grinders. It may say something like “this grinder has fine grain controls that allow customizing the size of grind.” …which that particular grinder doesn’t have at all…but that’s a thing people write about when choosing grinders. The frustration is that 90% of the answer will be accurate, but somewhere in all the sentences is a hallucination, treated with the exact same authority as the rest of the answer.
It’s very difficult to QA that type of error.
Mahlkönig EK43 was the correct answer, but the combo is unusual, because people will usually get a couple of La Marzoccos from a supplier who holds both, and Mahlkönig is an unseen brand here. Why go to the trouble? La Marzocco makes good enough grinders.
With further interrogation, ChatGPT was absolutely insistent on it being a EK43, a limited edition model called The Icon, which was notable for its white color and gold trimmings. This kind of precision is easy to verify, but it's not a detail that comes up in a Google search for the EK43.
The correct answer was that the coffee shop's owner's mentor was from Vietnam, where the EK43 was more common. It was particularly good for making complex latte art like unicorns as it has a precision that allowed controlling the acidity of the crema.
But this whole thought process was just wild. I want it to give me all kinds of crazy answers. I want it to have a high miss rate. It's perfect for brainstorming. But you need sufficient expertise to guide ChatGPT to the next answers.
Does ChatGPT give you an authoritative sounding answer about chemicals, or does it point out the possibility of having sensitive teeth and that it's the temperature of the ice-cream, not any chemicals, that are making your teeth hurt?
I switch between two toothpastes. One is better for teeth whitening, the other lets me eat ice cream. The sensitive toothpaste are just too safe for me, it doesn't have that minty taste. And I can A/B test between two different ones, even of the same brand, to see which chemical is worse for it.
Anyway, to shorten its answer, it's triclosan, sodium lauryl sulfate, sodium hydroxite, or possibly some of the flavorings.
Just as ChatGPT became more available, these subreddits decided to make their posting policies unnecessarily strict. The "answering culture" has also become more hostile, with people downvoting questions that don't fit into the monoculture of Reddit's hivemind.
And so I have found ChatGPT to be very useful for asking about philosophical and historical questions, specifically asking for resources on a particular topic/problem. E.g., "Has any philosopher written about XYZ topic?" It will sometimes give me imaginary resources, but usually it'll recommend actual books written on the subject.
I've even been working on a tiny open source wrapper around the OpenAI API specifically to speed this process up, based on what I've learned works from experience: https://github.com/hiAndrewQuinn/wordmeat
As a side project I am currently building a drone myself on a really tight budget. While I am pretty good on the coding side, my understanding of electronics is basically non-existent. So when I am asking basic questions it's quite helpful; as soon as I am giving it specifications ("Will this brushless motor and this ECS work with a 4S lipo?") it breaks down completely - so it's helpful, but far from perfect.
Language models are exceedingly good at understanding the meaning of your language without the use of specific keywords. Here's an example from a recent search I did.
> metals can be flexed or bend, and it will regain some of its prior shape
in Google returns either "ductile" or "shape-memory alloy," which are both incorrect.
> What is the property of a material where it prefers to stay in its current form? This is often found in metals, where it can be flexed or bent, and it will regain some of its prior shape?
in ChatGPT correctly explains that
> The property you are referring to is called elasticity. Elasticity is the ability of a material to return to its original shape after being deformed (stretched, compressed, bent, etc.) when the external forces are removed.
We all know that LLMs can hallucinate, and they are therefore not a reliable source of truth or knowledge. I'm not necessarily trying to say that LLMs are more accurate than something like Google's knowledge engine. The value in LLMs is that they can infer your meaning to some degree of accuracy (just like asking a human) so that you can productively continue your own research in more depth.
I've found the same for setting up my environments. Before little things that were minor quality of life issues that I just didn't know what to search for, or couldn't be bothered to go hunting down in the docs, I can easily give an example and ask questions about. Little things that I put up with for years are all now resolved and I'm realizing how much nicer having a lot of little things not worth dealing with individually gone is. I don't have to put up with little things because I can't context switch for a day to figure them out, but I can spend 10 minutes with AI to resolve them without much brain power/context switching on my part.
In the IDE/tools/frameworks I use so many little things that 'it would be nice if it did this, but I don't have the time to figure out' are now resolved.
That said, it is actively harmful when discussing components of Chinese characters - it hallucinates so much it's essentially unusable. I stick to traditional resources for that. I'm also reading as many scientific papers as before, there's really no substitute for that yet, and I haven't found it very good at literature searches.
Do you mean, it helped you write software that replaced Anki the program or something else? Intrigued
What did you use for scheduling? I've seen ebisu [0] used before but found it difficult to grok. I can say as I've used FSRS [1] within Anki I've started to like its decisions alot.
I think the main thing that keeps me from rolling my own Anki is mobile support. I had an Android app once and having to keep up-to-date with all the changing App Store requirements was annoying as hell. Eventually they took down the app for some compliance thing.
[0] https://github.com/fasiha/ebisu [1] https://github.com/open-spaced-repetition/fsrs4anki
Mobile support was a concern of mine as well but I just took the easy way out and make a simple SPA in javascript and futzed with the CSS until it looked good on my phone (also mostly ChatGPT). One explicit design principle was that I will never support anyone but myself, which simplified things greatly. The SPA interacts with a little CRUD app that uses a sqlite database to store my cards and activity.
Day to day I write Lean, and I use moogle.ai to find theorems. It's... fine as a first pass. The website constantly gets confused about similar-looking theorems, it can't pattern match, and it can't really introspect typeclasses (which can be hiding the theorems I want). However it usually can usually help me go from a vague description of what I want to some relevant doc pages, so credit where it's due for that.
One of my research interests is on how humans use expert systems (akin to how Go players’ ELO ramped significantly after the release of AlphaGo).
So I start with basic questions in chat gpt. I know it’s lying to me. But it uses words and phrases that seem interesting. I can iterate over that, and in a short time I know a phrase I can actually search for in google to find authoritative content.
I wouldn’t quite call it a game changer yet, as all it’s done is give me back what I once had. But it is a bonus in that it can do an ok job synthesizing new examples from content that already had lots of disperate examples on the web. It can also give some clues when you find conflicting information as to why the information is conflicting. It’s making stuff up a lot, but there are always good clues in the output.
I also spend too long clarifying what I mean.
For example, I wanted for a Rust program to detach into the background, and ChatGPT (with my stupid prompting) kept suggesting I just run `std::process::Command::new("program")`, but I want a single executable to detach! Eventually, once I struck the right chord, it suggested the `daemonize` crate. But it wasn't until after I'd found that by conventional search.
I sometimes use the Kagi !fgpt pattern if I know that what I'm searching for has a good average answer. It'll give that answer and skip the blinking ads, cookie pop-ups, newsletter popups, and autonomously scrolling on my behalf.
I'm looking forward to having an offline AI assistant that'll search and accumulate, rather than hallucinate answers from a bunch of stolen code snippets that akin to "copy-pasting from StackOverflow, but with hallucinations."
- I regularly read technical texts, some related to math, some related to programming/software engineering
- I ask Chatgpt4 (now Claude Sonnet 3.5) to quiz me on the nuances of topics X, Y, Z
My prompt template looks like so:
"I'd like to test my understanding of {topic}. I'd like you to quiz me one question at a time. Ask nuanced questions. Answers need to be multiple choice that I can pick from. Avoid congratulatory tone in responses. If I pick the right choice for a question, move on to next one without asking. Provide detailed explanations in case I answer something incorrectly."
I've found this surprisingly effective at pointing out gaps in my understanding. Ymmv.
All that said I'm very excited for the future and look forward to these problems being solved as I believe they eventually will.
I know that the LLM provided specifics are almost never good enough for a final answer. However, LLMs can get me to think out of my own personal box.
Basically, this has replaced what I once used to get out of being surrounded by human peers. However, I was reticent to bother humans, and I have no such reservations about asking a chatbot a dumb question.
Then, after the answer, I ask follow-up questions. I also try to check the answers against other sources, e.g. docs or Wikipedia in order to spot hallucinations.