Shameless plug but I made a similar tool called asXiv[1] which allows you to "ask" arXiv.org papers questions.
It also recommends questions on initial load that can help understand or explore the paper, here's a demo[2] from the popular Attention Is All You Need paper.
The code is all opensource[3], it uses the google 2.5 flash lite model to keep costs down (it's completely free atm), but that can be changed via env var if you run it locally.
I feel like this is thinking too small. I don’t want a better textbook. I want them to be basing this off of the experience of going to the most effective private tutors.
All the people at the forefront of AI really loved and thrived in highly academic settings from kindergarden to PhDs, their own lived experience doesn't match up with this product at all. Why are they making it?
EdTech has the worst returns of any industry in venture capital. Why?
There are no teachers who say that technology has generally improved experiences in classrooms, even if some specific technology-driven experiences like Khan Academy and Scratch are universally liked. Why?
When you look at Scratch, which I know a lot about, one thing they never do is allege that it improves test scores. They never, ever evaluate it quantitatively like that. And yet it is beloved.
Khan Academy: it is falling into the same trap as e.g. the Snoo. If you don't know what I'm talking about, it's about, who pays? Who is the customer? Khan Academy did a study that showed a thing. Kids are not choosing to watch educational YouTube videos because of a study. It is cozy learning.
But why does Khan Academy need studies for a test score thing? Why does Google? This is the problem with Ed Tech: the only model is to sell to districts, and when you sell to districts, you are doing Enterprise Sales. You can sometimes give them a thing that does something, but you are always giving them exactly what they ask for. Do you see the difference?
It doesn't matter if it's technology or if it is X or Y or Z: if the district asks for something that makes sense, great, and if it asks for something that doesn't make sense, or doesn't readily have the expertise to know what does and doesn't make sense, like with technology, tough cookie. Google will make something that doesn't make sense, if it feels that districts will adopt it.
We can go and try the merits of Learn Your Way, thankfully they provide a demo. All I'll say is, people have been saying, "more reading" is the answer, and there is a lot of fucking reading in this experience, but maybe the problem isn't that there isn't enough text to read. The problem is that kids do not want to read, so...
I'm a former physics teacher, and while I'm impressed by the technology, I think this is a low efficacy innovation.
The real challenge in teaching Newton's laws of motion to teenagers is that they struggle to deal with the idea that friction isn't always there. When students enter the classroom, they arrive with an understanding of motion that they've intuited from watching things move all their lives, and that understanding is the theory of impetus: https://en.wikipedia.org/wiki/Theory_of_impetus
An AI system that can interrogate individual students' understanding of the ideas presented and pose questions that challenge the theory of impetus would be really useful, because 'unteaching' impetus theory to thirty students at once is extremely difficult. However, what Google has presented here, with slides and multiple guess quizzes, is just a variation on the 'chalk and talk' theme.
The final straw that made me leave teaching was the head of languages telling me that a good teacher can teach any subject. Discussions about 'the best pedagogy' never make any consideration of what is being taught; there's an implicit assumption that every idea and subject should be taught the same way. School systems have improved markedly since they were introduced in the nineteenth century, but I think we've got everything we can out of the subject-agnostic approach to improvement, and we need to start engaging with the detail of what's being taught to further improve.
I wish instead Google would instead find a good way to have exercise books, with:
1) well-thought out exercises (covering all cases, whether in math or Spanish)
2) CORRECT solutions (just saying because even ChatGPT gets it wrong even for high school math)
3) that you can enter them using pen (if need be on an iPad)
Just a way to make zillions of exercises if I want to. And for my kids, the problem is these days teachers won't (AND mostly can't, they just don't know their subject) help them make a lot of exercises.
At first glance this seems fine, but pondering it for a moment, I think this is pretty bad. These analogies don't fully make sense. Also they kind of work on their own, but they clash together. A map is the same as a cookbook, and a list is the same as a recipe. A cookbooks contains recipes, so a map contains lists?
Figure 10 of the technical report https://services.google.com/fh/files/misc/ai_augmented_textb... has a third n=30 group labeled "LCG" that is mentioned nowhere else. It scores 100% on "I felt like today's educational tool helped me gain a good understanding of the content." so I wonder what's up with that.
No one who understands ai can rely on it to help us learn. I provided one with 100 citations I wanted to standardize and it deleted 10 and made up 10 to replace them. Can’t imagine this being used to replace a textbook or even explain a textbook.
Isn't this just showing the effects of actively engaging the learner by placing a topic in contexts familiar/favored by learner versus just reading about a topic?
Like, if you had made the text pdf readers do some manual thinking by working on trying to place the topic into the same type of familiar/favored context, wouldn't that have been the better comparison?
I think using GenAI for learning is cool and exciting (especially for autodidacts) but I'm not excited by this particular study structure.
I wonder how this will contribute to our current declining literacy rates in a social climate that's already rife with anti-intellectualism and isolation. Even if this worked well, it appears to be to be a step backwards.
Call me pessimistic, but this technology looks more poised to replace teachers in schools than supplement them.
The point of the rigidity and uniformity of school is not that it is the best way to get everyone to understand everything if they try. The point is that it forces all the kids to try. School is not just for the most interested, it is for everyone.
No AI you ever create will get a kid to choose learning how math works over doing basically anything else with their time. The point of school is not to teach, it is to discipline children to participate in education. Otherwise, why have it at all? Kids can find extensive information and guides for basically any topic they want on the internet right now.
what about just making textbooks with random info that is useful, i.e. no textbooks only on biology. just add random pages on all topics that people should know
I’m not sure this approach is the right one, but the problem resonates with me.
I vividly remember hitting some blocker (7th grade chem, 4th grade reading, 2nd grade dinosaurs), where I had a question that the teacher dismissed.
My mind was stuck (blocked) as it couldn’t get past the question I had, and in a public school setting, it wasn’t worth the time for the teacher to dive down the tangent (or they simply weren’t prepared).
My hope for LLMs in education, is that they can supplement traditional curriculums such that students can go “off the rails” while still being nudged back to the desired outcome.
- How do we know electrons “spin”?
- Why does that word behave differently than others (in English)?
- How big is a sauropod compared to a blue whale?
I’ve found that on my own journey through education, it’s these sparks of interest that drive towards deeper understanding, rather than surface level rote memorization.
TFA says: “What if students had the power to shape their own learning journey?”
In the context of nonfiction/textbooks, this is already possible!
I didn’t read “How to read a book”[0] until high school, but it opened the world for me on another silly blocker I had, which is that material should be consumed start to finish.
Hopefully with “AI” more students will learn that there are many paths towards understanding the world, and not just the curriculum in front of them.
Seems very similar when Microsoft invested in Apple back in the day when Apple was about to die.
Their concern was that they would be the only one OS company standing and be defacto a monopoly and regulated such. So, it was a away to keep your 'weak' competition alive just enough not to make you the sole provider.
Steve Jobs was able to turn around Apple in such a fashion that they become even bigger by letting go of the PC market and going mobile.
I'm excited by this multi-media approach. I've been avid self-learner for years, and I've often found the textbook format too dry, and forbidding, but ever since I started using NotebookLM, I'm diving into textbooks more and more. There is genuine value in creating a new format that meets learners where they are at.
This is one of the areas where LLMs are really useful. At their core they aren’t “thinking” so much as transforming and reorienting data.
What’s been most valuable for me is the way they create a kind of imperfect but effective Socratic dialogue with whatever I’m reading. I was the kid who always had my hand up in class, not to show off, but because I hated leaving something unexplained. Good teachers could make a text come alive by answering those questions.
LLMs give me some of that back outside the classroom. Even when I ask them to speculate, the process forces me to interrogate the material and refine my own model of it. That’s changed how I read, learn, and even how I experience novels.
So innovation on this “Socratic interface” and other interfaces is pointed in the right direction.
60 comments
[ 5.4 ms ] story [ 80.7 ms ] threadLink to an example:
https://learnyourway.withgoogle.com/scopes/AOcKkhsL/slides-n...
It also recommends questions on initial load that can help understand or explore the paper, here's a demo[2] from the popular Attention Is All You Need paper.
The code is all opensource[3], it uses the google 2.5 flash lite model to keep costs down (it's completely free atm), but that can be changed via env var if you run it locally.
1. https://asxiv.org
2. https://asxiv.org/pdf/1706.03762
3. https://github.com/montanaflynn/asxiv
EdTech has the worst returns of any industry in venture capital. Why?
There are no teachers who say that technology has generally improved experiences in classrooms, even if some specific technology-driven experiences like Khan Academy and Scratch are universally liked. Why?
When you look at Scratch, which I know a lot about, one thing they never do is allege that it improves test scores. They never, ever evaluate it quantitatively like that. And yet it is beloved.
Khan Academy: it is falling into the same trap as e.g. the Snoo. If you don't know what I'm talking about, it's about, who pays? Who is the customer? Khan Academy did a study that showed a thing. Kids are not choosing to watch educational YouTube videos because of a study. It is cozy learning.
But why does Khan Academy need studies for a test score thing? Why does Google? This is the problem with Ed Tech: the only model is to sell to districts, and when you sell to districts, you are doing Enterprise Sales. You can sometimes give them a thing that does something, but you are always giving them exactly what they ask for. Do you see the difference?
It doesn't matter if it's technology or if it is X or Y or Z: if the district asks for something that makes sense, great, and if it asks for something that doesn't make sense, or doesn't readily have the expertise to know what does and doesn't make sense, like with technology, tough cookie. Google will make something that doesn't make sense, if it feels that districts will adopt it.
We can go and try the merits of Learn Your Way, thankfully they provide a demo. All I'll say is, people have been saying, "more reading" is the answer, and there is a lot of fucking reading in this experience, but maybe the problem isn't that there isn't enough text to read. The problem is that kids do not want to read, so...
The real challenge in teaching Newton's laws of motion to teenagers is that they struggle to deal with the idea that friction isn't always there. When students enter the classroom, they arrive with an understanding of motion that they've intuited from watching things move all their lives, and that understanding is the theory of impetus: https://en.wikipedia.org/wiki/Theory_of_impetus
An AI system that can interrogate individual students' understanding of the ideas presented and pose questions that challenge the theory of impetus would be really useful, because 'unteaching' impetus theory to thirty students at once is extremely difficult. However, what Google has presented here, with slides and multiple guess quizzes, is just a variation on the 'chalk and talk' theme.
The final straw that made me leave teaching was the head of languages telling me that a good teacher can teach any subject. Discussions about 'the best pedagogy' never make any consideration of what is being taught; there's an implicit assumption that every idea and subject should be taught the same way. School systems have improved markedly since they were introduced in the nineteenth century, but I think we've got everything we can out of the subject-agnostic approach to improvement, and we need to start engaging with the detail of what's being taught to further improve.
1) well-thought out exercises (covering all cases, whether in math or Spanish)
2) CORRECT solutions (just saying because even ChatGPT gets it wrong even for high school math)
3) that you can enter them using pen (if need be on an iPad)
Just a way to make zillions of exercises if I want to. And for my kids, the problem is these days teachers won't (AND mostly can't, they just don't know their subject) help them make a lot of exercises.
"a list can be used for a recipe"
"a set can be used to list all the unique ingredients you need to buy for a week's meals"
"a map can be used for a cookbook"
"a priority queue can be used to manage orders in a busy restaurant kitchen"
"a food-pairing graph can show which ingredients taste good together"
Maybe I'm over-estimating the taste of 7th graders, but I feel like I would get sick of this really quickly.
Like, if you had made the text pdf readers do some manual thinking by working on trying to place the topic into the same type of familiar/favored context, wouldn't that have been the better comparison?
I think using GenAI for learning is cool and exciting (especially for autodidacts) but I'm not excited by this particular study structure.
Call me pessimistic, but this technology looks more poised to replace teachers in schools than supplement them.
No AI you ever create will get a kid to choose learning how math works over doing basically anything else with their time. The point of school is not to teach, it is to discipline children to participate in education. Otherwise, why have it at all? Kids can find extensive information and guides for basically any topic they want on the internet right now.
The entire "AI education" thing misses this.
I vividly remember hitting some blocker (7th grade chem, 4th grade reading, 2nd grade dinosaurs), where I had a question that the teacher dismissed.
My mind was stuck (blocked) as it couldn’t get past the question I had, and in a public school setting, it wasn’t worth the time for the teacher to dive down the tangent (or they simply weren’t prepared).
My hope for LLMs in education, is that they can supplement traditional curriculums such that students can go “off the rails” while still being nudged back to the desired outcome.
- How do we know electrons “spin”?
- Why does that word behave differently than others (in English)?
- How big is a sauropod compared to a blue whale?
I’ve found that on my own journey through education, it’s these sparks of interest that drive towards deeper understanding, rather than surface level rote memorization.
TFA says: “What if students had the power to shape their own learning journey?”
In the context of nonfiction/textbooks, this is already possible!
I didn’t read “How to read a book”[0] until high school, but it opened the world for me on another silly blocker I had, which is that material should be consumed start to finish.
Hopefully with “AI” more students will learn that there are many paths towards understanding the world, and not just the curriculum in front of them.
[0]https://en.m.wikipedia.org/wiki/How_to_Read_a_Book
Steve Jobs was able to turn around Apple in such a fashion that they become even bigger by letting go of the PC market and going mobile.
What’s been most valuable for me is the way they create a kind of imperfect but effective Socratic dialogue with whatever I’m reading. I was the kid who always had my hand up in class, not to show off, but because I hated leaving something unexplained. Good teachers could make a text come alive by answering those questions.
LLMs give me some of that back outside the classroom. Even when I ask them to speculate, the process forces me to interrogate the material and refine my own model of it. That’s changed how I read, learn, and even how I experience novels.
So innovation on this “Socratic interface” and other interfaces is pointed in the right direction.