If you read them, the skills are completely different (with the Jane Street skill being more complex). Socratic questioning [0] is a well-known technique. Indeed, a Github search returns 92 repositories for "socratic tutor skill" [1], suggesting that the well-known technique has been added to LLM tools independently by many people.
Same with dialog-tuning in the first place. That you could manipulate the context of unicorn-text generators like GPT-2 in order to actually have something like a conversation with them, or that a model could generate "private thoughts" before answering, were ideas which weren't as self-evident as they seem now. But still ideas a lot of people had at the same time.
Yep - just looked at my projects in Claude, and my first "socratic tutor" was registered on 2nd of December 2024. I forget where I got the inspiration though (probably X)
Yes, the comment was intended as a playfully hyperbolic nod to the lack of novelty and originality in the idea, but I guess the intended tone didn't carry through :)
This is really cool. I think we will see more of these, inching towards 'refined' thought and expression percolating from AI models into human minds.
I noticed that the quality of email text has gone up around my work these days. everyone is sounding like a master of drafting emails. Ofcourse Ai has a role in it. But real cool thing about this is, this is pulling up people's thinking to have a structure, minimalism and sequencing of expression. Same as how textile industry has enabled people to have an style and expression in clothing.
And, hilariously, as an AI booster, I personally dislike it, because now everyone sounds like they have a PR company ( and the same PR company come to think of it ). It was bad before in corporate land, but the level of standardization for optimized response is aggravating to me, when I am simply trying to have a short conversation.
Case in point, I reached about syllabus to my kid's teacher. Long story short, email ended in their boss, who sent me long winding non-answer that I had to resist to not knock down.
I dunno, generic executive summaries can be canned, but I would like to hope I pay them enough for an person to person exchange.
I have resorted to asking some people "just send me the prompt", like as messy as it might be, I will get the gist much faster than reading through Claudish or Gptinglsh which my eyes gloss over half the text
> everyone is sounding like a master of drafting emails
> this is pulling up people's thinking to have a structure, minimalism and sequencing of expression
I see the same for the former but the opposite on the latter. Everyone sounds like expert drafters because the AI is doing all of that. B
True to the "use it or lose it" principle, the people's thinking skills about the emails have weakened. They rely on giving the AI some bits of info, a document they never read, some vague instructions, and the AI molds the message with structure and sequence. Then I get to see it live when discussing with these people face to face and they still just spit out disparate bits of information with no fluid structure and sequence for the ideas because the only time they thought about the topic was when the AI wrote an email about it.
> Same as how textile industry has enabled people to have an style and expression in clothing.
I'd tend to agree with what you said, except this part. Mass produced clothing is fantastic for many reasons, but not for style and expression. Look at the traditional clothing from almost any part of the world and it was gorgeous and colorful. Now it's all just a t-shirts.
It was gorgeous and colourful, but highly impersonal. Traditional clothing, almost by definition, isn't really an enabler of personal style and expression. And you can't really compare traditional clothing to day-to-day clothing, these are entirely different purposes and codes.
Day-to-day clothing is definitely not "all just tshirts" nowadays, even in the mainstream. Women fashion is very varied, and while men's fashion is a lot poorer there's still a huge spectrum of expressive clothing that's available. I don't think personal style and expression has ever been as available and encouraged as it is today.
> It was gorgeous and colourful, but highly impersonal.
So is mass-produced clothing. Go into any store, and you will see the same shit with very few variations. For actual personality you have to search for niche/hand-made/small-productions etc.
BTW, each village was likely to have a different take on these clothes. And definitely each region. Today you'd be hard-pressed to tell an American, a European, and an Asian person apart based on their clothes alone.
Huh? At least where I from, you got cloth decorated by a person who did not produced copies the way machines do. That person was frequently you yourself.
> And you can't really compare traditional clothing to day-to-day clothing, these are entirely different purposes and codes.
I would interpret traditional clothing as "clothing they traditionally wore". It was not a uniform.
> Look at the traditional clothing from almost any part of the world and it was gorgeous and colorful.
Was it?
There's a lot of survivorship bias in old things; we don't see the things which weren't recorded. Old things had their style, that much is unavoidable, but before mass production normal people inherited their clothes because it was just that hard to create.
Also, a lot of the pigments we take for granted today didn't exist at any price in the pre-industrial era, and some which did involved trekking to a mine in Afghanistan: https://en.wikipedia.org/wiki/Ultramarine
spoke to a clothing historian in Europe years ago. Pigments, shoes, and washing have improved… But all clothes being tailored and more craftsman made the experience better.
She said she’d only choose the modern era because of the washing machine.
Clothing was extremely expensive historically relative to wages due to high labor requirements, so until the Industrial Revolution most clothing we now look back to and admire was the domain of the relatively wealthy. It might have had better craftsmanship, but many were literally in rags.
Before the stitching machine arrived in my village, all you can see was large rectangular pieces of cloths, delivered directly from the weaver craftsmen of the village, for men and women alike (only color difference - white for men and mostly dark green for women). The washermen still delivered clothes correctly across dozen's of families everyday, after washing. The famous Indian Saree is an example of unstitched clothing. Dhoti, lungi, turban and towel are other forms rectangular clothing.
People had cultivated an art in how the rectangular cloths are wrapped around them and they clothes are secured at their waist, or around their head. Talk to a Sikh or Afghan about turban.
It's not really good that email (and written content overall) quality has gone up. The problem is that there's tons of high quality content but no way to know which content is useful or good. Before we could know someone's personality from their style, now everyone's just wearing Balenciaga.
> But real cool thing about this is, this is pulling up people's thinking to have a structure, minimalism and sequencing of expression.
I don't agree. Writing introduces friction that forces you to think and reflect. If you let AI do that part, you are not only skipping the writing, but more importantly, the thinking.
I completely agree. The use of AI in programming and writing has enabled us to produce highly professional-looking output, yet we are losing our capacity for professional judgment.
That's been my conclusion trying to dive into the learning science for a learning tool I am building. Much like going to the gym, learning requires putting effort into producing an output. The fact there is a motorcycle next to every running track in every application is good for seeming like you did more effort, but not for actually getting better at the subject.
> I noticed that the quality of email text has gone up around my work these days. everyone is sounding like a master of drafting emails. Ofcourse Ai has a role in it.
I find that everyone _looks_ like they know how to write now, but the good writing has become even rarer and a big text is now indicative of laziness more than of work, which means that I get annoyed with big emails and messages, they are almost always purely AI
I dunno if you really need this. I asked AI to generate me a Lean tutorial with exercises and it did a really great job, one-shot with no skills. My 4th attempt to learn Lean and I'm actually making progress this time. (Mostly because you can say "I'm a bit stuck here, can you give me a hint" - it's really good at Lean even without access to the Lean compiler.)
Maybe it improves things a bit though. I might try it next time.
Socratic questions are not asked blindly. They are meant to help the student to realize the incompleteness or falsety of what they think they know. That us, to realise they do not actually know something and thus put them into a position of not knowing, which is the only way to gain understanding.
One man said, that enlightement (as in Zen) cannot be faked. Whoever had it, will be able to tell if someone else had it or not. There is no way to merely sound like it. Socratic method is like enlightement.
From my memory of enthusiastically working maybe 1/3 of Plato in seminar 25 years ago, if you read enough Socrates, you start to absorb a few base patterns he uses over and over again, much as you do in Euclid. This is me agreeing with you. You start to recognize conversational/logical pathways that are likely, however annoying or even sort of BS they feel at the beginning, to lead you to an impasse that will show you the boundaries of your operating area. "Hmm, Meno. Interesting question, but I'm actually not sure I know what knowledge even is. Since you know so much, maybe you can teach me." [Buckle up]
I think there could be some ways to fake this, using RAG or some other kind of agentic process, but in terms of the true meaning of the phrase, this is one of the things that is very far from being replaced by AI- an interaction between two people where a question that seems simple on the face of it is actually exactly what the learner needed to hear at the exact right time.
Very few teachers get to do this or are good enough, but it's just as much a question of relationships than anything else- how do you pose the exact right problem so that the learner can discover something for themselves.
It has another interesting AI-related angle. We do have an inner brain mechanism that is similar to AI. Every moment it reacts to the current input by scanning the accumulated "weights" and classify it, coming up with a canned answer. (It actually goes very deep, down to what we perceive as physical sensations.) And the point of Socratic questioning (and other techniques) is to turn that mechanism off! If we know we do not know something we won't take the canned "answer" because we know it is wrong. Instead we'll start just observing the thing and then, somehow, we gain something new. Which the internal LLM immediately tucks away into "weights".
And, of course, the input that stops the internal LLM cannot be some output of an external one. The teacher cannot give canned prompts; he must observe the pupil in the very state of not-knowing he wants to induce.
I think it was Plato who said that teaching is not transferring of some knowledge, it is lighting a fire.
Been using this with Opus 5.5 on Medium for a week or so. I've used the same model for learning before and I definitely prefer using this skill because it stops the model from the usual handholding. Although I'd say most of the effect can be replicated with this line: "When the user is wrong, do not give the answer. Ask a narrower question that exposes the contradiction in their reasoning."
Do you have readily available access to professionals in every field you're curious about? I'm interested in learning more about electricity but don't know anyone who's certified.
I wish I could just go to any university and talk to professors and have a socratic discussion about many subjects. Unfortunately doesn't work that way in reality.
I think it's a great idea, but I'm somewhat anti-skill.
Every single skill I see just strikes me as a prompt. In every case it's easier for me to create a description of what is needed specific to the task. If I need a longstanding process I will discuss with the agent and create a document.
This is one of the most flexible tools ever made, and I frankly am not going to accept unnecessary systematization of how I use it.
> Every single skill I see just strikes me as a prompt
Both of which are just blocks of text being stomped into an LLM.
I've got a feeling that prompting as we're doing currently (huge blocks of text with very little structure), at least when looking for a deterministic result, will be looked back on as if we were living in the Stone Age. Not sure what the alternative will be, but I'm imagining something like a CI pipeline definition in YAML.
A skill is usually auto-invoked and this one here has a bunch of trigger phrases. I personally dislike that. For this one especially I'd prefer explicit invocation.
its offtopic but that Programmed learning wikipedia article is really bad. It's clearly written by someone with personal stake in it and isn't remotely encyclopedic in tone.
Not sure what to do about it because I don't care enough to learn everything about this topic in order to fix it.
> Programmed learning (or programmed instruction) is a research-based system which helps learners work successfully. The method is guided by research done by a variety of applied psychologists and educators.
I have no idea what programmed learning is, only that it has an alternative name, that it has scientific street cred and that it is effective according to some nebulous definition of successful.
as opposed to:
> Mastery learning is an instructional strategy and educational philosophy that emphasizes the importance of students achieving a high level of competence (e.g., 90% accuracy) in prerequisite knowledge before moving on to new material.
which is gives me a more clear idea of what the core philosophy is behind it.
Can the programmed learning page be marked as low-quality, non-encyclopedic? That is already a good signal to have.
Programmed Learning (Programmed Instruction) goes back to instructional material created for the troops in WW2 so it has a history the predates both of those and was part of BF Skinner's research.
I see a lot of this at my company too - every re-inventing the same ideas over and over. I wonder how much people actually adopt other teams' implementation of this. In my experience I'm bombarded with a thousand different iterations of these things that everyone wants to advertise as a sort of "Look at me! I'm so AI!" but I simply don't have time to evaluate every different persons variation on these things, I certainly don't have time to really thoroughly benchmark them, and from experience a lot what I needed a carefully tuned skill to achieve 6 months ago is trivial for a better tuned agent today. I'm literally getting to the point where even my own skills are beginning to bit-rot.
I also like this skill. It's one of the few I've used the longest: more than a year now which is a century in LLM years. Although you have to remember to ask the LLM to fill its context before starting the questions.
Especially nice when you are not so sure how something should look like, so it makes you stop and spend a few hours answering questions and thinking about what you want to do. Combine that with the RFC skill and you get a good basis for your feature and then a skill to split the RFC to issues on Linear.
Trying it right now. it's pretty decent and it does sort of make my brain work. Pondering right now when the "bitter lesson" will make these skills useless.
Good thing with skills being just markdown text you can use them in any harness. Just imported into my phone harness (I already had a Socratic tutor like skill), but this crispier.
Oh this is so good. I asked it to teach me programming in Logo; though the model is not the best (GLM 4.7, on pi), the tutor is asking very good questions and is not too pedantic about exact answers.
For those new (beginners), install pi from pi.dev and then your favorite AI model (ask your model how to set up pi and it). Then ask your pi-based AI how to set up the tutor (just give it this link, it will set it up for you).
Then start pi by typing pi in the terminal; and to launch the tutor preface your question with: /skill:socratic-tutor. Like so:
183 comments
[ 1.3 ms ] story [ 113 ms ] thread[0]: https://en.wikipedia.org/wiki/Socratic_questioning [1]: https://github.com/search?q=socratic+tutor+skill&type=reposi...
My client just got done firing someone who thought filling the codebase up with files like this would be a good idea.
It would seem the AI revolution is happening at wildly different rates depending on where you are standing.
This is for the case where you the human want to learn something, instead of spewed out for you.
I noticed that the quality of email text has gone up around my work these days. everyone is sounding like a master of drafting emails. Ofcourse Ai has a role in it. But real cool thing about this is, this is pulling up people's thinking to have a structure, minimalism and sequencing of expression. Same as how textile industry has enabled people to have an style and expression in clothing.
Case in point, I reached about syllabus to my kid's teacher. Long story short, email ended in their boss, who sent me long winding non-answer that I had to resist to not knock down.
I dunno, generic executive summaries can be canned, but I would like to hope I pay them enough for an person to person exchange.
> this is pulling up people's thinking to have a structure, minimalism and sequencing of expression
I see the same for the former but the opposite on the latter. Everyone sounds like expert drafters because the AI is doing all of that. B
True to the "use it or lose it" principle, the people's thinking skills about the emails have weakened. They rely on giving the AI some bits of info, a document they never read, some vague instructions, and the AI molds the message with structure and sequence. Then I get to see it live when discussing with these people face to face and they still just spit out disparate bits of information with no fluid structure and sequence for the ideas because the only time they thought about the topic was when the AI wrote an email about it.
Calling this “thought” is an injustice.
I'd tend to agree with what you said, except this part. Mass produced clothing is fantastic for many reasons, but not for style and expression. Look at the traditional clothing from almost any part of the world and it was gorgeous and colorful. Now it's all just a t-shirts.
Day-to-day clothing is definitely not "all just tshirts" nowadays, even in the mainstream. Women fashion is very varied, and while men's fashion is a lot poorer there's still a huge spectrum of expressive clothing that's available. I don't think personal style and expression has ever been as available and encouraged as it is today.
So is mass-produced clothing. Go into any store, and you will see the same shit with very few variations. For actual personality you have to search for niche/hand-made/small-productions etc.
BTW, each village was likely to have a different take on these clothes. And definitely each region. Today you'd be hard-pressed to tell an American, a European, and an Asian person apart based on their clothes alone.
> And you can't really compare traditional clothing to day-to-day clothing, these are entirely different purposes and codes.
I would interpret traditional clothing as "clothing they traditionally wore". It was not a uniform.
Was it?
There's a lot of survivorship bias in old things; we don't see the things which weren't recorded. Old things had their style, that much is unavoidable, but before mass production normal people inherited their clothes because it was just that hard to create.
Also, a lot of the pigments we take for granted today didn't exist at any price in the pre-industrial era, and some which did involved trekking to a mine in Afghanistan: https://en.wikipedia.org/wiki/Ultramarine
She said she’d only choose the modern era because of the washing machine.
People had cultivated an art in how the rectangular cloths are wrapped around them and they clothes are secured at their waist, or around their head. Talk to a Sikh or Afghan about turban.
Please elaborate.
I’m not reading all that. I want one word answers.
Also noticed huge uptick in “call me” emails.
That’s a very good analogy!
I don't agree. Writing introduces friction that forces you to think and reflect. If you let AI do that part, you are not only skipping the writing, but more importantly, the thinking.
Here's some science to describe this intuitive process: On the effect of ChatGPT on classrooms: https://www.mdpi.com/2078-2489/17/3/299 On misatributing insight to LLMs: https://arxiv.org/abs/2604.10288 And the foundational literature before LLMs on removing friction from learning: https://bjorklab.psych.ucla.edu/wp-content/uploads/sites/13/...
I find that everyone _looks_ like they know how to write now, but the good writing has become even rarer and a big text is now indicative of laziness more than of work, which means that I get annoyed with big emails and messages, they are almost always purely AI
Maybe it improves things a bit though. I might try it next time.
One man said, that enlightement (as in Zen) cannot be faked. Whoever had it, will be able to tell if someone else had it or not. There is no way to merely sound like it. Socratic method is like enlightement.
I think about this, whenever I have questions which are related to morality and how people act.
I think this same phenomenon — faking competence, or good intentions — extends beyond spirituality as well.
Very few teachers get to do this or are good enough, but it's just as much a question of relationships than anything else- how do you pose the exact right problem so that the learner can discover something for themselves.
And, of course, the input that stops the internal LLM cannot be some output of an external one. The teacher cannot give canned prompts; he must observe the pupil in the very state of not-knowing he wants to induce.
I think it was Plato who said that teaching is not transferring of some knowledge, it is lighting a fire.
Every single skill I see just strikes me as a prompt. In every case it's easier for me to create a description of what is needed specific to the task. If I need a longstanding process I will discuss with the agent and create a document.
This is one of the most flexible tools ever made, and I frankly am not going to accept unnecessary systematization of how I use it.
Both of which are just blocks of text being stomped into an LLM.
I've got a feeling that prompting as we're doing currently (huge blocks of text with very little structure), at least when looking for a deterministic result, will be looked back on as if we were living in the Stone Age. Not sure what the alternative will be, but I'm imagining something like a CI pipeline definition in YAML.
Isn't this just what a skill is?
Not sure what to do about it because I don't care enough to learn everything about this topic in order to fix it.
> Programmed learning (or programmed instruction) is a research-based system which helps learners work successfully. The method is guided by research done by a variety of applied psychologists and educators.
I have no idea what programmed learning is, only that it has an alternative name, that it has scientific street cred and that it is effective according to some nebulous definition of successful.
as opposed to:
> Mastery learning is an instructional strategy and educational philosophy that emphasizes the importance of students achieving a high level of competence (e.g., 90% accuracy) in prerequisite knowledge before moving on to new material.
which is gives me a more clear idea of what the core philosophy is behind it.
Can the programmed learning page be marked as low-quality, non-encyclopedic? That is already a good signal to have.
[1] https://en.wikipedia.org/wiki/Active_learning
[2] https://en.wikipedia.org/wiki/Chunking_(psychology)
B.F Skinner. Teaching machine and programmed learning https://www.youtube.com/watch?v=jTH3ob1IRFo
Using LLMs this way was really refreshing, I could really feel how I started to think on my own again.
Especially nice when you are not so sure how something should look like, so it makes you stop and spend a few hours answering questions and thinking about what you want to do. Combine that with the RFC skill and you get a good basis for your feature and then a skill to split the RFC to issues on Linear.
Running it with my local models it is a pretty helpful way to study up something where you already have some bases.
For those new (beginners), install pi from pi.dev and then your favorite AI model (ask your model how to set up pi and it). Then ask your pi-based AI how to set up the tutor (just give it this link, it will set it up for you).
Then start pi by typing pi in the terminal; and to launch the tutor preface your question with: /skill:socratic-tutor. Like so:
/skill:socratic-tutor teach me Python.