I think the implication is that to be interesting you need to write from an individual's standpoint. That's why fiction written by LLMs sounds so boring (at least right now): because you can't amalgamate all the text in the world and not sound like an average.
> ‘Oh, do let me go on,’ said Wilde, ‘I want to see how it ends.’
People are average on average. OP is measuring LLM succes based on a super human test which most of us would likely fail. Creativity is just longer context and opinionated prompting. (For discussion purposes. I’m on 70% true.) Average Joe LLM and me are having a great time.
Not really on the topic of the FA, but I've heard a few times about the All Souls Exams and seen some sample essay prompts, and I would love to read some real essays written by test takers. Any pointers?
> The ultimate example may be All Souls College, which has a ritual, the Mallard Song, that occurs once a century.
You can't walk for more than five minutes in the UK without tripping over some nonsense like this. History is very important, and traditon has its place, but really? As a brit I find it all kind of tediously performative sometimes.
I sat the All Souls exam, taking the philosophy specialist papers, though I'm a math/physics/ML guy. It was a lot of fun, I really appreciate that there's somewhere in the world where these kinds of questions are asked in a formal setting. My questions/answers are written up in brief here [1]
I went to see the last Mallard Song. Just to say I went, of course. It looked like a bunch of weirdos in a courtyard to me, but it was a literally once-in-a-century event, and I was living less than a minute away, so why not?
I don't think I've ever heard of a scheduled ritual that has a longer period. You're guaranteed to never have anyone present at more than one of these, so surely many aspects of the ritual will wander quite far from the original?
As for LLMs on the All Souls test, it's predictable that it mostly whiffs. After all it takes in a diet of Reddit+Wikipedia+etc, none of which is the kind of writing they are looking for.
Reddit is a lot of crappy comments. If you have no grounding in reality (being a thing that lives in a datacentre), how are you going to curate it? Some subs are really quite good, but most are really quite bad. It's not easy to get guidance, of the kind you would get if you sat with a professor for three or four hours a week for a few years, which is what the humanities students actually do.
Wikipedia is a great reference work, but it tends to not have any of the kinds of connections you're supposed to make in these essays. It has a lot of factual stuff, so questions about Persia will look ok, like in the article. But questions that glue together ideas across areas? Nah. Even if that's in the dataset somewhere, how is the LLM supposed to know that the sort of refined writing of a cross-subject academic is the highest level of the humanities? It doesn't, so it spits out what the average Redditor might glue together from a bit of googling.
A few years ago, the Turing Test was universally seen as sufficient for identifying intelligence. Now we’re scouring the planet for obscure tests to make us feel superior again. One can argue that the Turing Test was not actually adequate for this purpose, but we should at least admit how far we have shifted the goalposts since then.
The LLM examples for "Water" surely put it in the top 10% of people (let's say, of adult native English speakers who are literate by UNESCO standards). The average person can't string two written sentences together, never mind write a coherent essay "from an opinionated, individual point of view" in a single draft.
That might still make it the worst candidate in the All Souls exams, because those obviously select for people who are interested in writing essays of this sort.
But I'm also curious whether the LLM could compete given a suitable prompt. If it was told to write an idiosyncratic, opinionated essay, and perhaps given a suitable source material - "you are Harry Potter" but someone less well known but still with a million words of backstory - couldn't it do it? The chat bots we have today are bland because we value blandness. Customers are willing to pay for the inoffensive corporate style that can replace 90% of their employees at writing. Nobody is paying billions of dollars for a Montaigne or a Swift or even a Paul Graham to produce original essays.
This was good, the tldr point is LLMs suck at natural writing, particularly long form. Or more abstractly they don't have complex original ideas, so can't do anything that requires this.
It's not surprising as it's very hard to train for or benchmark.
Also should add I don't think anyone serious thinks that long form writing or ideation is what they're for - assuming an LLM would be good at this is a side effect of anthropomorphism / confusion. It doesn't mean an LLM isn't good at summarizing something or changing unstructured data into structured or all of the other "cognitive tasks" that we expect from AI.
The author seems impressed with Claude's job answering the Achaemenid Persia question, but, just taking a look at it, if I had started a conclusion paragraph with "In conclusion," in my more rigorous university courses, I'd have been pilloried for it.
> The same patterns emerge for all LLM answers to these questions. They converge on an optimal path through a thicket of concepts.
This so concisely explains most of the problems and power of these tools. If your goal is to get a reasonably good answer on a reasonably well trod subject you’re going to be very happy with their output. If you push them outside of that they quickly fall into either producing reasonable sounding but incorrect outputs, hallucinations, or failure.
16 comments
[ 3.0 ms ] story [ 35.3 ms ] thread> ‘Oh, do let me go on,’ said Wilde, ‘I want to see how it ends.’
Pretty great line.
[1] https://www.asc.ox.ac.uk/people
You can't walk for more than five minutes in the UK without tripping over some nonsense like this. History is very important, and traditon has its place, but really? As a brit I find it all kind of tediously performative sometimes.
[1] https://www.reddit.com/r/oxforduni/comments/q0giir/my_all_so...
* Oops, they link to my post at the bottom. Sorry for the redundancy.
I don't think I've ever heard of a scheduled ritual that has a longer period. You're guaranteed to never have anyone present at more than one of these, so surely many aspects of the ritual will wander quite far from the original?
As for LLMs on the All Souls test, it's predictable that it mostly whiffs. After all it takes in a diet of Reddit+Wikipedia+etc, none of which is the kind of writing they are looking for.
Reddit is a lot of crappy comments. If you have no grounding in reality (being a thing that lives in a datacentre), how are you going to curate it? Some subs are really quite good, but most are really quite bad. It's not easy to get guidance, of the kind you would get if you sat with a professor for three or four hours a week for a few years, which is what the humanities students actually do.
Wikipedia is a great reference work, but it tends to not have any of the kinds of connections you're supposed to make in these essays. It has a lot of factual stuff, so questions about Persia will look ok, like in the article. But questions that glue together ideas across areas? Nah. Even if that's in the dataset somewhere, how is the LLM supposed to know that the sort of refined writing of a cross-subject academic is the highest level of the humanities? It doesn't, so it spits out what the average Redditor might glue together from a bit of googling.
a) It was not meant as a "test" by Turing, rather as a thought experiment.
b) It does not require intelligence to pass tests that claim to be it. See:
https://en.wikipedia.org/wiki/Eugene_Goostman
That might still make it the worst candidate in the All Souls exams, because those obviously select for people who are interested in writing essays of this sort.
But I'm also curious whether the LLM could compete given a suitable prompt. If it was told to write an idiosyncratic, opinionated essay, and perhaps given a suitable source material - "you are Harry Potter" but someone less well known but still with a million words of backstory - couldn't it do it? The chat bots we have today are bland because we value blandness. Customers are willing to pay for the inoffensive corporate style that can replace 90% of their employees at writing. Nobody is paying billions of dollars for a Montaigne or a Swift or even a Paul Graham to produce original essays.
It's not surprising as it's very hard to train for or benchmark.
Also should add I don't think anyone serious thinks that long form writing or ideation is what they're for - assuming an LLM would be good at this is a side effect of anthropomorphism / confusion. It doesn't mean an LLM isn't good at summarizing something or changing unstructured data into structured or all of the other "cognitive tasks" that we expect from AI.
This so concisely explains most of the problems and power of these tools. If your goal is to get a reasonably good answer on a reasonably well trod subject you’re going to be very happy with their output. If you push them outside of that they quickly fall into either producing reasonable sounding but incorrect outputs, hallucinations, or failure.