Personally I’m having a blast reading AI generated fiction. As long as the direction is human, and often enough corrected to keep the minor inconsistencies out, the results are pretty good.
For me it’s no different from generating code with Claude, except it’s generating prose. Without human direction your result ends up as garbage, but there’s no need to go and actually write all the prose yourself.
And I guess that just like with code, sometimes you have to hand craft something to make it truly good. But that’s probably not true for 80% of the story/code.
ai fiction really shines in baffling surreal prompts that it tries hard to satisfy. Here's an example:
"let's write a story where donald trump is giving a speech to a crowd as people slowly discover he is secretly a northern red oak tree in a human suit to the shock of fans and reporters! The tips of his fingers become branches as he tries to deny it as wildly impossible meanwhile his human disguise continues to fail"
I think this is similar to AI generated images: it puts a new creative tool in the hands of people who might have had good ideas, but didn't have a mastery of the medium. In that respect, it's cool: if you had a great idea for a sci-fi story but no talent for writing, and if an LLM let you realize your vision, that's neat. It has some negative externalities for the craftsmen, but overall, more creativity is hardly a bad thing.
The real problem is that the most lucrative uses of the tech aren't that. It's generating 10,000 fake books on Amazon on subjects you don't care about. It's cranking out SEO spam, generating monetizable clickbait, etc.
Perhaps what this is pointing out is that a lot of writers of the genre of "fantasy" produce mostly formulaic, trope-laden piles of crap that AI is pretty good at mimicking?
It does suggest that publishers might want to screen new writing with a quick "Did AI write this?" and only publish the ones where it is obvious to humans that AI did not write it.
The story plot was "Meeting a dragon". As both a human and a writer, challenge accepted:
Long ago, there lived a golden dragon whose fractal-like scales gleamed in the glow of the morning in her cave. She was known for her kindness, and many came not with sword or spear, but with humble requests - for you see, it was widely believed that the mystical scales of a dragon would heal illness, cure ailments, and provide fortune.
One such visitor timidly looked up at her great shining body and beseeched, "Oh glorious dragon, might I have a single scale?"
Of course, the dragon replied warmly. She delicately, almost lovingly, with a slight twinge, used a single claw to prise off a single golden scale, leaving a dull patch.
Over the eons, more and more people would come as supplicants. The scales were used for good luck, for warmth, to ward off evil, as the draconic equivalent of a rabbit's foot.
In the end, the poor dragon was stripped bare - the fire from her burning furnace now showed clearly through to her patchwork, sensitive, and naked skin.
When winter came, she huddled in the cold darkness. And still, when a peasant would come asking for a scale - just one, a single scale nothing more, she would not refuse. In her eternal generosity she would carefully break off another. This time it took longer to find one left upon her body, as the humans had stripped her bare like a tree come winter.
Then thus came a knight. "I'm sorry, good sir, but I have no scales left to give," she said pitiably.
"Why, your scale was a choking hazard and wasn’t labeled not for ages under 5! Prepare for a class-action lawsuit and also to be impaled upon a lance."
The End.
I'll pretend I intended it as a parable of the destructive nature of mass tourism or something something Lorax something something truffula trees.
We're at a key milestone in the IP wars; IP was inducted into our civilization to protect the few who had the ability to create.
This idea is well past it's due date. We should move to a liberal IP regime, with copyright strictly reduced to 7-10 years, with all works then entering public domain. Our society will universally thrive with the abundance that will come.
I understand and empathize that a class of vocations today will go away, but so did lamplighters. The roles may become extinct; but we will endure as a people.
Interesting study. I think the use of AI boils down to this: is the product independent of the process and the context? Or is it dependent on it in some way? I think, when it comes to art, the latter is truer than the former, and most use of AI in creative fields is predicated on trying to convince people to engage with art in an extremely shallow way (art strictly as soulless, time filling entertainment).
When an author writes a novel, the novel does not exist in a vacuum. The author's persona and the cultural exchange that emerges around the text also becomes an important part of the phenomenal status of the the work and its cultural recognition. Even when an author remains pseudonymous and makes no appearance, this too is part of the work.
If an author uses AI as a tool and takes care to imbue the output with artistic and personal relevance, it probably will become an art object, but its status may or may not be modulated by the use of AI in the process to the extent that the use does/doesn't affect people's crafting interpretations of the work, or the author's own engagements. Contrarily, AI generated work that has close to no actual processual involvement on the part of the author will almost always have slop status, just because it's hard to imagine an interpretive culture around such work that doesn't at some point break down in the face of the inability to connect the worm with other cultural touchstones or the actual experience of a human being. Maybe it could happen, but if it did, at that point the status of the work is till something different in so far as it would be a marker not of human experience, as literature traditionally has been, but something quite new and different literature-cum-hypermarket(we already had mass market) product.
There is an Asimov story called "Someday" in which a toy computer called a Bard generates random fairy tales and reads them to children.
In the story two children try to hack their Bard, to make it tell more interesting modern stories, by feeding it a new vocabulary of modern words. In the end, it just generates the same old fairy tale plots using the new words it has learned.
I really feel like that story embodies today's AI generated stories. I've tried to get ChatGPT to generate original fairy tales and whatever plot prompt I give it, it spits out what is essentially the same dull story every time.
I always enjoy spotting a good anachronism in a sci-fi story (societies with space travel but still use typewriters), but this is a case of really spot on prediction.
The best fantasy/sci-fi literature involves a lot of world building.
For some, the world building came first and the stories were an offshoot of that.
Tolkien needed a world and stories to bring life to the languages he was inventing.
Raymond E Feist's Midkemia was a massive collaborative effort for a RPG world. He has stated: "I don't write fantasy; I write historical novels about an imaginary place. At least that's how I look at it."
Here are my notes and guesses on the stories in case people here find it interesting. Like some others in the blog post comments I got 6/8 right:
1.) probably human, low on style but a solid twist (CORRECT)
2.) interesting imagery but some continuity issues, maybe AI (INCORRECT)
3.) more a scene than a story, highly confident is AI given style (CORRECT)
4.) style could go either way, maybe human given some successful characterization (INCORRECT)
5.) I like the style but it's probably AI, the metaphors are too dense and very minor continuity errors (CORRECT)
6.) some genuinely funny stuff and good world building, almost certainly human (CORRECT)
7.) probably AI prompted to go for humor, some minor continuity issues (CORRECT)
8.) nicely subverted expectations, probably human (CORRECT)
My personal ranking for scores (again blind to author) was:
So for me the two best stories were human and the two worst were AI. That said, I read a lot of flash fiction, and none of these stories really approached good flash imo. I've also done some of my own experiments, and AI can do much better than what is posted above for flash if given more sophisticated prompting.
I think if you compared the AI stories to works by “top” authors, the results wouldn’t really be as close. No one is confusing a story by Kafka or Conrad with a ChatGPT one.
Because unfortunately, one reason why readers can’t tell the difference between the AI and human authors is because they don’t have much exposure to the greats. The average person reads something like 2 books a year, and they probably aren't reading Nabokov.
I love what he is doing but really hope the voting interface was better. Also I wonder what the results would be if there are AI-assisted stories, but maybe real authors would hate to do that.
I put the basic prompt to 5 medium thinking in the API (because most of the writing gains seems to be tucked away in the reasoning mode and because you can't trust the router for this stuff) and this is what i got. It's not flash fiction, more a short story, but i'm impressed.
I'm very surprised according to results people struggled with identifying [3] and [4] as AI.
IMO both are simply bad and both contain usual telltales in spades (continuity problems, failed or trite metaphors/analogies, semantic failures, overall feeling of 'wtf is even being attempted here').
I'm not so surprised that people struggled with identifying [1] as human - the confounding factor is that this flash story is unpleasantly written, and it's not easy to realize that its failure modes (eg. trying to cram too much in too short a text) are rather human like. And I'm sure the fact that arguably the hardest to digest and rather bad human story opens the poll might somewhat influence the further analyses.
As others in the poll I failed to identify [5] as AI even though in hindsight the telltales are also there. That's because I rather liked it, and as a result it was harder to be vigilant. I also was very undecided on [8]. Finally I scored 6/8, but I wouldn't say it was easy.
Shame that comparing to the previous contest https://mark---lawrence.blogspot.com/2023/09/so-is-ai-writin... is not straightforward. In that one I scored 9/10 while having very easy time (I didn't even finish reading some of them before making up my mind). I also felt completely excused with my only failure, incorrectly identifying as AI the story written in the style of exhaustingly banal fan fiction. But frankly I found almost all the human stories in the previous edition better then the current ones.
In retrospect ChatGPT4 was a terrible writer. ChatGPT5 seems to be an improvement to the admittedly worrying point. Still not impossible to discover though.
However these are my impressions only and it looks maybe I was lucky and I should not generalize it? According to the website people had serious trouble discerning gpt4 writing also 2 years ago. And I'm rather shocked they did. And that they scored some of those banal AI stories positively.
If it's not luck on my part, then maybe discerning AI writing is a skill very different from 'writing' or 'being deeply interested in literature', skills of people who usually frequent this blog?
Maybe this is cope, maybe it's my dislike of modern fantasy, a genre that keeps repeating the same narrative elements to get the same reactions out of the reader, but I found the stories kinda bland, and to me they all seemed like something ChatGPT could have written on a good day. Compare them to Behm-Steinberg's Taylor Swift [0], which has a far more interesting premise, significant character development, and good style (to be fair, story number 6 also has relatively good style). Or to Samatar's The Huntress [1], a poetic piece of flash fiction that is willing to leave much more ambiguity and really shows what's possible in terms of style.
A lot of folks here seem sure that models cannot hold long-form context. In my experience that is not true. The real issues are model choice and how you frame the task. For the past few months I have been generating full-length fiction on topics I care about, and the quality is good enough that friends in blind reads get just as absorbed as with solid human-written work.
Example of a long book I put together in about 20 minutes: docs.google[dot]com/document/d/1mA-q1ugWRa6BaOghUTH17kYdjHpaNPIWgoNKg6OvOko/edit?usp=sharing
Yes, there are rough edges, but the character arcs and the lore stay coherent from start to finish.
Disclosure: I am building an iOS app that lets anyone spin up a book with any cast and starting situation, in a voice close to a favorite author, and then steer the story while reading. On-demand long-form content is already here. It is just not packaged very conveniently yet.
23 comments
[ 3.3 ms ] story [ 47.1 ms ] threadFor me it’s no different from generating code with Claude, except it’s generating prose. Without human direction your result ends up as garbage, but there’s no need to go and actually write all the prose yourself.
And I guess that just like with code, sometimes you have to hand craft something to make it truly good. But that’s probably not true for 80% of the story/code.
"let's write a story where donald trump is giving a speech to a crowd as people slowly discover he is secretly a northern red oak tree in a human suit to the shock of fans and reporters! The tips of his fingers become branches as he tries to deny it as wildly impossible meanwhile his human disguise continues to fail"
After some back and forth, here is what I got:
https://9ol.es/md/trump
No human would write something that crazy...
The real problem is that the most lucrative uses of the tech aren't that. It's generating 10,000 fake books on Amazon on subjects you don't care about. It's cranking out SEO spam, generating monetizable clickbait, etc.
This is neither new nor news. "The Well-Tempered Plot Device" is almost 4 decades old (see: https://news.ansible.uk/plotdev.html).
It does suggest that publishers might want to screen new writing with a quick "Did AI write this?" and only publish the ones where it is obvious to humans that AI did not write it.
Long ago, there lived a golden dragon whose fractal-like scales gleamed in the glow of the morning in her cave. She was known for her kindness, and many came not with sword or spear, but with humble requests - for you see, it was widely believed that the mystical scales of a dragon would heal illness, cure ailments, and provide fortune.
One such visitor timidly looked up at her great shining body and beseeched, "Oh glorious dragon, might I have a single scale?"
Of course, the dragon replied warmly. She delicately, almost lovingly, with a slight twinge, used a single claw to prise off a single golden scale, leaving a dull patch.
Over the eons, more and more people would come as supplicants. The scales were used for good luck, for warmth, to ward off evil, as the draconic equivalent of a rabbit's foot.
In the end, the poor dragon was stripped bare - the fire from her burning furnace now showed clearly through to her patchwork, sensitive, and naked skin.
When winter came, she huddled in the cold darkness. And still, when a peasant would come asking for a scale - just one, a single scale nothing more, she would not refuse. In her eternal generosity she would carefully break off another. This time it took longer to find one left upon her body, as the humans had stripped her bare like a tree come winter.
Then thus came a knight. "I'm sorry, good sir, but I have no scales left to give," she said pitiably.
"Why, your scale was a choking hazard and wasn’t labeled not for ages under 5! Prepare for a class-action lawsuit and also to be impaled upon a lance."
The End.
I'll pretend I intended it as a parable of the destructive nature of mass tourism or something something Lorax something something truffula trees.
This idea is well past it's due date. We should move to a liberal IP regime, with copyright strictly reduced to 7-10 years, with all works then entering public domain. Our society will universally thrive with the abundance that will come.
I understand and empathize that a class of vocations today will go away, but so did lamplighters. The roles may become extinct; but we will endure as a people.
When an author writes a novel, the novel does not exist in a vacuum. The author's persona and the cultural exchange that emerges around the text also becomes an important part of the phenomenal status of the the work and its cultural recognition. Even when an author remains pseudonymous and makes no appearance, this too is part of the work.
If an author uses AI as a tool and takes care to imbue the output with artistic and personal relevance, it probably will become an art object, but its status may or may not be modulated by the use of AI in the process to the extent that the use does/doesn't affect people's crafting interpretations of the work, or the author's own engagements. Contrarily, AI generated work that has close to no actual processual involvement on the part of the author will almost always have slop status, just because it's hard to imagine an interpretive culture around such work that doesn't at some point break down in the face of the inability to connect the worm with other cultural touchstones or the actual experience of a human being. Maybe it could happen, but if it did, at that point the status of the work is till something different in so far as it would be a marker not of human experience, as literature traditionally has been, but something quite new and different literature-cum-hypermarket(we already had mass market) product.
In the story two children try to hack their Bard, to make it tell more interesting modern stories, by feeding it a new vocabulary of modern words. In the end, it just generates the same old fairy tale plots using the new words it has learned.
I really feel like that story embodies today's AI generated stories. I've tried to get ChatGPT to generate original fairy tales and whatever plot prompt I give it, it spits out what is essentially the same dull story every time.
I always enjoy spotting a good anachronism in a sci-fi story (societies with space travel but still use typewriters), but this is a case of really spot on prediction.
For some, the world building came first and the stories were an offshoot of that.
Tolkien needed a world and stories to bring life to the languages he was inventing.
Raymond E Feist's Midkemia was a massive collaborative effort for a RPG world. He has stated: "I don't write fantasy; I write historical novels about an imaginary place. At least that's how I look at it."
This is what you won't see AI doing...yet.
1.) probably human, low on style but a solid twist (CORRECT) 2.) interesting imagery but some continuity issues, maybe AI (INCORRECT) 3.) more a scene than a story, highly confident is AI given style (CORRECT) 4.) style could go either way, maybe human given some successful characterization (INCORRECT) 5.) I like the style but it's probably AI, the metaphors are too dense and very minor continuity errors (CORRECT) 6.) some genuinely funny stuff and good world building, almost certainly human (CORRECT) 7.) probably AI prompted to go for humor, some minor continuity issues (CORRECT) 8.) nicely subverted expectations, probably human (CORRECT)
My personal ranking for scores (again blind to author) was:
6 (human); 8 (human); 4 (AI); 1 (human) and 5 (AI) -- tied; 2 (human); 3 and 7 (AI) -- tied
So for me the two best stories were human and the two worst were AI. That said, I read a lot of flash fiction, and none of these stories really approached good flash imo. I've also done some of my own experiments, and AI can do much better than what is posted above for flash if given more sophisticated prompting.
Because unfortunately, one reason why readers can’t tell the difference between the AI and human authors is because they don’t have much exposure to the greats. The average person reads something like 2 books a year, and they probably aren't reading Nabokov.
https://pastebin.com/huGhbX7u
IMO both are simply bad and both contain usual telltales in spades (continuity problems, failed or trite metaphors/analogies, semantic failures, overall feeling of 'wtf is even being attempted here').
I'm not so surprised that people struggled with identifying [1] as human - the confounding factor is that this flash story is unpleasantly written, and it's not easy to realize that its failure modes (eg. trying to cram too much in too short a text) are rather human like. And I'm sure the fact that arguably the hardest to digest and rather bad human story opens the poll might somewhat influence the further analyses.
As others in the poll I failed to identify [5] as AI even though in hindsight the telltales are also there. That's because I rather liked it, and as a result it was harder to be vigilant. I also was very undecided on [8]. Finally I scored 6/8, but I wouldn't say it was easy.
Shame that comparing to the previous contest https://mark---lawrence.blogspot.com/2023/09/so-is-ai-writin... is not straightforward. In that one I scored 9/10 while having very easy time (I didn't even finish reading some of them before making up my mind). I also felt completely excused with my only failure, incorrectly identifying as AI the story written in the style of exhaustingly banal fan fiction. But frankly I found almost all the human stories in the previous edition better then the current ones.
In retrospect ChatGPT4 was a terrible writer. ChatGPT5 seems to be an improvement to the admittedly worrying point. Still not impossible to discover though.
However these are my impressions only and it looks maybe I was lucky and I should not generalize it? According to the website people had serious trouble discerning gpt4 writing also 2 years ago. And I'm rather shocked they did. And that they scored some of those banal AI stories positively.
If it's not luck on my part, then maybe discerning AI writing is a skill very different from 'writing' or 'being deeply interested in literature', skills of people who usually frequent this blog?
[1] https://tinhouse.com/the-huntress/ [0] https://gulfcoastmag.org/stories/2015-barthelme-prize-winner...
Example of a long book I put together in about 20 minutes: docs.google[dot]com/document/d/1mA-q1ugWRa6BaOghUTH17kYdjHpaNPIWgoNKg6OvOko/edit?usp=sharing Yes, there are rough edges, but the character arcs and the lore stay coherent from start to finish.
Disclosure: I am building an iOS app that lets anyone spin up a book with any cast and starting situation, in a voice close to a favorite author, and then steer the story while reading. On-demand long-form content is already here. It is just not packaged very conveniently yet.