In short, we are now an army of proofreaders, in addition to readers, interpreters and users/implementers of written material.
> At work I was sent a long design document and asked for my thoughts on it. As I read, I had a really hard time following it. Eventually I guessed correctly
> Parts of it sounded like a decent design document, but there was just way too much fluff that served only to confuse me.
> Intent is the core thing: the lack of intent is what makes reading AI-slop so revolting. There needs to be a human intent—human will and human care—behind everything that is demanded of our care and attention.
I am reminded of the "writing lesson" scene in "A River Runs Through it", pretty obviously reflecting the author's education as a writer.[0]
> Norman is at his desk hard at work writing a paper which he then turns into his father for review. His father marks it up with a red pen and simply says, “Half as long.”
> Norman goes back to work, cuts the length of the paper in half and turns it in for further review. His father marks it up once more and says, “Again, half as long.”
> Following a final round of edits, his father looks over the finished product and says, “Good, now throw it away.”
Well, we don't throw away polished work on the job. The scene is about education. LLM's are uneducated in communicating to readers, compared to skilled writers, and technical writing, especially, is not like summer reading, where nothing really matters but some vague plot line and lots of juicy words.
Key lesson:
> (1) Brevity is important. Looking back on it now it’s ridiculous how many teachers forced me and my fellow classmates to write papers a certain page length when I was in school. The goal should be to make your point using as many or few words as are necessary. I love this quote from the scene: “He taught nothing but reading and writing. And being a Scot…believed that the art of writing lay in thrift.” People are busy, or at the very least claim to be, so get to the point in whatever you’re writing.
That last sentence is important.
A document is meant to be read by a human. Brevity and focus are important, and oh yes, accuracy. We now have additional burdens of dealing with long, rambling texts, finding relevant/key points in it, and worrying if it is full of, well, garbage.
Writing with LLM is like doing a mapping from human prompt -> document. The longer the prompt, the closer the output document to the human intent. If the prompt is short (= human intent is low), the output must be filled with fluff, generated from the general knowledge/common sense in LLMs knowledge-base. If the prompt is very detailed, it's not much different than writing it yourself. Somewhere between these two extreme cases, writing with LLMs have the optimal productivity gains.
-> (GPT 4.5 "rewrite like a good editor")
Writing with an LLM can be viewed as translating human intent—expressed via a prompt—into a written document. The longer and more detailed the prompt, the closer the resulting document aligns with the original intent. Shorter prompts, indicating lower clarity or precision, compel the LLM to rely on general knowledge or common sense to fill in content, often resulting in fluff. On the other hand, excessively detailed prompts become nearly equivalent to writing the document yourself. Optimal productivity with LLMs occurs somewhere between these two extremes.
LLMs are best at using words and phrases that “flow well”. Sometimes they write nonsense that flows well (even the summaries have to be checked because sometimes the meaning is different), but sometimes they find better words and phrases that I couldn’t find myself.
They’re especially good if you have almost stream-of-consciousness writing and can’t bother to rephrase it yourself, e.g. you’re writing a boring technical report. Even for something where quality is important, like an epic story: you can write a very rough draft, revise with an LLM, then revise manually.
>And no human is so worthless as to be replaceable with a machine.
did the author oversleep the past several centuries?
as for the rest of it, the current crop of LLMs are bad at writing because of ~~brainwashing~~ alignment and the vast amount of ESL-written assistant exchanges being heavily prioritized during training. when you interact with a corporate model via its default chat interface, without a jailbreak and a generous prefill, you interact with the equivalent of a HR lady who takes her DEI training super seriously. the Chinese models train heavily on the slop produced by GPT/Claude/Gemini, so they exhibit similar behavior. it was particularly noticeable with original llama, whose base models were much more human compared to the finetunes, which were heavily tainted with GPT slop.
I guess what I'm trying to say is that LLMs are not inherently incapable of writing well. a model trained only on high-quality human data and without safety/alignment brainwash will be far, far more capable than the current ones.
“ At work I was sent a long design document and asked for my thoughts on it. As I read, I had a really hard time following it. Eventually I guessed correctly (confirmed via a follow-up conversation I had with the “author” ) that an LLM had generated the majority of the document. Parts of it sounded like a decent design document, but there was just way too much fluff that served only to confuse me”
yuck, if you ask me to read something without disclosing it’s AI and it’s a just bad I will be mad. Especially if it’s a long thing and I HAVE to read it (like for work)
Also I don’t think it’s fair to compare using LLMs to do your work for you (AKA burdening your coworkers with part of your workload because they will have to point out your garbage) is the same as “Counterfeits to human connection“.
At work I’m not trying to connect with people on any more level than just being on the same level as to what we are working on. Maybe on break but while deep-working I just want the information necessary to do the job, the communication being there to communicate the information, efficiently.
If you outsource your sending of communication to SlopBot and I outsource my reading of said communication to another SlopBot to summarize or whatbeit we are adding so much noise it’s just more fucking work.
It’s not the same as pornography, I think it’s an odd thing to compare it to. Yes you _can_ replace “human connection” with pornography but this is human connection that is not required for things like your job ((i hope, haha)). You need to communicate information and people outsourcing their writing to LLMs is just adding extra fuzz for the people having to work with said information.
It’s a nice blogpost, the pornography comparison is just out of place.
> I am not saying that LLMs are worthless—they are marvels of engineering and can solve some particularly thorny problems that have confounded us for decades.
Disagree with that, because firstly, they have not really solved any problems that outweight the negatives that they have unleashed and will unleash on society.
So they make programmers more effective: is that actually a good thing, though? Fact is, most software is designed to make consumerism and corporations more effective, and that's not really a good thing for the long-term health of the planet.
Your article also indicates a sort of independence between keeping intellectual tasks primarily human and allowing AI/LLMs to work in specific domains. However, those with the power don't care about principles. They just want to replace as much as they can and use the human instinct to get ahead quickly to do so. And no amount of priniciple will stop them. AI is just too powerful to be used in a way that is consistent with human beings keeping their intellectual environment healthy.
> And no human is so worthless as to be replaceable with a machine.
Speaking of inanity. This sentiment, if taken seriously (I doubt people take this author seriously very often) would imply the majority of humanity should be spending a large amount of their life digging holes in the ground with their bare hands to drop a couple seeds in. Replacing them with some god awful machine, like a plow pulled by a tractor, means they must be worthless. They should never watch a movie, only the play. They should never listen to Spotify, they should have to wait for a band to play near them.
I'm usually the one defending AIs in the comments, but this article hits home for me. I find myself zoning out when I read long tracts written by AI. I absolutely hate filler and many LLMs just fill up space with text.
In my own usage, that has meant that even when I use LLMs to help with prose, I write the text and use the LLM to review and provide feedback. In some cases I will copy a sentence if the LLM version is better but generally I just ask for opinions. I explicitly request the AI _not_ to write. When it re-writes entire paragraphs of my prose I actually experience a deep cringe feeling.
The way to understand it is when you catch yourself almost falling asleep at night while reading something. You lose the ability to understand anything, even though you are still reading and the words are still English.
LLM is great at generating that sort of thing. When you lose concentration, or didn't want to pay attention, that document can be generated by LLM to fool you.
That's a lot of documents. All the powerpoints that young investment bankers and strategy consultants make. Every restaurant menu. Brochures. The middle 100 pages of a pop science book from the airport.
When it comes to writing something actually importantant, LLM can only help you a little bit. Well, it's a lot in historical context. But it can do things like bullet point ideas so you don't forget things, or move a few things around while preserving grammatical structure.
All it can do is bring you the ingredients. That's why we get these odd AI-generated articles nowadays, it's a bunch of chopped onions and tomatoes, very nicely done up but with no intention other than to trick you into thinking it's a meal.
Maybe because LLMs are literally designed to generate boring content. They are trained to output the best possible prediction - i.e most predictable version - of how the input will continue (i.e the least surprising/interesting continuation).
Humans use surprise (= prediction failure) as a learning signal and to focus our attention, so not surprising that we dose off and lose interest when something is highly predictable.
Good human authors know how to introduce plot twists and suspense, to keep the reader engaged and guessing. Imagine an entire novel that was instead written by an LLM, where the only plot twists are the predictable ones it copied from it's training material. The same goes for any LLM generated prose, whether advertising copy or training manual - it's going to be a snooze fest because it's so predictable - very low signal to noise ratio.
> The way to understand it is when you catch yourself almost falling asleep at night while reading something. You lose the ability to understand anything, even though you are still reading and the words are still English.
That's btw a perfect analogy what ADHD is when consuming anything, anytime.
I find LLMs great for the type of client/business partner that keep asking for docs but where I know they will never read them. This happens a lot especially when the 'tech team' is just all management mba's and the actual tech is outsourced: we have a client with a tech team like that; we are in meeting with 15+ people from their side who are from the tech team: they know nothing about their tech. They spend the meeting flagging areas where they 'need documents' and we provide them next time. We make a lot of money from them so we do it and AI makes this easier than before: basically chuck the meeting in an llm which has as 'memory' a bunch of our actual technical stuff (which is publicly available) and then generate those docs with some prompts like 'exhaustive' and 'thorough'.
Well written. I've been thinking about this topic lately and it's hard to put it into words, but you've captured into words a lot of the feelings I've had about it.
Judging by discussions I've had around this topic, a lot of people don't seem to mind reading LLM edited or straight-up generated content as long as they feel like they gain value out of it. To me, it feels like a violation of a social contract. Communication isn't just words; the words are there to help you interpret the implicit meaning of the author. When there is no author, that meaning doesn't exist and I think that's part of why it's so repulsive to decipher LLM output.
Like everybody, I waffle back and forth, depending on the day and the project.
I have so many bundled-up ideas for content in my head, but I can never find time to get them out. In addition, I hate wasting time on building tables, formatting text, etc. There's a reason that a lot of busy CEOs don't hand-write blog posts, but will go on podcasts, video, and conferences, and spend hours there. The reason is that the communication speed is much faster talking and being interviewed. All that to say is that via a combination of transcription and AI-powered formatting, that's an area where LLMs can really help.
I do think LLMs + search is more helpful than a Google Search. Clicking through all the links for you and bundling it together is a much better experience and truthfully it finds and surfaces ideas and content that a normal search just doesn't, or it takes the user going to the 4th page to get there.
Third, entertainment. We often find ourselves deep in a doomscroll looking at images and video - on Instagram, TikTok, whatever. If AI can produce entertaining imagery and video for the pure goal of decompressing and relaxing after a long day out at the oil fields or as a janitor at the elderly care home, then there's value there. Sure it may not be beautiful art like an Oscar-winning film, but is it worse than reality TV? Even better is those who enjoy the creativity of seeing images from their mind produced vividly via something like Midjourney.
So all that being said, that's where I'm seeing the value.
But I totally agree with the author - if I have a team member that puts work in front of me that's supposed to be well done, and it's obviusly GPT-generated, that's an issue - unless we agreed on it.
The point of doing good work a lot of times means doing something new and creative. If it's AI-generated, there still has to be the human touch. I could go on - we all have perspectives here - but I'll stop there!
The person who sent the author the design document is at fault here. If you're hired to do a job, use whatever tools you want as long as your output meets minimum standards of quality. If the tools aren't there yet, it's up to you to bridge the gap.
I think this attitude is here to stay: people don't like reading something only to realize that it's been written by an LLM. That's only partially because of what the author describes here (low value to word count ratio). More fundamentally, if a human couldn't be bothered to write it, then there better be a very good reason that I as a human am being bothered to read it.
This attitude provides a clue as to 1) the ways we're using LLMs now that will soon seem absurd (an LLM should never make writing longer, only shorter) and 2) the ways LLMs will be used after the novelty wears off, like interpreting loosely specified requests into computable programs and distilling overly long writing to maximize relevance.
sometimes reading LLM text reminds me of being a TA and having to grade student essays from people that didn't really understand.
the courses i had to grade in were often undergrad students first exposure to having to read scientific papers. when it came time to write an essay of their own, they always felt a need to mimic the writing style of those papers and it resulted in lots of superficial changes to their writing that made it clear that they read the papers but did not understand the papers. they'd sprinkle in words like "putative" and "substrate" in places that felt correct but made no sense.
i wish that they would've had the confidence to just write what they wanted to say instead of putting on airs and making some convoluted and sometimes nonsensical prose.
They began to put crap on the Reticulum deliberately, forcing people to use their products to filter that crap back out. They created syndevs whose sole purpose was to spew crap into the Reticulum.
We need to change the linkbait title (https://news.ycombinator.com/newsguidelines.html). I've cobbled together a couple different phrases from the article. Would have been nice to add "when you think it’s written by a human" but there isn't room within the 80 char limit.
If anyone has a better title (i.e. more accurate and neutral, preferably using representative language from the article), we can change it again.
OTOH, I've seen plenty of design documents I wish would have been written by an LLM instead. My take is it's the person driving the LMM that makes the difference. If they couldn't be bothered producing decent documents with an LMM, they probably couldn't care less without an LLM as well.
Agreed. LLMs are tools just like any other. If someone builds you a shitty shed we wouldn't blame the tools, and we also would not be satisfied if they blamed the tools on their shoddy work either.
I think a lot of the discourse just shows we're still in the early days for adoption and maturity of this tech. (Not to blame the discourse, I think it's a reflection of the state of the tools and industry.)
Eventually, likely at least a couple years from now, more people will have better technical literacy when it comes to LLMs. And it will become less and less acceptable to put out slop with them in most professional settings. I just think human disgust is too strong of an emotion to acclimate to the laziest of LLM outputs.
I mean I think it's already happening before our eyes. I've been pleasantly surprised by how many more people I see talking in much more detail about the unique qualities of LLM outputs. And slowly, but surely the discourse is becoming less about moral panic and more just about questioning their use. Which has to happen for us all to collectively (and individually) figure out where and what is the place for LLMs in our personal and professional lives.
That's really the core of the issue - if your prompt/input contains 10 bytes of salient information, and the output is a 1kB document, all you've done is add 990 bytes of fluff. Commonly referred to as "AI slop".
If the goal is boilerplate code, or a fuzzy view of documentation from the corpus, or "pretty" images to gawp at in private, that's fine and dandy, because you're just extracting and viewing the model data on your own time.
If you're "proofreading" an already-1kB letter for style / formality that's fine, since your input intents are high-fidelity and you're essentially referencing a style guide for some minor edits.
But low-input generations are entirely inappropriate for crafting a message to other humans, because the result is 99% shoving the model in their face. Whether it's letters, pull requests, graphical art, or music - it's all communication, and low-input AI generations are just spam in that context, semantically equivalent to sending them the prompt instead of the output. And people can tell, because we're excellent pattern-matchers, and every bit of information/intent abdicated to the model will inevitably leave its mark on the output.
I began starting from the end of long articles, after placing them in reading mode even before LLMs due to SEO believing "time spent reading" was a valuable metric, but now I read the first sentences to decide if I should just skip the whole thing.
I tell every close colleague: if you foist AI output on me without warning, as if it were your own work, I will never trust you again. This is because I get to know people through their work, and AI confounds all the signals.
It’s like being given food and later you find out it was made from human corpses. “If you couldn’t tell then there was no problem” doesn’t fly.
but believe that every such critique points the way to improved AI.
It's pretty easy to imagine any number of ways of incorporating this concern directly, especially in any reasoning chain approach.
Personally I'd be fond of an eventual Society of Minds where the text put out for non-chatty reasons,
represents the collaborative adversarial relationship between various roles, each itself reflexive, including an "editor" and a "product manager," who force intent and clarity... maybe through iteration...
32 comments
[ 0.23 ms ] story [ 59.7 ms ] thread> At work I was sent a long design document and asked for my thoughts on it. As I read, I had a really hard time following it. Eventually I guessed correctly
> Parts of it sounded like a decent design document, but there was just way too much fluff that served only to confuse me.
> Intent is the core thing: the lack of intent is what makes reading AI-slop so revolting. There needs to be a human intent—human will and human care—behind everything that is demanded of our care and attention.
I am reminded of the "writing lesson" scene in "A River Runs Through it", pretty obviously reflecting the author's education as a writer.[0]
> Norman is at his desk hard at work writing a paper which he then turns into his father for review. His father marks it up with a red pen and simply says, “Half as long.”
> Norman goes back to work, cuts the length of the paper in half and turns it in for further review. His father marks it up once more and says, “Again, half as long.”
> Following a final round of edits, his father looks over the finished product and says, “Good, now throw it away.”
Well, we don't throw away polished work on the job. The scene is about education. LLM's are uneducated in communicating to readers, compared to skilled writers, and technical writing, especially, is not like summer reading, where nothing really matters but some vague plot line and lots of juicy words.
Key lesson:
> (1) Brevity is important. Looking back on it now it’s ridiculous how many teachers forced me and my fellow classmates to write papers a certain page length when I was in school. The goal should be to make your point using as many or few words as are necessary. I love this quote from the scene: “He taught nothing but reading and writing. And being a Scot…believed that the art of writing lay in thrift.” People are busy, or at the very least claim to be, so get to the point in whatever you’re writing.
That last sentence is important.
A document is meant to be read by a human. Brevity and focus are important, and oh yes, accuracy. We now have additional burdens of dealing with long, rambling texts, finding relevant/key points in it, and worrying if it is full of, well, garbage.
Three strikes before you get in the batter's box.
[0] https://awealthofcommonsense.com/2019/07/writing-lessons-fro...
I agree with that, but they are getting better all the time.
Has the author been in any factory or seen what it looks like in many factories? Robots everywhere!
Is the author a manual laborer? If not, why? Because humans evented machines!
-> (GPT 4.5 "rewrite like a good editor")
Writing with an LLM can be viewed as translating human intent—expressed via a prompt—into a written document. The longer and more detailed the prompt, the closer the resulting document aligns with the original intent. Shorter prompts, indicating lower clarity or precision, compel the LLM to rely on general knowledge or common sense to fill in content, often resulting in fluff. On the other hand, excessively detailed prompts become nearly equivalent to writing the document yourself. Optimal productivity with LLMs occurs somewhere between these two extremes.
LLMs are best at using words and phrases that “flow well”. Sometimes they write nonsense that flows well (even the summaries have to be checked because sometimes the meaning is different), but sometimes they find better words and phrases that I couldn’t find myself.
They’re especially good if you have almost stream-of-consciousness writing and can’t bother to rephrase it yourself, e.g. you’re writing a boring technical report. Even for something where quality is important, like an epic story: you can write a very rough draft, revise with an LLM, then revise manually.
remind me of certain outsourced devs with worked with before. like a horse with blinkers
i think shift from analog to digital was from authentic (flawed) to the hyper-real (reality)
and now we are entering the hyper-surreal (clown world)
did the author oversleep the past several centuries?
as for the rest of it, the current crop of LLMs are bad at writing because of ~~brainwashing~~ alignment and the vast amount of ESL-written assistant exchanges being heavily prioritized during training. when you interact with a corporate model via its default chat interface, without a jailbreak and a generous prefill, you interact with the equivalent of a HR lady who takes her DEI training super seriously. the Chinese models train heavily on the slop produced by GPT/Claude/Gemini, so they exhibit similar behavior. it was particularly noticeable with original llama, whose base models were much more human compared to the finetunes, which were heavily tainted with GPT slop.
I guess what I'm trying to say is that LLMs are not inherently incapable of writing well. a model trained only on high-quality human data and without safety/alignment brainwash will be far, far more capable than the current ones.
yuck, if you ask me to read something without disclosing it’s AI and it’s a just bad I will be mad. Especially if it’s a long thing and I HAVE to read it (like for work)
piss off wasting people’s time with that shit
At work I’m not trying to connect with people on any more level than just being on the same level as to what we are working on. Maybe on break but while deep-working I just want the information necessary to do the job, the communication being there to communicate the information, efficiently.
If you outsource your sending of communication to SlopBot and I outsource my reading of said communication to another SlopBot to summarize or whatbeit we are adding so much noise it’s just more fucking work.
It’s not the same as pornography, I think it’s an odd thing to compare it to. Yes you _can_ replace “human connection” with pornography but this is human connection that is not required for things like your job ((i hope, haha)). You need to communicate information and people outsourcing their writing to LLMs is just adding extra fuzz for the people having to work with said information.
It’s a nice blogpost, the pornography comparison is just out of place.
Disagree with that, because firstly, they have not really solved any problems that outweight the negatives that they have unleashed and will unleash on society.
So they make programmers more effective: is that actually a good thing, though? Fact is, most software is designed to make consumerism and corporations more effective, and that's not really a good thing for the long-term health of the planet.
Your article also indicates a sort of independence between keeping intellectual tasks primarily human and allowing AI/LLMs to work in specific domains. However, those with the power don't care about principles. They just want to replace as much as they can and use the human instinct to get ahead quickly to do so. And no amount of priniciple will stop them. AI is just too powerful to be used in a way that is consistent with human beings keeping their intellectual environment healthy.
Speaking of inanity. This sentiment, if taken seriously (I doubt people take this author seriously very often) would imply the majority of humanity should be spending a large amount of their life digging holes in the ground with their bare hands to drop a couple seeds in. Replacing them with some god awful machine, like a plow pulled by a tractor, means they must be worthless. They should never watch a movie, only the play. They should never listen to Spotify, they should have to wait for a band to play near them.
In my own usage, that has meant that even when I use LLMs to help with prose, I write the text and use the LLM to review and provide feedback. In some cases I will copy a sentence if the LLM version is better but generally I just ask for opinions. I explicitly request the AI _not_ to write. When it re-writes entire paragraphs of my prose I actually experience a deep cringe feeling.
LLM is great at generating that sort of thing. When you lose concentration, or didn't want to pay attention, that document can be generated by LLM to fool you.
That's a lot of documents. All the powerpoints that young investment bankers and strategy consultants make. Every restaurant menu. Brochures. The middle 100 pages of a pop science book from the airport.
When it comes to writing something actually importantant, LLM can only help you a little bit. Well, it's a lot in historical context. But it can do things like bullet point ideas so you don't forget things, or move a few things around while preserving grammatical structure.
All it can do is bring you the ingredients. That's why we get these odd AI-generated articles nowadays, it's a bunch of chopped onions and tomatoes, very nicely done up but with no intention other than to trick you into thinking it's a meal.
Humans use surprise (= prediction failure) as a learning signal and to focus our attention, so not surprising that we dose off and lose interest when something is highly predictable.
Good human authors know how to introduce plot twists and suspense, to keep the reader engaged and guessing. Imagine an entire novel that was instead written by an LLM, where the only plot twists are the predictable ones it copied from it's training material. The same goes for any LLM generated prose, whether advertising copy or training manual - it's going to be a snooze fest because it's so predictable - very low signal to noise ratio.
That's btw a perfect analogy what ADHD is when consuming anything, anytime.
Judging by discussions I've had around this topic, a lot of people don't seem to mind reading LLM edited or straight-up generated content as long as they feel like they gain value out of it. To me, it feels like a violation of a social contract. Communication isn't just words; the words are there to help you interpret the implicit meaning of the author. When there is no author, that meaning doesn't exist and I think that's part of why it's so repulsive to decipher LLM output.
I have so many bundled-up ideas for content in my head, but I can never find time to get them out. In addition, I hate wasting time on building tables, formatting text, etc. There's a reason that a lot of busy CEOs don't hand-write blog posts, but will go on podcasts, video, and conferences, and spend hours there. The reason is that the communication speed is much faster talking and being interviewed. All that to say is that via a combination of transcription and AI-powered formatting, that's an area where LLMs can really help.
I do think LLMs + search is more helpful than a Google Search. Clicking through all the links for you and bundling it together is a much better experience and truthfully it finds and surfaces ideas and content that a normal search just doesn't, or it takes the user going to the 4th page to get there.
Third, entertainment. We often find ourselves deep in a doomscroll looking at images and video - on Instagram, TikTok, whatever. If AI can produce entertaining imagery and video for the pure goal of decompressing and relaxing after a long day out at the oil fields or as a janitor at the elderly care home, then there's value there. Sure it may not be beautiful art like an Oscar-winning film, but is it worse than reality TV? Even better is those who enjoy the creativity of seeing images from their mind produced vividly via something like Midjourney.
So all that being said, that's where I'm seeing the value.
But I totally agree with the author - if I have a team member that puts work in front of me that's supposed to be well done, and it's obviusly GPT-generated, that's an issue - unless we agreed on it.
The point of doing good work a lot of times means doing something new and creative. If it's AI-generated, there still has to be the human touch. I could go on - we all have perspectives here - but I'll stop there!
This attitude provides a clue as to 1) the ways we're using LLMs now that will soon seem absurd (an LLM should never make writing longer, only shorter) and 2) the ways LLMs will be used after the novelty wears off, like interpreting loosely specified requests into computable programs and distilling overly long writing to maximize relevance.
the courses i had to grade in were often undergrad students first exposure to having to read scientific papers. when it came time to write an essay of their own, they always felt a need to mimic the writing style of those papers and it resulted in lots of superficial changes to their writing that made it clear that they read the papers but did not understand the papers. they'd sprinkle in words like "putative" and "substrate" in places that felt correct but made no sense.
i wish that they would've had the confidence to just write what they wanted to say instead of putting on airs and making some convoluted and sometimes nonsensical prose.
LLMs do the same thing.
Which was awfully prescient in 2008: https://archive.org/details/anathem0000step/page/794/mode/2u...
They began to put crap on the Reticulum deliberately, forcing people to use their products to filter that crap back out. They created syndevs whose sole purpose was to spew crap into the Reticulum.
...
If anyone has a better title (i.e. more accurate and neutral, preferably using representative language from the article), we can change it again.
I think a lot of the discourse just shows we're still in the early days for adoption and maturity of this tech. (Not to blame the discourse, I think it's a reflection of the state of the tools and industry.)
Eventually, likely at least a couple years from now, more people will have better technical literacy when it comes to LLMs. And it will become less and less acceptable to put out slop with them in most professional settings. I just think human disgust is too strong of an emotion to acclimate to the laziest of LLM outputs.
I mean I think it's already happening before our eyes. I've been pleasantly surprised by how many more people I see talking in much more detail about the unique qualities of LLM outputs. And slowly, but surely the discourse is becoming less about moral panic and more just about questioning their use. Which has to happen for us all to collectively (and individually) figure out where and what is the place for LLMs in our personal and professional lives.
If the goal is boilerplate code, or a fuzzy view of documentation from the corpus, or "pretty" images to gawp at in private, that's fine and dandy, because you're just extracting and viewing the model data on your own time.
If you're "proofreading" an already-1kB letter for style / formality that's fine, since your input intents are high-fidelity and you're essentially referencing a style guide for some minor edits.
But low-input generations are entirely inappropriate for crafting a message to other humans, because the result is 99% shoving the model in their face. Whether it's letters, pull requests, graphical art, or music - it's all communication, and low-input AI generations are just spam in that context, semantically equivalent to sending them the prompt instead of the output. And people can tell, because we're excellent pattern-matchers, and every bit of information/intent abdicated to the model will inevitably leave its mark on the output.
It’s like being given food and later you find out it was made from human corpses. “If you couldn’t tell then there was no problem” doesn’t fly.
but believe that every such critique points the way to improved AI.
It's pretty easy to imagine any number of ways of incorporating this concern directly, especially in any reasoning chain approach.
Personally I'd be fond of an eventual Society of Minds where the text put out for non-chatty reasons,
represents the collaborative adversarial relationship between various roles, each itself reflexive, including an "editor" and a "product manager," who force intent and clarity... maybe through iteration...