i noticed something similar yet different. because, sadly, most of what i read today is llm output (the chain of thoughts, or the summary at the end of a big chunk or work), i tend to "adopt" this style in my writing, hence, writing as an llm without using one. humans are an amazing creatures at copying stuff and adapting to our environment
A billion humans use AI daily, they write about 0.5T tok and read about 2T tok (estimated). In a week they amount to more than the whole training set of GPT-4. Human language is being shaped back by AI language.
> But if I want to communicate with humans, my humanity has to shine (or at least ooze) out of my prose.
Yeah. I want to be able to touch a human being (or at least have one touch me) through the words. LLMs poison that, even if a human prompted the LLM.
Translation... I think I object to translation less than Martin does. There's a human back there, and they're writing all the words, and then an LLM is translating them. The LLM may not translate them perfectly, but they aren't going to introduce "load-bearing" when it wasn't there.
The bit about using LLM when you're writing for a bureaucratic process rather than a human was interesting, at least to me. I hadn't thought of it that way before, but I think it makes sense.
> As more people delegate writing to LLMs they will also delegate thinking and comprehension to LLMs. I don't want to do that, where's the fun of writing then?
I enjoy cooking from scratch, and I really don't like frozen foods, microwave/TV dinners, or even DoorDashing
but we're past complaining/arguing/shaming folks who don't cook every meal from scratch
you know what, I don't prefer eating it, but if my next door neighbor loves microwaving frozen $3 burritos, then go ahead
to some people, food is art, creativity, life
to others, food is fuel
Something like 90% of food in all US restaurants is from Sysco, US Foods, or similar.
Some of them purchase ingredients from those companies, and then make the food using those ingredients.
A great deal of restaurants, though, simply purchase pre-made frozen foods from those distributors, re-heat, and sell it to you at a premium without disclosing while you think it was freshly made. Mac & cheese, stuffed peppers, lasagna, burritos, most 'bar snacks' you can think of, etc.
The analogy here is that a lot of consumers recognize those restaurants that are simply reselling pre-made food without much value-add, and choose not to eat there.
yeah i love this comparison- that's why i avoid places that are clearly sysco wrappers (or massive chains that are just wrappers around their own supply chain)
basically any food establishment that has "supply chain operation" is slop to me; I guess it's similar to my distaste in reading AI slop too
I don't fault ppl for going (and enjoying) those too- Chili's can be enjoyable after a few beers
yes yes definitely, restaurants should not be microwaving food for paying customers (ok if some components are) but like... not serve a completely microwaved dish
happened to me once lol, will never forget that place ever again (blech)
This analogy looks quite good. Heavily processed food is not heathy for your body and heavy use of AI and other dopamine shortcuts like social media are not healthy for your brain. You have to approach both with causion.
I think the idea is if you are going to share LLM writing then why not just share the prompt which is shorter and has the message you want to convey anyway.
That is not to say that I don't think there is a place for LLM assisted writing. Writing is an exercise for the author and translates to oral argumentation so I don't think that fades away. But LLMs could do some "audience normalization". What if I could make a blogpost that is dynamic based on experience of the reader? There are lots of opportunities, but versus other fields where optimizing for laziness can be beneficial I don't think lazy writing currently has much value add.
Largely speaking eating is an activity that only involves yourself. Whereas, writing and thinking are largely activities to communicate ideas between yourself and others. Basically it matters more how this is done.
I’m fine with my neighbour eating TV dinners, but if they invite me for a meal I expect to not be served a TV dinner, or to at a minimum be informed beforehand that that’s what going to be served.
Same thing with LLM-assisted writing. It’s one thing to do it for yourself, quite another to share it with other folks without warning.
I agree. I honestly think convenient food is perfectly fine for people most of the time. Recently I have been playing with "eternal stews" which I guess are logically the opposite of fast foods, but at the same time are focused on availability and convenience over intentionality and mastery (at least the way I make them).
Most food is literally just fuel. I think that is fine. And at some level, eating 3$ burritos for a quick lunch on the daily really makes you appreciate the depths of a home made risotto even more. :]
To stick with that analogy, you also wouldn't serve a tv dinner to a guest unless you were somehow incapacitated. You certainly wouldn't do it if you were running a restaurant.
Add to that the fact that AI is significantly worse for you than TV dinners, and has an outsized harm to humanity, and the conclusion is plain:
There's a fine line between using an llm to write a personal blog vs using it to help you come up with an account plan on why you want to sell to a company that will be used internally.
personally, i'm for LLM and AI usage in writing if it helps the writer communicate ideas, their stories, and triumphs in ways they may never have been able to before. I just had one of my posts removed from the front page here which was disheartening because it feels as though what was judged was prose rather than the technical insight in to a niche problem, which seems counterintuitive considering i've always thought of Hacker News being an inclusive place for the technical and engineering skills rather than their level of the english language.
I find a lot of LLM writing exhausting to read. It is like I have to exert extra energy to attenuate my BS detector to cut through what is actually being said, and why.
This is an interesting corollary to a thing I'm seeing where "devs who thought their job was writing code" are in like existential danger.
Technical skills aren't worth shit and never were, except as far as they enable you to engage with other people in ways you and they find valuable. This used to be really far! It made it easy to believe that the technical skills on their own were what was valuable.
But the engagement was always the valuable thing, generating technical insight was a necessary prerequisite for it. Now anyone with a $20 claude sub can generate technical insight into niche problems, so what's left is everything else. Everything of value.
> This is an interesting corollary to a thing I'm seeing where "devs who thought their job was writing code" are in like existential danger.
I think you're misunderstanding that anxiety. I think this is what's going on:
1. For a lot of people, using technical skills like coding is the part of the job they enjoyed.
2. Closely related to the above point: for a lot of people, technical skills was the part they were good at.
> Technical skills aren't worth shit and never were, except as far as they enable you to engage with other people in ways you and they find valuable. This used to be really far! It made it easy to believe that the technical skills on their own were what was valuable.
> But the engagement was always the valuable thing, generating technical insight was a necessary prerequisite for it. Now anyone with a $20 claude sub can generate technical insight into niche problems, so what's left is everything else. Everything of value.
3. More broadly: if "they" don't find the "technical skills" you provided valuable anymore, why do they need to keep you around? Why not cut your 10-person team down to 3 people (the manager, architect, and technical lead) and fire you and the other 6? Not everyone can re-invent themselves as a product person, even if they had the skills. It's a game of musical chairs.
> I just had one of my posts removed from the front page here which was disheartening because it feels as though what was judged was prose rather than the technical insight in to a niche problem, which seems counterintuitive considering i've always thought of Hacker News being an inclusive place for the technical and engineering skills rather than their level of the english language.
The problem with LLM voice is that it sends the signal that it's a likely waste of effort to try to figure out if there's any "technical insight" there, because more often than not, there isn't.
Maybe it's not fair that you're paying the price for others using LLMs to abuse their readers, but it's the reality that you have to deal with. It's also not fair to expect readers to wade through all that abusive LLM prose in order to find your (assumed) non-abusive post.
Once LLM writing becomes indistinguishable from human writing, it will be really sad. There will just be a collapse of human engagement, and you might as well just talk to your machine because then at least you know what you're getting and aren't being deceived.
in some ways I agree, however I think that speaks more about the reader themselves. Just like when communicating with someone who's first language isn't english, I don't disregard the meaning and intent behind their words simply because the way they communicate isn't formally correct. Regarding my post that was removed, it was lovely to see the comments of people that had enjoyed the technical write up. Me personally, i'll always value an attempt at engaging and communicating than none at all.
Good lord, someone not being good at english does NOT mean that they put not effort or care into their attempted output. It is RACIST to make that association and thats why its an obvious faux pas.
An LLM prose article ABSOLUTELY can mean that. To attempt to conflate the two things is maliciously misleading.
Assuming the author had to input good points for the LLM to expand it into a full length post, just give me the points. Give both formats if one must. Translate, spellcheck, fix mistakes with an LLM. Don’t inflate 3 bullet points to a 10 minute read.
I can’t distinguish between well meaning slop and blogspam slop. I might be able to if I’m well informed on the topic and can point out inaccuracies. Otherwise I just don’t know what is accurate and what is filler or outright wrong. LLM-isms are very distracting and exhausting to sort through. It just doesn’t make for a good reading experience.
Yes, this. I mean, if you're writing a novel meant to be read, maybe not. But if you're just trying to communicate 3 ideas, write them out in a way I can quickly tell if it's worth reading more.
There are few things (relatively speaking, I mean online activities, etc) more annoying than seeing a click-baity headline, and then having to read 17 paragraphs to actually get to the point that I realize I already knew and understood it, and it's just slop padding.
Imagine you cooked something great, a deliciously tender cake or something. I ask you if I can try some, you say sure, cut off a little piece for me, then you drop it and tell I can have it on the floor. That is what AI is doing for writing. Why should I care how the cake would taste, if I don't like eating on the floor?
i don't really agree with that view, but i respect your opinion. In my view, i see it more like someone who's never baked a cake before, offering you a slice, and whether or not it's good may be up to you to decide.
AI doesn't help the writer communicate ideas. It helps them obfuscate those ideas behind bloated language where even the writer can't discern if it's good or not.
It's the modern equivalent of going to a thesaurus and replacing all your natural words with five-dollar ones.
If you're not confident in your English then just write your brilliant ideas in broken English. Or at most do a direct translation. But don't fluff it with AI's nonsense.
I just can't seem to stay interested and read LLM text. It just seems to lack going somewhere, its information density is often really low and its full of odd isms that I don't find very interesting to read. I don't have a good radar for detecting the various LLM particular speech patterns but everytime I have an issue with an article there is always people talking about how its clearly AI written.
I find the same thing with written results out of AI "research", it is a slog to read through it and most of it can be skipped even if it can sometimes be useful its an unenjoyable read.
I can imagine that one of the aspects of being social creatures is that we are drawn to the entropy/surprise factor of natural language produced by a brain that isn't our own. Deriving a shared understanding without being able to precisely model the language output of someone else is like, the fundamental substance of conversation.
Maybe that's the key ingredient that's missing from LLM-generated text. Even when there is actual content there, there's this lack of surprise / uncertainty in its delivery that we require to stay engaged. We may not know what will be communicated, but knowing exactly how it will be communicated just kills our innate interest.
This presumes an LLM has something to say. The reason the text doesnt make much sense, is because it doesnt. It isnt "on topic" because it has no topic to be on. Sentences go on, are joined irrelevantly, and so on because theyre sampled from a distribution of "possibly relevant sentences". They aren't being addressed to a reader (whose mind is being modelled), on a topic the writer wishes to express.
Communication expresses an understanding. LLMs arent communicating.
LLMs generate code well enough. Code has no natural language semantics. Programming's condition of satisfaction is that the machine executes correctly. The condition of satisfaction of communication is that the communication expresses what there is to be communicated.
In the limit where an LLM is trained to repoduce an exmplar of communication, it seems as if it does so. In that limit, LLMs can reporduce anything.
What I find they do badly in code is building abstraction and breaking the problem down in a way a human does. They don't seem to really build simplifying components unless you expressly tell them too, they are more than happy to just nest 10 deep or embed a 20 line called back function and use lots of shortened names. I find I can improve the code just by organising better and choosing some abstractions that make sense to the humans and then ironically you save tokens too because the AI will reuse it, but it wont create most of it itself. Its bad for domain work unless you are explicit about it.
I know what you mean. The latest models are _better_, but there's still something off about the prose they generate. Obvious tells are mostly gone now, but some internal spider-sense still makes me narrow my eyes.
One of my favourite rules: perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. This is invaluable to writing. So much text from an LLM feels superfluous, like I could've said the same wall of text with just a few sentences. Writing is tightly coupled with thinking: refining thoughts, arguments, ideas, discarding them, mulling over them. LLM text always raises this nagging question in my mind: am I reading what you think, or what your agent parroted back and you accepted?
There is no way that solely talking to an agent induces the same depth or amount of thinking than when you are actually writing for yourself. It just takes too much discipline to not offload your thinking.
And the discipline gets attacked even more when all you read is LLM text. For example in code reviews there is no noticeable value in answering LLM generated review comments. I remember taking great care answering human-written reviews comments with much thought to either further build on their suggestions or to forward knowledge about the system when I had to defend a particular change.
Now, some bot will read my carefully written response and forgets about it. There is no value anymore.
No wonder that our pull request comments feel like the dead internet theory right now.
I am absolutely pro AI assisted coding. But communication between humans HAS to stay human. I am currently experiencing the opposite and it’s bad.
I find that it often isn't just too low, but also too high at the same time. It has a lot of information that is irrelevant, burying far too little relevant information for the work needed to extract it. I wonder if this is a side effect of attention mechanism, where the AI's ability to track information in a document is far less hampered by other information, leading to it organizing information to be both dense per word but any specific concept to be sparsely spread throughout the writing. I also wonder if this is why the order feels off, because the AI doesn't need for the information to be well ordered to the same extent that a human does, so the order of storing information is often overlooked when storing it into a document (and even information that needs to be ordered might have the ordering information separate from how it is ordered in the document).
> In time it may be that LLMs will be so good that we can't tell the difference.
I have to believe that we are within a couple of years if not months of this. This problem is too well-known to not be a major focus of the frontier labs' efforts.
My own feeling is that this signal will perhaps never fully go away. Not in months; most likely not in a couple of years.
Not least because it’s getting worse rather than better. But also because as AI gets better at writing like humans, humans will change how they communicate to avoid being confused with it.
Which will get a lot easier once human writing is nearly extinct. Now that could happen in two years.
I wrote down a few of my thoughts a few weeks ago at https://notes.pault.ag/why-write/ -- I find myself scanning text to see if it's LLM before I actually engage with the content.
It is getting harder to do by the day, and has slowly moved from a mental exercise (look for specific markers) into a "gut feel" based on how the pieces fit together.
I’ve started embracing my grammar flaws and typos. Emails and chats may get a proof, but only from me or another human, then I send it. I’ve deleted Grammarly because I think the flaws make the comms more authentic in the age of AI.
And of course I’ve dropped my use of the em-dash after being accused of being AI myself.
Ever since I learned it's as simple as option-hyphen on a Mac, I've loved the em-dash. Still don't have the hang of typing it on a Chromebook or Windows PC though.
> I'm wary of them, partly due skeptism about their accuracy, but also because it rather misses the point.
Here the reader wonders whether "to" was omitted intentionally to make a point?
> But I also would prefer rather a genuine human's clunky prose to the smooth but soulless LLM.
Amen. This reminds me of a bit from Kitchen Confidential about how Anthony Bourdain would prefer a simple and plain home cooked meal over an ornate restaurant meal any day of the week. I read because I want to hear what a human has to say. It's the errors and malapropisms and style that give a unique and human voice.
I'll repost a comment I made a while back describing the process I use, which I think balances AI in keeping your voice.
If you care about your voice, don't let LLMs write your words. But that doesn't mean you can't use AI to think, critique and draft lots of words for you. It depends on what purpose you're writing it for. If you're writing an impersonal document, like a design document, briefing, etc then who cares. In many cases (scientific papers, legal documents) you already have to write them in a voice that is not your own. Go ahead and write these in AI. But if you're trying to say something more personal then the words should be your own, AI will always try to 'smooth' out your voice, and if you care about it, you gotta write it yourself.
Now, how do you use AI effectively and still retain your voice? Here's one technique that works well: start with a voice memo, just record yourself maybe during a walk, and talk about a subject you want, free form, skip around jump sentences, just get it all out of your brain. Then open up a chat, add the recording or transcript, clearly state your intent in one sentence and ask the AI to consider your thoughts, your intent and ask clarifying questions. Like, what does the AI not understand about how your thoughts support the clearly stated intent of what you want to say. That'll produce a first draft, which will be bad. Then tell the AI all the things that don't make sense to you, that you don't like, just comment on the whole doc, get a second draft. Ask the AI if it has more questions for you, you can use live chat to make this conversation go smoother as well, when the AI is asking you questions, you can talk freely by voice. Repeat this one or two more times, and a much finer draft will take shape that is closer to what you want to say. During this drafting state, the AI will always try to smooth or average out your ideas, so it is important to keep pointing out all the ways in which it is wrong.
This process will help you with all the thinking involved being more up-front. Once you're read and critiqued several drafts, all your ideas will be much more clear. Then, sit down and write your own words from scratch, they will come much easier after all your thoughts have been exercised during the drafting process.
As someone who is aggressively pro AI, I've also noticed how awful everything is to read lately. It's always so hard to put my finger on the exact thing, but it's a mixture of everything sounding identical and everything being a rambling unconcise mess. I find I routinely start reading interesting posts and just .. stop part way, abandoning it. A site I read stories on has been flooded with AI generated content, all of it terrible, and it just makes me retreat to things created before 2023 as a way of filtering it all out.
Gentle reminder that most AI token spend is used for TPS report type writing not Cutting edge fiction writing. Creative writing, or general human-to-human communication was never really the target. It's not surprising the first majority use of this was code.
Endless headlines like, "aArON SORkIn SAYs hE WiLL Never UsE ai to WRite" doesn't change that base fact. No one asked him to, other than clickbait journalists.
If you find this hard to believe, just think about relative market sizes.
I suspect it’s partially an average / regression to mean.
If you built a model only based on long form journalism and The New Yorker voice, including both the template structure of the article and the grammar and syntax, the model would replicate the patterns it is built upon. You could do the same for high vocabulary rap.
When you stuff all of English or all of human thought into it with somewhat equal weight, including very technical writing the sum is a lack of style and elegance and personality.
The second problem is that writing often doesn’t happen linearly. You write bookends and fill in, edit and revise. You sort of know where you’re going and can then make decisions how to get there. Language models may benefit from being redesigned to perform passes. Select a template of some sort, fill in the mad lib, then run a detail pass, then run a persuasion / emotion pass, then run a style pass. Current model writing very much sounds like I would expect autocomplete to.
Isn't this easy for frontier 'labs' to fix? If the user requests lots of writing and prose, then use an auxiliary model which has been trained on 'award-winning' writing instead of all the crap on the internet?
Award-winning writing has all kinds of different styles. The average of them will probably get you some boring median soup once again, though maybe a bit less bland than the LLMs trained on oceans of LinkedIn and Reddit slop.
It’s heavily RL’d and it’s gotten worse because they’re using LLM judges, that’s why it’s gotten worse. The last good writing model was GPT 4.5.
I suspect they’re leaning heavily on some kind of auto verifier, and that’s why the writing sucks, because good writing is so subjective.
And they’re probably nerfing what it can get from training as they’ve lost lawsuits for scraping copyrighted work, so they’ll purposefully avoid anything that references a specific writers style, etc.
The "regression to the mean" or "it's just the training data" hypothesis doesn't work. Pretrained models do not have these weird writing styles and sound much more human. It's only after instruction tuning -- SFT, DPO, etc. -- that LLMs develop weird writing styles.
That is, the original training on massive text corpora seems to produce an average of human writing, but that average looks pretty average. It doesn't feel like LLM slop. But the instruction tuning on specific tasks (answer this prompt, summarize this text, do this math problem), which turns text predictors into chatbots, seems to create strange style features as a side effect. And because that tuning step depends on company-specific training data (rather than just a crawl of the entire Internet), each company's LLM has a different style.
It's not that difficult to measure this. Linguists have had measures of style variation for a long time, though NLP and AI researchers seem to have forgotten. We've generated parallel human and AI corpora to measure it, and can show specifically which writing features change: https://arxiv.org/abs/2410.16107
Currently we're working on tweaking the instruction tuning process to see if we can pin down which steps and what types of tasks tend to produce the weird style features.
A model trained to minimize statistical prediction errors is going to sound ... predictable (aka boring). A (good) human author tries to keep their writing interesting, while an LLM strives for predictable instead.
It's not just predicting content but also the form/shape of the response, highly shaped by RLHF preferences, which seems to encourage longer form responses which need padding with filler to generate the "preferred" amount of verbiage.
It seems most of these models have been trained to interpret questions as wanting a long-form authoritative/comprehensive-sounding response (even when midwit filler is all it has to offer), rather than initiating a short-turn conversation which would better allow you to steer the conversation. Apparently Muse is much better in this regard, but so far I've resisted signing up for any Meta/FaceBook service, and not inclined to break this winning streak!
Don't these sort of conflict with being "aggressively pro AI"?
> how awful everything is to read lately
> flooded with AI generated content, all of it terrible
> everything sounding identical and everything being a rambling unconcise mess.
I can see the benefits of AI and enjoy using the technology - but, like many have said, it's wreaking havoc on the "content" that's out there nowadays. Gotta take some bad with the good, I suppose.
I can like the concept of the engine, and the leverage it provides, but I hate traffic and get anxious about having an accident on the freeway. And the people behind the industrial revolution have destroyed the climate.
I love computers and the internet, and the internet has ruined a lot of people and things around me.
It's possible to be interested in a new technology and enjoy its leverage while still being extremely apprehensive about how it intersects with game theory and politics. The race to the bottom of the barrel keeps coming for the jugular of every cool new thing.
I’ve been starting to think that a lot of the variance in how people see the world can be explained by whether their attention is drawn to first-order localized effects, or nth-order distributed effects. This is not the same thing as optimism vs. pessimism, but it sure is correlated!
Well, it's quite simple if you think about it. Intelligence is all about awareness of the imminence of death and the finite amount of time given to any of us. So, we humans intrinsically evolved to make the most of it. The majority of the tools we create have the ultimate purpose - to save time. Even money is nothing but a way to cut some time. It's not really about time but experiences - we want to have as many and as rich as we can get, and we innately try to cut time somewhere else, so we can use what we saved for better experiences.
Therefore, inventing cars was an easily predictable event. There's not a single hypothetical timeline or dimension where humans don't end up with cars - to transport things and to travel faster. Similarly, it was always simply inevitable - we would have invented something like LLMs, sooner or later.
Cars required significant sacrifices - they are absolutely deadly machines, they kill thousands of people every single day, kill wildlife, endager our own existence, and we're kind of okay with that. I can absolutely assure you, if someone tomorrow invents a gravitational jump travel that allows moving things between continents in minutes, even with a success rate as low as 10%, we would still be using it.
So, yes. We can't un-invent cars and AI, the only way to move forward is to adjust to this new reality and extract the best possible experiences, preferably by not robbing other people of their - i.e., enjoy life while it lasts, it still remains finite.
The part I find grating is the attempts to punch everything up. ("It's not X. It's Y"). Like, its a react badge component, it doesn't need punching up. You didn't find the smoking gun in a murder mystery, just a misplaced CSS rule.
I've noticed this as well, I've been comparing it to 4Chan Greentext memes. Everything is a short punchy sentence, and every sentence has the same "weight" to it. There's no flow or rhythm, it's like reading off bullet points.
I’ve got the same bright red line for 2023. E.g. I treat a handful of youtube channels with long electronic jams almost like oldies at this point - keep listening to the same stuff from at least two years ago because it’s the authentically good stuff.
LLMs have made the internet unreadable. It used to be just Google search results were SEO-gamed empty prose, but now it's every blog post, news article, product announcement ... everything. I've stopped reading basically everything online except user comments on a few websites. It's the last place remaining to actually connect with other human minds, but even that seems like it won't last forever.
to AI verbosity and weird use of diction is natural, to humans it makes it hard to parse. The number of time i start reading long winded AI written design docs, and i just can't get into the flow after reading a few sentences and have to keep re reading certain sentences.
Agree but also want to point out that this started with the SEO boom of the last decade where sites started rambling so they would keep you on the site longer and get more SEO "points", they were articles meant for a robot to catch humans.
AI has made it even worse and lots of content on the web, for both these reasons, is not worth you time (now wonder users don't read most of the text on our apps!)
In AI-generated entertainment, the names "Marcus", "Chloe", "Sterling", "Meridian", and "Apex" keep appearing. Those aren't common names, but LLMs seem to prefer them.
This applies to code btw, it's so much nicer to read a human made pull requests. It's also much faster as people tend to be lazy and produce denser code.
Personally, for blog posts, documentation (unless I flat out vibe code the entire thing) etc are all written by me by hand. If the model can save me hours of "oops that was really dumb of me" why can't I take time to read and document things by hand? I think everyone's embracing AI in the wrong ways. Maintain your knowledge of things, and keep the documents nice. There's still room for letting your agent manage agent specific docs, but for the humans, just write it yourself, ask the model for feedback and tweak it by hand. Let the machine talk to the machines.
Broken record: LLMs are exceptionally useful writing critics if you know how to hold them correctly, but you can't use a single word they suggest to you --- the better the LLM's word choices seem, the more likely the LLM is infecting your voice. Better, by a lot, to prompt the LLM not to suggest any words, but just offer objective critiques.
I see many Youtubers who are certainly still writing their own scripts, but every once in a while you can tell they absolutely used a Claude correction in one place.
IMO, it's still better to not let an AI critique your writing style at all, only the facts and claims you're making against its context / MCPs. Your own writing style is a reflection of your mind's thought process and, especially on forums like this one, is the only way to imbue your ideas with your personality.
People have been talking about the importance of curation for more than a decade.
Because many good ideas are not immediately obvious good ideas, in an environment at risk of information overload, we need some way to quickly vet ideas to see whether they are worth investing the time to consider them. (It is often the case that the best ideas require considerable investment of time and effort to be judged as good, and, it's also often the case that this determination can be made only at the very end of such a process.)
“Don't judge a book by its cover,” sure, but, historically, there have been superficial markers of care and attention that we've used to these ends. These are obviously very flawed signals, but, you know, maybe if you took a lot of time to craft a message, I don't feel that bad if I waste a bunch of time trying to decode it. These markers—attention to detail (e.g., spell and grammar checking,) clear-structure (e.g., “bottom line up front,”) and typographical flourishes (e.g., smart quotes and em-dashes and extended unicode like diæreses…)—were an indication of some modicum of quality control and care. I, myself, used to brag about or show-off having set up .Xcompose keybindings (https://wiki.archlinux.org/title/Xorg/Keyboard_configuration...) which was just rare enough across the general population to be noteworthy.
All of this that was once somewhat expensive is now cheap or basically free.
Even if I make a point of going back to a pre-UTF-8 Usenet style (or even intentionally insert run-on sentences or misspellings or grammatical errors,) the cost of an LLM to style-transfer its output to match this is barely much of anything. All it means is I wrote one prompt for the first draft, and then one more prompt for the refinement.
Additionally, some LLM habits are actually good habits. Putting the bottom-line up front (in bold, even!) is a great habit in a business e-mail! Decision-makers genuinely struggle with reading through long paragraphs of detail!
Unfortunately, the problem with the LLM output is that it takes good, valuable, superficial or structural habits that it can cheaply apply, and it drapes those around inane, banal, cloying, muddy, meandering meaninglessness.
Today, we can maybe still judge a book by its cover, because even one more prompt worth of refinement is still too much effort, but I don't think that's amounts to much, since the frequency and loudness of complaints of LLM writing means that model trainers are going to train their models to adapt to whatever signals we use for quick vetting. I am skeptical whether this training will concurrently improve the novelty, innovativeness, or other intangible qualities of the actual message. (With the models, as they exist today, and as we use them today, at least in the field of software development, I see extremely poor capability at generating innovation… Personally, I don't even like the quality of the Python code they spit out, but they sure do type real fast, and, like, pejoratives aside, unexciting, mediocre, corporate-y code does have a real place in the world.)
So what happens when the economics of production in the medium significantly outpace the capabilities of production in the message? The LLM can write a comment nearly indistinguishable from any given human… except when you actually take the time to read through it, it's just inane drivel. Thoughtless and a waste-of-time. And there's basically no way to manage your investment of time and effort to tell one way or the other.
You know, maybe it was never the case that superficial markers of effort ever meant anything at all. Maybe there never was a way to tell from the envelop whether the letter is worth reading… and it was confirmation bias all along. Maybe there really are no shortcuts to judge whether an idea has any merit…
Because, for sure, some of the real bangers of comments I've posted on HN and similar forums—all my XCompose-configuration aside—have gotten relatively few upvotes!
> All human communication is based on internally simulating the presumed mind that was/will-be at the other end
Which requires a bunch of things an LLM lacks, such as theory of mind, knowing who it is talking to, and willingness/ability to engage in short turn conversation to assess and steer the conversation according to the feedback it is getting from the person it is talking to.
Thank you for posting this.
I don't have a lot to add on the subject, but I found it interesting and thought-provoking given the age we live in.
I find it really hard to tell real human and LLM text apart, even though some people claim they can, calling it "AI slop".
> 3: Michael Nygard commented: “There you really aren't trying to communicate thoughts from one person to other people. You're communicating with a process rather than a human. A human may open the envelope but that person is acting as a component in a cybernetic organism.”
But of course: umanufactuting itself creates " productive mechanism whose parts are human beings, as Adam Ferguson’s describes the industrial workshop: "an engine, the parts of which are men".
121 comments
[ 0.27 ms ] story [ 23.1 ms ] threadYeah. I want to be able to touch a human being (or at least have one touch me) through the words. LLMs poison that, even if a human prompted the LLM.
Translation... I think I object to translation less than Martin does. There's a human back there, and they're writing all the words, and then an LLM is translating them. The LLM may not translate them perfectly, but they aren't going to introduce "load-bearing" when it wasn't there.
The bit about using LLM when you're writing for a bureaucratic process rather than a human was interesting, at least to me. I hadn't thought of it that way before, but I think it makes sense.
I enjoy cooking from scratch, and I really don't like frozen foods, microwave/TV dinners, or even DoorDashing
but we're past complaining/arguing/shaming folks who don't cook every meal from scratch
you know what, I don't prefer eating it, but if my next door neighbor loves microwaving frozen $3 burritos, then go ahead
to some people, food is art, creativity, life to others, food is fuel
also applies to writing and thinking
It’s not ok to write stories with AI and post them for others to read claiming “I did that”
Get the difference?
Some of them purchase ingredients from those companies, and then make the food using those ingredients.
A great deal of restaurants, though, simply purchase pre-made frozen foods from those distributors, re-heat, and sell it to you at a premium without disclosing while you think it was freshly made. Mac & cheese, stuffed peppers, lasagna, burritos, most 'bar snacks' you can think of, etc.
basically any food establishment that has "supply chain operation" is slop to me; I guess it's similar to my distaste in reading AI slop too
I don't fault ppl for going (and enjoying) those too- Chili's can be enjoyable after a few beers
happened to me once lol, will never forget that place ever again (blech)
right. he is talking about the first group. where did you get the 'shame' part?
I like your analogy. Microwaved empty calories perfectly describes llm output. You are going to pay the price sooner or later.
That is not to say that I don't think there is a place for LLM assisted writing. Writing is an exercise for the author and translates to oral argumentation so I don't think that fades away. But LLMs could do some "audience normalization". What if I could make a blogpost that is dynamic based on experience of the reader? There are lots of opportunities, but versus other fields where optimizing for laziness can be beneficial I don't think lazy writing currently has much value add.
It's your turn to host supper club this week and you serve frozen diners on your nice tableware.
Same thing with LLM-assisted writing. It’s one thing to do it for yourself, quite another to share it with other folks without warning.
Most food is literally just fuel. I think that is fine. And at some level, eating 3$ burritos for a quick lunch on the daily really makes you appreciate the depths of a home made risotto even more. :]
Add to that the fact that AI is significantly worse for you than TV dinners, and has an outsized harm to humanity, and the conclusion is plain:
AI should not be imposed on others.
Technical skills aren't worth shit and never were, except as far as they enable you to engage with other people in ways you and they find valuable. This used to be really far! It made it easy to believe that the technical skills on their own were what was valuable.
But the engagement was always the valuable thing, generating technical insight was a necessary prerequisite for it. Now anyone with a $20 claude sub can generate technical insight into niche problems, so what's left is everything else. Everything of value.
I think you're misunderstanding that anxiety. I think this is what's going on:
1. For a lot of people, using technical skills like coding is the part of the job they enjoyed.
2. Closely related to the above point: for a lot of people, technical skills was the part they were good at.
> Technical skills aren't worth shit and never were, except as far as they enable you to engage with other people in ways you and they find valuable. This used to be really far! It made it easy to believe that the technical skills on their own were what was valuable.
> But the engagement was always the valuable thing, generating technical insight was a necessary prerequisite for it. Now anyone with a $20 claude sub can generate technical insight into niche problems, so what's left is everything else. Everything of value.
3. More broadly: if "they" don't find the "technical skills" you provided valuable anymore, why do they need to keep you around? Why not cut your 10-person team down to 3 people (the manager, architect, and technical lead) and fire you and the other 6? Not everyone can re-invent themselves as a product person, even if they had the skills. It's a game of musical chairs.
The problem with LLM voice is that it sends the signal that it's a likely waste of effort to try to figure out if there's any "technical insight" there, because more often than not, there isn't.
Maybe it's not fair that you're paying the price for others using LLMs to abuse their readers, but it's the reality that you have to deal with. It's also not fair to expect readers to wade through all that abusive LLM prose in order to find your (assumed) non-abusive post.
Once LLM writing becomes indistinguishable from human writing, it will be really sad. There will just be a collapse of human engagement, and you might as well just talk to your machine because then at least you know what you're getting and aren't being deceived.
An LLM prose article ABSOLUTELY can mean that. To attempt to conflate the two things is maliciously misleading.
I can’t distinguish between well meaning slop and blogspam slop. I might be able to if I’m well informed on the topic and can point out inaccuracies. Otherwise I just don’t know what is accurate and what is filler or outright wrong. LLM-isms are very distracting and exhausting to sort through. It just doesn’t make for a good reading experience.
There are few things (relatively speaking, I mean online activities, etc) more annoying than seeing a click-baity headline, and then having to read 17 paragraphs to actually get to the point that I realize I already knew and understood it, and it's just slop padding.
It's the modern equivalent of going to a thesaurus and replacing all your natural words with five-dollar ones.
If you're not confident in your English then just write your brilliant ideas in broken English. Or at most do a direct translation. But don't fluff it with AI's nonsense.
If the ideas are good they will be recognized.
I find the same thing with written results out of AI "research", it is a slog to read through it and most of it can be skipped even if it can sometimes be useful its an unenjoyable read.
Maybe that's the key ingredient that's missing from LLM-generated text. Even when there is actual content there, there's this lack of surprise / uncertainty in its delivery that we require to stay engaged. We may not know what will be communicated, but knowing exactly how it will be communicated just kills our innate interest.
Communication expresses an understanding. LLMs arent communicating.
LLMs generate code well enough. Code has no natural language semantics. Programming's condition of satisfaction is that the machine executes correctly. The condition of satisfaction of communication is that the communication expresses what there is to be communicated.
In the limit where an LLM is trained to repoduce an exmplar of communication, it seems as if it does so. In that limit, LLMs can reporduce anything.
And the discipline gets attacked even more when all you read is LLM text. For example in code reviews there is no noticeable value in answering LLM generated review comments. I remember taking great care answering human-written reviews comments with much thought to either further build on their suggestions or to forward knowledge about the system when I had to defend a particular change.
Now, some bot will read my carefully written response and forgets about it. There is no value anymore.
No wonder that our pull request comments feel like the dead internet theory right now.
I am absolutely pro AI assisted coding. But communication between humans HAS to stay human. I am currently experiencing the opposite and it’s bad.
s/time/effort
I find that it often isn't just too low, but also too high at the same time. It has a lot of information that is irrelevant, burying far too little relevant information for the work needed to extract it. I wonder if this is a side effect of attention mechanism, where the AI's ability to track information in a document is far less hampered by other information, leading to it organizing information to be both dense per word but any specific concept to be sparsely spread throughout the writing. I also wonder if this is why the order feels off, because the AI doesn't need for the information to be well ordered to the same extent that a human does, so the order of storing information is often overlooked when storing it into a document (and even information that needs to be ordered might have the ordering information separate from how it is ordered in the document).
I have to believe that we are within a couple of years if not months of this. This problem is too well-known to not be a major focus of the frontier labs' efforts.
Not least because it’s getting worse rather than better. But also because as AI gets better at writing like humans, humans will change how they communicate to avoid being confused with it.
Which will get a lot easier once human writing is nearly extinct. Now that could happen in two years.
I wrote down a few of my thoughts a few weeks ago at https://notes.pault.ag/why-write/ -- I find myself scanning text to see if it's LLM before I actually engage with the content.
It is getting harder to do by the day, and has slowly moved from a mental exercise (look for specific markers) into a "gut feel" based on how the pieces fit together.
And of course I’ve dropped my use of the em-dash after being accused of being AI myself.
Here the reader wonders whether "to" was omitted intentionally to make a point?
> But I also would prefer rather a genuine human's clunky prose to the smooth but soulless LLM.
Amen. This reminds me of a bit from Kitchen Confidential about how Anthony Bourdain would prefer a simple and plain home cooked meal over an ornate restaurant meal any day of the week. I read because I want to hear what a human has to say. It's the errors and malapropisms and style that give a unique and human voice.
If you care about your voice, don't let LLMs write your words. But that doesn't mean you can't use AI to think, critique and draft lots of words for you. It depends on what purpose you're writing it for. If you're writing an impersonal document, like a design document, briefing, etc then who cares. In many cases (scientific papers, legal documents) you already have to write them in a voice that is not your own. Go ahead and write these in AI. But if you're trying to say something more personal then the words should be your own, AI will always try to 'smooth' out your voice, and if you care about it, you gotta write it yourself.
Now, how do you use AI effectively and still retain your voice? Here's one technique that works well: start with a voice memo, just record yourself maybe during a walk, and talk about a subject you want, free form, skip around jump sentences, just get it all out of your brain. Then open up a chat, add the recording or transcript, clearly state your intent in one sentence and ask the AI to consider your thoughts, your intent and ask clarifying questions. Like, what does the AI not understand about how your thoughts support the clearly stated intent of what you want to say. That'll produce a first draft, which will be bad. Then tell the AI all the things that don't make sense to you, that you don't like, just comment on the whole doc, get a second draft. Ask the AI if it has more questions for you, you can use live chat to make this conversation go smoother as well, when the AI is asking you questions, you can talk freely by voice. Repeat this one or two more times, and a much finer draft will take shape that is closer to what you want to say. During this drafting state, the AI will always try to smooth or average out your ideas, so it is important to keep pointing out all the ways in which it is wrong.
This process will help you with all the thinking involved being more up-front. Once you're read and critiqued several drafts, all your ideas will be much more clear. Then, sit down and write your own words from scratch, they will come much easier after all your thoughts have been exercised during the drafting process.
No more LLM voice! Every now and then you might get a nonsensical artifact or two, but that's the price you pay for better groceries.
Endless headlines like, "aArON SORkIn SAYs hE WiLL Never UsE ai to WRite" doesn't change that base fact. No one asked him to, other than clickbait journalists.
If you find this hard to believe, just think about relative market sizes.
If you built a model only based on long form journalism and The New Yorker voice, including both the template structure of the article and the grammar and syntax, the model would replicate the patterns it is built upon. You could do the same for high vocabulary rap.
When you stuff all of English or all of human thought into it with somewhat equal weight, including very technical writing the sum is a lack of style and elegance and personality.
The second problem is that writing often doesn’t happen linearly. You write bookends and fill in, edit and revise. You sort of know where you’re going and can then make decisions how to get there. Language models may benefit from being redesigned to perform passes. Select a template of some sort, fill in the mad lib, then run a detail pass, then run a persuasion / emotion pass, then run a style pass. Current model writing very much sounds like I would expect autocomplete to.
I suspect they’re leaning heavily on some kind of auto verifier, and that’s why the writing sucks, because good writing is so subjective.
And they’re probably nerfing what it can get from training as they’ve lost lawsuits for scraping copyrighted work, so they’ll purposefully avoid anything that references a specific writers style, etc.
That is, the original training on massive text corpora seems to produce an average of human writing, but that average looks pretty average. It doesn't feel like LLM slop. But the instruction tuning on specific tasks (answer this prompt, summarize this text, do this math problem), which turns text predictors into chatbots, seems to create strange style features as a side effect. And because that tuning step depends on company-specific training data (rather than just a crawl of the entire Internet), each company's LLM has a different style.
It's not that difficult to measure this. Linguists have had measures of style variation for a long time, though NLP and AI researchers seem to have forgotten. We've generated parallel human and AI corpora to measure it, and can show specifically which writing features change: https://arxiv.org/abs/2410.16107
Currently we're working on tweaking the instruction tuning process to see if we can pin down which steps and what types of tasks tend to produce the weird style features.
It's not just predicting content but also the form/shape of the response, highly shaped by RLHF preferences, which seems to encourage longer form responses which need padding with filler to generate the "preferred" amount of verbiage.
It seems most of these models have been trained to interpret questions as wanting a long-form authoritative/comprehensive-sounding response (even when midwit filler is all it has to offer), rather than initiating a short-turn conversation which would better allow you to steer the conversation. Apparently Muse is much better in this regard, but so far I've resisted signing up for any Meta/FaceBook service, and not inclined to break this winning streak!
> how awful everything is to read lately > flooded with AI generated content, all of it terrible > everything sounding identical and everything being a rambling unconcise mess.
I can see the benefits of AI and enjoy using the technology - but, like many have said, it's wreaking havoc on the "content" that's out there nowadays. Gotta take some bad with the good, I suppose.
I love computers and the internet, and the internet has ruined a lot of people and things around me.
It's possible to be interested in a new technology and enjoy its leverage while still being extremely apprehensive about how it intersects with game theory and politics. The race to the bottom of the barrel keeps coming for the jugular of every cool new thing.
Well, it's quite simple if you think about it. Intelligence is all about awareness of the imminence of death and the finite amount of time given to any of us. So, we humans intrinsically evolved to make the most of it. The majority of the tools we create have the ultimate purpose - to save time. Even money is nothing but a way to cut some time. It's not really about time but experiences - we want to have as many and as rich as we can get, and we innately try to cut time somewhere else, so we can use what we saved for better experiences.
Therefore, inventing cars was an easily predictable event. There's not a single hypothetical timeline or dimension where humans don't end up with cars - to transport things and to travel faster. Similarly, it was always simply inevitable - we would have invented something like LLMs, sooner or later.
Cars required significant sacrifices - they are absolutely deadly machines, they kill thousands of people every single day, kill wildlife, endager our own existence, and we're kind of okay with that. I can absolutely assure you, if someone tomorrow invents a gravitational jump travel that allows moving things between continents in minutes, even with a success rate as low as 10%, we would still be using it.
So, yes. We can't un-invent cars and AI, the only way to move forward is to adjust to this new reality and extract the best possible experiences, preferably by not robbing other people of their - i.e., enjoy life while it lasts, it still remains finite.
It makes no sense on being "aggressively pro AI" without any payment. It is like being an unpaid shill.
Yeah, thanks dude. We appreciate your efforts to enshittify everything.
Edit: Sure, downvote, but the parent comment is really not very different from "arsonist puzzled and upset that so many buildings have burned down".
AI has made it even worse and lots of content on the web, for both these reasons, is not worth you time (now wonder users don't read most of the text on our apps!)
In AI-generated entertainment, the names "Marcus", "Chloe", "Sterling", "Meridian", and "Apex" keep appearing. Those aren't common names, but LLMs seem to prefer them.
IMO, it's still better to not let an AI critique your writing style at all, only the facts and claims you're making against its context / MCPs. Your own writing style is a reflection of your mind's thought process and, especially on forums like this one, is the only way to imbue your ideas with your personality.
Because many good ideas are not immediately obvious good ideas, in an environment at risk of information overload, we need some way to quickly vet ideas to see whether they are worth investing the time to consider them. (It is often the case that the best ideas require considerable investment of time and effort to be judged as good, and, it's also often the case that this determination can be made only at the very end of such a process.)
“Don't judge a book by its cover,” sure, but, historically, there have been superficial markers of care and attention that we've used to these ends. These are obviously very flawed signals, but, you know, maybe if you took a lot of time to craft a message, I don't feel that bad if I waste a bunch of time trying to decode it. These markers—attention to detail (e.g., spell and grammar checking,) clear-structure (e.g., “bottom line up front,”) and typographical flourishes (e.g., smart quotes and em-dashes and extended unicode like diæreses…)—were an indication of some modicum of quality control and care. I, myself, used to brag about or show-off having set up .Xcompose keybindings (https://wiki.archlinux.org/title/Xorg/Keyboard_configuration...) which was just rare enough across the general population to be noteworthy.
All of this that was once somewhat expensive is now cheap or basically free.
Even if I make a point of going back to a pre-UTF-8 Usenet style (or even intentionally insert run-on sentences or misspellings or grammatical errors,) the cost of an LLM to style-transfer its output to match this is barely much of anything. All it means is I wrote one prompt for the first draft, and then one more prompt for the refinement.
Additionally, some LLM habits are actually good habits. Putting the bottom-line up front (in bold, even!) is a great habit in a business e-mail! Decision-makers genuinely struggle with reading through long paragraphs of detail!
Unfortunately, the problem with the LLM output is that it takes good, valuable, superficial or structural habits that it can cheaply apply, and it drapes those around inane, banal, cloying, muddy, meandering meaninglessness.
Today, we can maybe still judge a book by its cover, because even one more prompt worth of refinement is still too much effort, but I don't think that's amounts to much, since the frequency and loudness of complaints of LLM writing means that model trainers are going to train their models to adapt to whatever signals we use for quick vetting. I am skeptical whether this training will concurrently improve the novelty, innovativeness, or other intangible qualities of the actual message. (With the models, as they exist today, and as we use them today, at least in the field of software development, I see extremely poor capability at generating innovation… Personally, I don't even like the quality of the Python code they spit out, but they sure do type real fast, and, like, pejoratives aside, unexciting, mediocre, corporate-y code does have a real place in the world.)
So what happens when the economics of production in the medium significantly outpace the capabilities of production in the message? The LLM can write a comment nearly indistinguishable from any given human… except when you actually take the time to read through it, it's just inane drivel. Thoughtless and a waste-of-time. And there's basically no way to manage your investment of time and effort to tell one way or the other.
Because, for sure, some of the real bangers of comments I've posted on HN and similar forums—all my XCompose-configuration aside—have gotten relatively few upvotes!
Same thing with conference talks. The ones with half a million or a million views on YouTube… the real magical ones, I think, are the ones no one seemed to pay any attention to… e.g., https://www.youtube.com/@PyDataTV/search?query=james%20powel...
Which requires a bunch of things an LLM lacks, such as theory of mind, knowing who it is talking to, and willingness/ability to engage in short turn conversation to assess and steer the conversation according to the feedback it is getting from the person it is talking to.
But of course: umanufactuting itself creates " productive mechanism whose parts are human beings, as Adam Ferguson’s describes the industrial workshop: "an engine, the parts of which are men".