Having just finished rewriting a 20-pager that went from outline -> slop -> finished product, I can say with certainty that it’s not a case of AI being too intelligent.
It could be a case of AI being differently intelligent, though. Here are some of the patterns I encountered:
Repetition: AI never breaks a sweat repeating the same thing over and over. This is fine with code because there are more tools to catch and eliminate it and even so, it matters less as long as the code works.
Jumbled statement order: AI has a glossary of facts and it is not very careful about the order of… shall I say, revelation… of facts. We have a certain way of building understanding in deductive steps. AI writing disregards that. (Aside: I actually started using an Answer Set Programming tool to keep the LLM on track.)
Mixing attribution and control: we have a sense of what things are passive, what are active, what controls and what follows or reacts in a situation. AI mixes it up badly and suddenly you have stuff like “the lead shade is red but it has nothing to show for its presence around orange” instead of saying that the red accent doesn’t contrast well against orange.
Post-training flavours: I think this is clear from how Claude models write vs. other trainers’. Even accounting for intelligence differences, Claude has an empirically worse writing style.
I see the distinction between differently intelligent and more intelligent. It still means that along some axis of intelligence, it is more intelligent than us. My intuition is that if that edge continues it will bleed into other axes.
Repetition: Humans repeat a lot! There is the famous advice about "Tell them what you are going to say, say it, tell them what you said".
Jumbled order: I think humans jumble order a fair bit too. Especially when they are in the process of explaining vs deducing. Even while deducing, i think we often infer first then reason later.
Mixing attribution and control is very interesting point! Makes me think of the philosophical definition of causality.
2 comments
[ 4.1 ms ] story [ 13.4 ms ] threadIt could be a case of AI being differently intelligent, though. Here are some of the patterns I encountered:
Repetition: AI never breaks a sweat repeating the same thing over and over. This is fine with code because there are more tools to catch and eliminate it and even so, it matters less as long as the code works.
Jumbled statement order: AI has a glossary of facts and it is not very careful about the order of… shall I say, revelation… of facts. We have a certain way of building understanding in deductive steps. AI writing disregards that. (Aside: I actually started using an Answer Set Programming tool to keep the LLM on track.)
Mixing attribution and control: we have a sense of what things are passive, what are active, what controls and what follows or reacts in a situation. AI mixes it up badly and suddenly you have stuff like “the lead shade is red but it has nothing to show for its presence around orange” instead of saying that the red accent doesn’t contrast well against orange.
Post-training flavours: I think this is clear from how Claude models write vs. other trainers’. Even accounting for intelligence differences, Claude has an empirically worse writing style.
Repetition: Humans repeat a lot! There is the famous advice about "Tell them what you are going to say, say it, tell them what you said".
Jumbled order: I think humans jumble order a fair bit too. Especially when they are in the process of explaining vs deducing. Even while deducing, i think we often infer first then reason later.
Mixing attribution and control is very interesting point! Makes me think of the philosophical definition of causality.