We analyzed 7,877 responses from 250 models (same prompts, controlled conditions) and found interesting patterns that don't show up in standard benchmarks.
→ Name hallucination convergence: "Professor Chen" appears 279 times across unrelated creative writing prompts. 19/250 models independently chose "Sir Reginald" for a knight character.
→ Provider fingerprinting via punctuation: Claude averages 5.90 em dashes per 1K words. Gemini: 9.18 exclamation marks. Mistral: 3.48 emoji. Distinct enough for classification.
→ Cultural reference bias: 59:1 Western to non-Western ratio. 1,069 Western references vs 18 non-Western across the full corpus.
→ Humor convergence: 42% of models open with the identical atom joke. Format convergence is total: 100% produce setup-punchline one-liners. Zero attempt observational comedy.
→ Visual generation defaults: "Draw a surprise animal" produces a fox 40% of the time (DeepSeek: 67%, Llama: 0%).
Full writeup in the link (the complete research is gated)
1 comment
[ 3.6 ms ] story [ 14.9 ms ] thread→ Name hallucination convergence: "Professor Chen" appears 279 times across unrelated creative writing prompts. 19/250 models independently chose "Sir Reginald" for a knight character.
→ Provider fingerprinting via punctuation: Claude averages 5.90 em dashes per 1K words. Gemini: 9.18 exclamation marks. Mistral: 3.48 emoji. Distinct enough for classification.
→ Cultural reference bias: 59:1 Western to non-Western ratio. 1,069 Western references vs 18 non-Western across the full corpus.
→ Humor convergence: 42% of models open with the identical atom joke. Format convergence is total: 100% produce setup-punchline one-liners. Zero attempt observational comedy.
→ Visual generation defaults: "Draw a surprise animal" produces a fox 40% of the time (DeepSeek: 67%, Llama: 0%).
Full writeup in the link (the complete research is gated)