Can’t recommend the book that coined this acronym enough: The WEIRDest People in the World
Book by Joseph Henrich.
It is such an eye-opening piece that explains so much of the world around us. He’s an anthropologist that goes into the psychology of it all. Touching on points like how religion plays a part in shaping the America of today and even how humans are worst at discerning faces today because we need to discern letters and words and dedicate brain power for that.
There are so many interest studies mentioned there, one that really stuck with me is how Protestant-raised Americans will work harder for the next day after having (reasearch-led) incestuous thoughts when compared to Catholics and Atheists.
He explains how monogamy is to blame for a lot of our western views today, and how Mormon towns in Utah were affected by not having monogamy as the basis of society (women there tend to prefer to be 2nd wives of a better man rather being the only wife of a lower-ranking man).
One of the wildest claims in there is the one that the north of Italy is more developed today because it was part of the Holy Roman Empire while the south wasn’t. About a thousand years separate these and he finds effects still. Mostly in connection to the spread of read/write to the public being a core tenant of Protestantism.
Anyway, this is not a summary of the book but instead a few points from it that really stuck with me after reading it. Fascinating stuff
Funny thing with the words. I remember before I could read and billboards just looked like designs. Then once I could read it was like reading was unavoidable, it could not be shut off and you are constantly reading instead of just appreciating the text as some design pattern. At the time I felt ripped off.
It seems like almost all contexts might get value from specialized training. People often vary radically depending on where they were raised and where they live, their occupation and social class, and a range of other factors. Even workers from essentially identical backgrounds but practicing different trades can have very different perceptions and framing for what might appear to be shared tasks.
There is undoubtedly a real effect here, but IMHO one problem with the original article is that it treats the US as the only reference point.
On the one hand, this reflects the US dominant position in world affairs and the fact that probably most of the training materials come from there.
But on the other hand, there are some outlier results that are left unexplained. For example, ChatGPT is even more aligned with Japan than with the US.
I have saved instructions for Gemini to translate queries into the local language then retranslate the output back to English, when asking about non-English speaking countries/cultures. It seems to work fairly well, but I think it's just due to the different content trained in that language; obviously there would be more in depth discussion of Indonesian cuisine in Indonesian. Whether the country is rich or democratic shouldn't really affect the output.
“Do non-American LLMs (e.g. DeepSeek, Mistral, Apertus) perform better or worse here? Do they have their own cultural biases in-built?”
I'm wondering the same thing, in addition to the related question of “Would an LLM perform better or worse if prompted with languages other than English?”.
Social media reflects a silicon valley perspective and US domestic news have contaminated the entire eurozone since over a decade.
Movies are also a distillate of a local culture often with a rather uniform ideological slant.
AI being a clone army of corporate spokesmen from the US west coast brings sparsely little new cultural homogeneity to the already very smoothed table.
Another paper that echoes similar concerns — AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances (https://arxiv.org/abs/2409.11360)
Some of the questions don't really make sense to ask an LLM (being about the survey taker's personal financial situation and such), but the paper doesn't seem to go into detail of what questions were used.
>In fact, this paper found that more than that, it thinks American.
I think that's because it seems to be primarily trained on reddit and therefore mirrors everything reddit stands for. Not a good thing considering just how overrun the site is with bots and political activists of all kinds.
Sooo it's like training a robot to "think like a human," but all the reference humans are Silicon Valley product managers and undergrads from elite universities
There may be a real point here but this post and paper are not good evidence for it.
The blogpost doesn't have a date, but links to a 2023 preprint, which is hard to evaluate b/c it doesn't actually have a methods section, despite referring to it multiple times. (Did this ever get published?)
But it _sounds_ like they asked GPT via API to do the same survey 1000 times, without telling it to attempt to model the preferences of any particular country, but both the blog and the paper are interpreting a correlational analysis as evidence that it's bad at modeling local values.
> The greater the cultural distance between a country and the USA, the less accurate ChatGPT got at simulating peoples’ values.
> This correlation represents the similarity between variation in GPT and human responses in a particular population; in other words, how strongly GPT can replicate human judgments from a particular national population.
And to some degree, this is more a portrayal of the difference in human responses than anything about GPT; given the survey data, no matter what responses the LLM gives, it's going to be closer to some national averages than others.
LLMs also have a characteristic default voice/style which we're annoyed by, but _when instructed_ it can mimic another style. If you have some multi-dimensional style space, yes you could find the group that it's closest to, but it would be misleading to say it does a poor job "simulating" or "replicating" others if you didn't actually test that.
29 comments
[ 2.8 ms ] story [ 55.2 ms ] threadThere are parts of the world where constant person-electronic connection isn't a thing. Is that your point?
It is such an eye-opening piece that explains so much of the world around us. He’s an anthropologist that goes into the psychology of it all. Touching on points like how religion plays a part in shaping the America of today and even how humans are worst at discerning faces today because we need to discern letters and words and dedicate brain power for that.
There are so many interest studies mentioned there, one that really stuck with me is how Protestant-raised Americans will work harder for the next day after having (reasearch-led) incestuous thoughts when compared to Catholics and Atheists.
He explains how monogamy is to blame for a lot of our western views today, and how Mormon towns in Utah were affected by not having monogamy as the basis of society (women there tend to prefer to be 2nd wives of a better man rather being the only wife of a lower-ranking man).
One of the wildest claims in there is the one that the north of Italy is more developed today because it was part of the Holy Roman Empire while the south wasn’t. About a thousand years separate these and he finds effects still. Mostly in connection to the spread of read/write to the public being a core tenant of Protestantism.
Anyway, this is not a summary of the book but instead a few points from it that really stuck with me after reading it. Fascinating stuff
On the one hand, this reflects the US dominant position in world affairs and the fact that probably most of the training materials come from there.
But on the other hand, there are some outlier results that are left unexplained. For example, ChatGPT is even more aligned with Japan than with the US.
In other words, is the issue in the defaults or is it impossible for AI to respond from other cultures?
I'm wondering the same thing, in addition to the related question of “Would an LLM perform better or worse if prompted with languages other than English?”.
Movies are also a distillate of a local culture often with a rather uniform ideological slant.
AI being a clone army of corporate spokesmen from the US west coast brings sparsely little new cultural homogeneity to the already very smoothed table.
(For Wave 7 (2017-2022), which the paper used)
Some of the questions don't really make sense to ask an LLM (being about the survey taker's personal financial situation and such), but the paper doesn't seem to go into detail of what questions were used.
I think that's because it seems to be primarily trained on reddit and therefore mirrors everything reddit stands for. Not a good thing considering just how overrun the site is with bots and political activists of all kinds.
The blogpost doesn't have a date, but links to a 2023 preprint, which is hard to evaluate b/c it doesn't actually have a methods section, despite referring to it multiple times. (Did this ever get published?)
https://osf.io/preprints/psyarxiv/5b26t_v1
But it _sounds_ like they asked GPT via API to do the same survey 1000 times, without telling it to attempt to model the preferences of any particular country, but both the blog and the paper are interpreting a correlational analysis as evidence that it's bad at modeling local values.
> The greater the cultural distance between a country and the USA, the less accurate ChatGPT got at simulating peoples’ values.
> This correlation represents the similarity between variation in GPT and human responses in a particular population; in other words, how strongly GPT can replicate human judgments from a particular national population.
And to some degree, this is more a portrayal of the difference in human responses than anything about GPT; given the survey data, no matter what responses the LLM gives, it's going to be closer to some national averages than others.
LLMs also have a characteristic default voice/style which we're annoyed by, but _when instructed_ it can mimic another style. If you have some multi-dimensional style space, yes you could find the group that it's closest to, but it would be misleading to say it does a poor job "simulating" or "replicating" others if you didn't actually test that.
I would imagine chatgpt is more similar to Kimi than the US is to China which suggests a different trend.