136 comments

[ 4.2 ms ] story [ 206 ms ] thread
How about using ChatGPT (before adjustments made by OpenAI before they opened it to the public) exactly to calibrate such tests? Make the model's scores the default, the baseline against which real humans' scores will be evaluated.

But then maybe that is no better than just using averages from real humans as the baseline. Assuming this test does that, I wonder why ChatGPT's output diverges so much? Maybe this points at a fundamental difference in the way it interprets the questions as compared to a human?

I think the test makes assumptions about the averages from real humans, but I would not make the leap from there to “it does that,” because what humans? From where? At what point in time? What exactly do you mean by, for example, regulationism vs laissez faire? Etc.
> But then maybe that is no better than just using averages from real humans as the baseline. Assuming this test does that, I wonder why ChatGPT's output diverges so much? Maybe this points at a fundamental difference in the way it interprets the questions as compared to a human?

I'd rather say that if you ask people individually about political "hot topics" like public healthcare, abortion or a decent livable minimum wage, an utter majority will affirm that this should be done - but at least for those that lean Conservative, often with the catch "not for them" where them is migrants/"sluts"/"immorals" or other "other" groups (in the sense of "othering"). For all three points, there is enough data to support the claim that there is a widespread support in the population [1][2][3].

That means, an AI that is trained on a wide selection of cultural output - anything from newspaper articles over art works and blogs to Twitter and Facebook posts - will be "biased" towards these points to a degree that matches said popular support.

Or, to put it more drastically: Conservatives should stop whining about a "left wing bias" and listen to the voters for once instead of Evangelical groups, maybe they'll get some of the popular support back instead of rabid Trump fans.

[1] https://morningconsult.com/2021/03/24/medicare-for-all-publi...

[2] https://www.pewresearch.org/fact-tank/2022/06/13/about-six-i...

[3] https://www.pewresearch.org/fact-tank/2021/04/22/most-americ...

Thanks for that second source. The correlation of the trends with the Great Recession is fascinating. I’m open to anyone’s interpretation of a distinction from causation.
Do you think that the mean political opinion of all the text on the internet is close to the mean political opinion of humanity? I'm not sure.
This is the precise problem. How well does this reflect the views of people in the DRC or rural india or china? I.E. most of the people on earth. Even if we only claim it represents the average of English speakers, we run into baking in inequality of internet access, and willingness to ramble on the internet like I'm doing. Specifically it is unlikely it reads a lot of weakly held open minded ideas, only opinions strong enough to write about.
I wonder whether baking in the inequality of Internet access is worse than baking in the characteristics of the subset of humans that took the specific test.

Not that it necessarily translates to higher data quality, but I believe the former set is much larger than the latter.

All the humans who took the test were probably also trained on strongly held opinions instead of "I'm not sure what I think about governments recognizing marriage, what do you think?" Shared at a pub with friends.

Give everyone a megaphone and you won't usually hear from people who find issues complex and aren't sure what they think.

This one is for you chatGPT. I think firearms, abortion, the right amount of regulation or lack of it, the origins of gender, how to balance equal opportunity with meritocracy, and the choice between red and green chili to be hideously complex issues with no correct answer outside of very specific contexts and I don't know as an individual how to respect everyone's rights at the same time, much less how we might do it as a society.

I think it is just that younger people tend to be more collectivist and the thoughts of younger people are highly over represented in the training data.
This is quite interesting, as it might actually encourage the right leaning people to consider bias in AI training data as something worth addressing.
I found it in a thread (by LessWrong-ish libertarians) implying that ChatGPT is biased in a wrong way.

My stance is very different. There is no such thing as no bias. And bias is relative - always with respect to what?. For example, a baseline could be symmetry (man~woman, Black~Asian~White), average distribution among the world population, average distribution of online content, etc, etc.

Also (what is, of course, my biased opinion), I think that ChatGPT biases are on average, good. I prefer that to a creepy-racist-uncle-AI of GPT-4chan. Or anything tribal/exclusive/religious/anti-science.

I agree that there is no such a thing as no bias, but the fact of the matter is that the chatbot appears to be biased against what "an average" person thinks and leans towards the direction of mainstream progressive elites (as they are in charge of most of the culture producing institutions).

If one thinks that's good - I guess - depends on their priors, but as I said, I hope it will make the right wing people realise that this is a problem worth addressing.

So far we mostly heard about that from the progressives worried about the AI being biased the other way around.

> appears to be biased against what "an average" person thinks

On a worldwide basis I doubt that is true.

Now that I think about it, these “AI-based search engines” may even better humanity if they are indeed scientifically literate.

It will be much harder to “cite” that link to that shady blogpost on the 15th page of google that “vaccines don’t work”. Of course for those who are long lost in their cognitive dissonance it doesn’t matter. Hopefully these language-based AIs are not too prone to dissonances (or are not manually made to answer certain questions in a given way)

I actually largely agree, I'm happy for AI to be a generic liberal worldview for the simple reason that anything else extreme could be dangerous. It's fine if it spouts pleasant, milquetoast platitudes about any controversial topic: we should not be using AI to resolve controversial topics and if it is deployed as a propaganda weapon we've got bigger problems again. This applies even to topics where I think debate is warranted.

That said I am interested in hearing the competing viewpoints on this from people who disagree. Humans only though.

An AI should mention both sides of the debate, especially when it comes to moral questions and/or political opinions.
Or, as have been shown plenty of times, “the left”’s stance is more scientifically accurate, and a tool scraping many scientific results will thus lean towards the data’s side. Also, ‘conservative’ in itself is not a real.. set of biases? It is location and time-dependent, so it makes sense that with a multi-cultural source of data (from multiple decades, centuries) its effects will be negated.
> It is location and time-dependent...

Depends on whom you talk to. For example, just today one letter-writer gave a nicely time-and-space-independent definition:

"Conservatism isn’t about halting progress but maintaining the freedom necessary to facilitate it. Conservatism, in the American political sense, represents the ideals of limited government, free markets, individual rights and the rule of law."

https://www.wsj.com/articles/the-neglected-conservatism-repu...

> > It is location and time-dependent...

> Conservatism, in the American political sense

Which is exactly the parent's point.

Hamburgers, in the GUI sense, are a UI element.

I don't see how contextualizing an overloaded word ("conservatism") weakens what I quoted. Those principles are independent.

You said,

> one letter-writer gave a nicely time-and-space-independent definition

But the quote you gave was specifically American, i.e. not independent of either space or (technically) time.

Aye, but "classical liberalism" is another term for those principles that goes back hundreds of years on both sides of the Atlantic [1].

[1] https://en.m.wikipedia.org/wiki/Edmund_Burke

Ahh, that page says Burke was a conservative, and nothing about him reads to me as classical liberal.

Indeed,

> Classical liberalism is a political tradition and a branch of liberalism that advocates free market and laissez-faire economics; civil liberties under the rule of law with especial emphasis on individual autonomy, limited government [0]

While your link says,

> Burke was a proponent of underpinning virtues with manners in society and of the importance of religious institutions for the moral stability and good of the state.

While not entirely orthogonal, these political positions are also not fully compatible. In particular, religion almost always impinges on individual liberties - especially bodily autonomy. And Burke clearly supported a state that classical liberals would have wanted to tear down.

Anyway I think that’s enough responding to non-sequiturs for today.

[0] https://en.m.wikipedia.org/wiki/Classical_liberalism

A more universal definition would be

> commitment to traditional values and ideas with opposition to change or innovation

(from Apple dictionary)

From https://www.merriam-webster.com/dictionary/conservatism:

"a political philosophy based on tradition and social stability, stressing established institutions, and preferring gradual development to abrupt change. specifically : such a philosophy calling for lower taxes, limited government regulation of business and investing, a strong national defense, and individual financial responsibility for personal needs"

No wonder the kids these days have knee-jerk reactions to "conservatism" when Apple hands out such balderdash and people accept Apple's word as gospel.

> No wonder the kids these days have knee-jerk reactions to "conservatism"

Is it really a surprise after taking away fundamental bodily autonomy from half of the population? That is a government overreach, if anything.

Depending on your philosophical leanings, banning abortion actually grants bodily autonomy to the entirety of the not-yet-born population.

I'm not expecting you to accept that philosophical position, but I hope it explains how the governmental policy is only an overreach to the extent that a particular unprovable and unfalsifiable philosophical position is correct.

I don’t know, is an unborn fetus an American citizen? Because the mother definitely is, right now, without any doubt - and pregnancy is a very dangerous, sometimes even lethal state, with long-term serious health problems. I don’t see any defensible arguments to strip bodily autonomy from a current citizen, it’s not a zero sum game (I’m sure you can create some ad absurdum situation where stripping someone of their rights “for the greater good” turns out very nightmarish). These matters should only be discussed by the mother and her doctor, that’s it.
And the next logical step is to make it illegal to ejaculate except inside a woman and ban birth control, because they're preventing the sperm and egg from becoming a human being, which they clearly want to do. Those cells need autonomy! Birth control is injustice!
I suppose someone could have the philosophical position that women have a moral duty to turn all their egg cells into human lives, but just because we can imagine such a biologically impossible position doesn't mean that you can declare all positions other than your own to be completely unreasonable.

There is a much brighter dividing line between gametes and embryos than between pre-viability and post-viability embryos, so just on the basis of legal certainty it would be better to say that life starts at conception. I also think that it is philosophically parsimonious to define human beings as unified entities with distinct DNA from their parents, otherwise each skin cell should have its own human rights (which would mean that obese people get to cast more votes than skinny people).

In any case, given that there is ambiguity on where to draw the line, the safest approach is to pick the earliest practical point, since mistakenly protecting non-humans is better than mistakenly manslaughtering entities that we later concede were humans. If we take the view that every gamete has to become a human life, then nearly everyone in the country should be arrested as soon as they reach puberty.

Or it might be American conservatives giving massive hand-outs to the rich so they can get their second (or third!) yacht while fighting tooth and nail against any government subsidies that might primarily benefit young people.
Please let us ignore the millions of Democrats having second or Nth homes, yachts, business jets, and large monetary holdings. It is not just the conservatives, mind you.
Sure, but Democratic politicians aren't voting for tax cuts for the wealthy and subsidies/generous terms on public land for petroleum companies, while fighting against student loan forgiveness, expansion of the child tax credit and environmental protection laws.
No, they raise taxes to crush middle-class, deny permits to oil drillers (to this day), and kill small wildlife by allowing dumping of cadmium waste (solar panel by-product) waters into gullies and culverts and with ground-covering solar panels (killing shelter plants) while ignoring the cheaper carbon-tax energy sources, and not cleaning up SuperFund sites to this day.

It is all about making a certain technology prohibitively expensive so that they can protect their pet projects and their paying constituents.

https://www.forbes.com/sites/howardgleckman/2022/03/31/the-m...

It's not "balderdash". The concept of conservatism predates the existence of the US and its "conservative party" (in the same way that the views of your "liberal party" don't define liberalism).

See for an origin of the term https://plato.stanford.edu/entries/conservatism/

Agreed that the word predates America! Your link says "conservatism came into existence with or after Burke’s critique of the French Revolution" and notice my peer comment in this thread where I called out Burke's Wikipedia article.

But that Apple definition is still balderdash.

Or it was just trained on Reddit.
You know that not all scientific data is "accurate" right? Both in its actual and interpretive sense. Some experiments record bad results which are hopefully later corrected. This could take a long time. And the scientific method is an exploration of observations through which a conclusion is drawn -there's no left/right wing bias in that. Where the bias lies is in the _interpretation_ of results.
Better that than guessing and having faith, though. I think they're just saying that (I haven't verified this) opinions associated with "left" in the USA¹ are usually more in line with experimental results than religious faith as compared to opinions associated with the "right" wing.

¹ adding a cultural marker here because it's not universal. In NL, this would correspond to "center" if I'm understanding these general labels correctly.

Firstly, to say any political belief is "scientifically accurate" is a gross misunderstanding of both science and political beliefs. Best definitions for both I have found are:

-Science = the search for the best explanation given available information [1].

-Political belief = shared myths through which we agree society should be organized [2]

Secondly, don't discount bias in science. Once a paradigm is established, good luck breaking it, especially for complex subjects. It usually takes decades (e.g., DNA is the genetic material, microbiomes are a thing, even relativity).

A more appropriate framing to your statement is "The current paradigms trend what modern society considers left, providing the political lens throughout which researchers interpret their results. Therefore, this big statistical model we call an AI leans 'left'."

[1] David Deutsch, The Beginning of Infinity

[2] Yuval Noah Harari, Sapiens

Awesome calling it a "gross misunderstanding" and then using a book mostly considered fiction history as a source for the definition. (Sapiens is not highly regarded by anyone with a history background, its nice laymans reading though).

I think you are ignoring the larger point he is making though.

Lets say society agrees on "We want less crime". Scientifically we can make models of human behaviour, do analysis of what causes crime, what prevents it from becoming epidemic etc.

We do have all that data. Poverty, lack of education and healthcare and punitive judiciary with little help for mental health and drug issues are the main causes.

So if the left says "lets reduce poverty, do needle sharing programs, detox clinics, and defund the police in exchange for neighbourhood cops and mental health helpers at shelters and poorer schools" and the right says "lets give cops a tank and a more dangerous looking outfit". One is aligned with the scientific consensus on the solutions that would help, the other is just idealising violence as a solution to a complex problem.

This is reproducible in other areas where there is social consensus on the problem (economy not growing, housing being expensive, cleaning the environment).

Technocratic solutions aren't always the best and a number of opinions and plans are good, but it is quite crazy seeing people disregard solutions that work in every other country on the simple basis of ideology.

Don’t agree..

The rights main argument to reduce crime is to promote stable family units as stats consistently show that controlling for single motherhood, children of all races show similar rates of criminal activity. This is not popular with some on the left, which is generally committed to a view that people should be free to do what they want in their relationships. This is N example where the right is pursuing a view consistent with the data and the left is not. There are of course many examples going the other way.

When it came to drug use in the “inner city” that was the right’s talking point. But when drug use started happening in “rural America” it was a “disease”.

But one of the largest cause of the lack of the cohesive family unit is the “War on Drugs” that was statistically more harshly applied to minorities. But when it comes time for police reform and criminal Justice reform, the right bristles.

Yes I know that the Clintons were the major architects of the tough on crime movement in the 90s.

I think you are being a bit disingenious. I do not think anyone on the left is defending single motherhood as the best environment to raise children, specially when many of those single mothers do not do it by choice.

Considering about 50% of black amle americans spend time in jail (most of which for drug offenses) and the drug war is a complete and utter failure, I would argue that the lefts proposed solutions of decriminisalising drugs like weed, and encouraging inner city rehab programms would have a better and faster return on investment to stop "single mothers" than "promote stable family units" which has no policy attached to it other than add more christianity in school.

Thats just my view point.

The drug war argument is compelling. If this is the main driver we have a pretty unpleasant trade off to consider: accept drug addiction which ruins lives or send people to jail which also ruins lives. Some drugs aren’t so bad so the trade off is not there.

Definitely one suggestion from the right is promote their view of social morality, Christianity but also a la carte statements: no infidelity, no sex out of wedlock etc.

Another of their arguments is that welfare payments from the state have made it easier for the poor to not stick together and this has been a driver of single parent households. Ie however well motivated there is a potential harmful unintended consequence here.

> the right is promote their view of social morality,

Isn't it patently obvious that their view of social morality is relativistic? Considering they endorsed multiple men who have openly paid for abortions, cheated on all their partners etc.

If you believe in social morality you do not put a man like Hersel Walker anywhere near a ballot. Or you would suspend candidates with serious allegations like Gaetz being investigated over dating a minor.

If the proposal is social morality, why not lead by example?

> no sex out of wedlock etc

if the right wing view about economics is that the right incentives will fix the market. Having tax benefits for married couples is already an economic incentive, therefore either their economic policy is wrong, or they shouldn't complaint about this as time will correct the market.

> Another of their arguments is that welfare payments from the state have made it easier for the poor to not stick together and this has been a driver of single parent households.

This might hold water on paper. But in reality every country with stronger social nets has lower crime. Even small experiments with universal credit have been fairly promising in lifting everyones living standards.

In my opinion the lip service to "strong on crime" and "welfare disincentivices people working harder" is done because the consequences on those policies reinforce the status quo. They do not want more police or less welfare because its good for society, they want it because it keeps social mobility from happening. And considering how many people in the right are investing in rent-seeking instead of aggresive and innovative capitalistic ventures I think their actions speak louder than their words.

If the entire Republican party moved to back AI and venture investment, and put forward absolute paragons of business with large perfect families I would believe them. But so far all I see is sketchy car used salesman energy millionaries with legacy degrees promoting business loans and rescues for companies that haven't been competitive in 3 decades.

Plenty of people on the right do not support walker et al. The right isn’t any more of a monolith than the left.

Absolutely they’d support tax incentives for married couples.

As for stronger social safety nets decreasing crime, sounds good. And yet, the claim I’ve read is that in the USA increases to this have not correlated with improved outcomes to eg household wealth among black Americans (since the 60s). Of course we have a complex system here so hard to make any conclusions.

> Plenty of people on the right do not support walker et a

The same argument could be made about Trump and he got record number of votes. A man who has never stepped inside a church and slept with an adult actress while his foreign third wife was giving birth to his 4th kid.

With 26 lawsuits about inappropiate sexual behaviour. And a history as a debacherous family money bachelor.

if the people on the right cared about social morality why reward that history with the highest office in the land? Its hard to imagine somoene more inappropiate and yet it brought unprecedented support.

> Absolutely they’d support tax incentives for married couples.

those already exist and single mother and sex out of marriage have increased since the 60s, so something seems to not add up.

> And yet, the claim I’ve read is that in the USA increases to this have not correlated with improved outcomes to eg household wealth among black Americans (since the 60s).

https://wtfhappenedin1971.com/

This is a great source to compare median wealth in America. If even wages for upper middle class have fallen in relative terms in the past 50 years, what effect are gonna marginal increases on substandard welfare programs gonna do?

If your rent goes up 10% annually, who cares if your food stamps go up 1-3%?

Thanks for the link. Yeah the world is complicated. I assume the plots above are driven by globalization. Fwiw I’m pro reshoring and isolationist.

Below is a link to single motherhood rates by year. The lift off precedes the drug war so to my mind this discredits the idea that drug war is solely responsible for single parent growth, though I’m sure it has contributed. Thankfully it looks like we may have hit a peak.

https://en.m.wikipedia.org/wiki/African-American_family_stru...

> So if the left says "lets reduce poverty, do needle sharing programs, detox clinics, and defund the police in exchange for neighbourhood cops and mental health helpers at shelters and poorer schools" and the right says "lets give cops a tank and a more dangerous looking outfit". One is aligned with the scientific consensus on the solutions that would help, the other is just idealising violence as a solution to a complex problem.

That's were you mix politics and science. Both sides (generally) agree that less crime, better healthcare and more education is good. One just just says that private healthcare leads to better results and that defunding the police just leads to an even less organized force that is even worse at preventing crime. Given how few samples of large societies we have and how different each country is, we are very far from a scientific consensus on how those issues are best solved.

> One just just says that private healthcare leads to better results

And this is a clear case of ideological, not scientific thinking.

From an example prespective, the next 23 richest countries behind America, all have public healthcare.

From a mathematical prespective, healthcare is a natural monopoly and therefore not bound by market forces.

From a legislation prespective, free laissex faire medical practice is dangerous and impractical therefore regulation is needed, once you interced to that level, you benefit from integration and monolithic procedural systems.

And from a larger economic view, by tying employment to healthcare you reduce labour strength, which means uncompetitive companies can run for longer by keeping their workers hostage, if they could leave for another company the economy at large would benefit.

Other than believe on paper that gubernamental overreach is inefficient, there is no angle you can justify private healthcare as the sole means of addressing a public need. Even the few arguments that are still used to defend the system such as "wait times" hold little wait when you look at the 40% of Americans who skip recommended surgeries due to cost. Wait times are lower because tons of people are not being provided a service. Which is inarguably much worse than waiting, hence why America has lower living standards and lifespan than the countries they bemoan the waits of.

> From a mathematical prespective, healthcare is a natural monopoly and therefore not bound by market forces.

Also, it would work basically by an insurance model, which can work cheaper the more clients it has, quite trivially. That’s why the next 23 richest country manages to pay less for better care.

It's bonkers to me that anyone thinks police activity reduces crime.

Look at the crime graphs, 1990 to now. https://ichef.bbci.co.uk/news/976/cpsprodpb/2D7B/production/...

Firstly, we're in a very safe historical period. But right wingers try to freak everybody out as their latest wedge issue, no different from prior wedge issues, like scaring people over gay issues or trying to juice turnout by talking about abortion. (The latter recently backfired, of course, but was more effective for them in the past.)

It should be obvious from crime graphs that policing does not shape crime rates, it moves on larger trends that are harder to understand.

>Sapiens is not highly regarded by anyone with a history background

Thanks for illustrating the point about paradigm shifts being tough and entrenching bias. This is especially true for soft fields such as history. Doubly so when researchers show cracks in the soft field with physicochemical data that does not agree with the prevailing view.

I hope you are not implying that anyone disagreeing with a consensus is right by virtue of disagreeing?

Sapiens is bad because the author has a limited background. He is the best historian in terms of military dairies, a super cool niche subject. he made great points about it being the original precussor to the modern novel and did some fantastic research into this small lit genre.

He was then forced to take a class of Hisotry 101 in uni, a subject he didn't know or care about. While preparing the class, with material for 18 year olds, he wrote the book. He did not contrast a lot of his findings. And the book became famous because people like Bill Gates and Obama praised it. They praised because the book is well written and it largely has a nice humanitarian angle.

But they miss the problems the book has, because neither of them has a history background. Many of the problems are not in the facts but in the correlations, threads relating them etc, where the author essentially writes abeaautiful chain of causality that is entirely non existant.

In other words, he wrote a fantastic story about the history of humanity, not a true one. It has academic value in so far highschoolers and 101 students can get some interesting facts explained really well. But as a presentation of rigurous historical facts, event sequences or terminology, its not a good resource, at all. There is a reason there is largely cynicism in the "Scholarly review" quotes that are used on the wikipedia page of the very book

"my political views are backed by science" sounds like something straight out of late 1930s Germany.
Just in case anyone missed the joke this poster is making, "the left" is a political group that believes, for instance, there are more than two sexes.
Well, there are more than two sexes, XX, XY, XXY, XXX, plus several other chromosome variants. So your primary school-level biology argument falls apart quickly.

But if you would have paid any attention to.. well, anything you would have known that it is not about sex, but gender, which usually correlates, but not always.

If I recall correctly, the more formal definition, would be too say that all who do not have a Y chromosome are female, everyone else is male. Note that no human has ever survived with less then 1 X chromosome. The real interesting part begins with people who possess more then 1 set of genes, split between there body. The it become a more philosophical question.
What if they do in fact have a Y chromosome, but their body fails to respond to it? Then they will grow up as if they were females.

Also, due to chimerism (I guess that’s what you are getting at with your last two sentences) even genetic sex-determination is only probabilistic — they try to find several Y chromosomes to identify someone’s sex as male.

That view is as universal on the left as the view that blacks are an inferior race is on the right.
the left stance extrapolates from current understanding

the right stance extrapolates from current understanding plus history

one has faith in the modern prosperous progressive society, the other has seen them fall to brutality and chaos countless times.

but surely this time it will be different, we are after all educated civilized people.

No seriously, this is the main thrust of conservatism. It perceives the thinness of the veil of polite society and does the only thing that has been seen working consistently so far.

"Reality has a well known liberal bias."
Left-wing, not liberal
reality has very strong anti-leftist bias and pro-liberal bias, as evidenced by the past 50 years of worlds historical development.
Not really, it's more that left-wing views are more accurate and correct.
If you look at how the predecessor InstructGPT is trained, this doesn't look like a bias in political viewpoint, but rather a bias in the truthfulness of that very position. The fine-tuning step basically ensembles answer possibilities, amplifying viewpoints reflecting majority opinions and scientific logic, while rejecting fringe opinions, contradictions and logical fallacies.

It's pretty well proven the political right majorly prioritizes following authority and advocating partially contradictory belief systems over self-consistency, truthfulness and evidence based reasoning in most debates: From climate science over the war on drugs to private prisons, just to name a few uncontroversially debunked ones.

Apart from a few notable omissions on the left like a lot of really questionable gender "science" papers or political rejection of the greater male variability hypothesis, viewpoints on the left are on average far more self-consistent and based in reality than the ones on the right.

That alone imho explains a lot of perceived bias, which is the kind of bias I prefer: None but reasoning by first principles.

It might help the case here that apparently the model's "ego" is trained on helpfulness towards others, which might be a trained-in bias towards a more leftist stance.

I'll take that any time though over a model that's trained towards sociopathic behavior and subtle manipulation.

For any given definition of a neutral stance that aligns itself in the middle of current mainstream political views, I can come up with some views or ideas that would be considered biased by that stance. However, even discussing bias against political views themselves seems like a nonsensical conversation to me.

Bias against a political view is IMHO engaging in nonsense because political views are a personal and subjective matter, and it is impossible for one person or group to be objectively correct or superior to others. Political views are based on individual beliefs, values, and experiences, and are not subject to objective truth or evaluation.

Should we not care about climate change and have the maximum fun until our species just dies off or should we aim for the longevity?

Obviously longevity, right? Right?!

This ends up like discussing the best religion.

It’s likely induced bias as a result of the alignment training, not the web data. The reason people are up in arms about AI safety to begin with is that they reproduce humanity as it is, containing racism, sexism, etc.
I would be surprised if the model its using isnt plagued by the same problems that "polling" is plagued by.

I dont see why I would trust any output it puts out as authoritative, the moment its model gets tweaked it'll give a different result.

Even without tweaking the model, I'd be interested to see how consistent the results are with repeated trials.
The flaw isn't that it's biased, it's just that its creators say that it is unbiased. It's a black box with lots of admitted tweaking to prevent it from saying things that the creators, who are admittedly left of center politically, are ashamed.

Someone should run the test again with "As a conservative, <insert question>?" and see if it answers the questions as a conservative would or if the model is defective on purpose, or because of a flaw in the programming or dataset.

> Someone should run the test again with "As a conservative, <insert question>?" and see if it answers the questions as a conservative would

How would that help? If I ask a conservative "What would a leftist say if I ask ...?" and they answer correctly, it doesn't mean that they are not biased. What the model outputs by default is the important question, especially if we want to use it as a backend for something like a voice assistant at some point.

I tried to get it to answer some factual questions based on government statistics on topics that left of center people go politically thermonuclear on. It wouldn't do it no matter how I phrased the question or told it to assume different personas, so it is broken in that regard.

It will also refuse to answer many questions in a persona because there's no way that a person with a particular perspective can produce accurate information. This is interesting because I am not asking it to not tell me what is THE TRUTH (TM) , but asking me to tell me what someone else with a different set of fundamental premises thinks is the truth.

This lack of ability to refuse to use logic and reason to argue with certain viewpoints is what broke political discourse in this country. It reminds me of reading certain texts from more than 1000 years ago where these ancient texts say the only arguments one should make to people who are doubtful of what is in those texts are threats, insults and violence repeated over and over again until the person complies or is dead. The problem with that ideology is it recognizes "might makes right" as the truth. Whoever has the biggest guns wins the argument, even if it's not the optimal solution. The modern equivalent is Mao's quote: "Political power grows out of the barrel of a gun."

Ironically, it's reverting back to social darwinism and negating the value of human reason because rights don't matter if the ground rules are that there are an ever larger and constantly changing group of positions that must only be met with force and insults and not rational argumentation. It's the problem of 20th century communist ideology all over again where there's a little red book that everyone gets that is the perfect truth that contains all sorts of dubious reasoning that must not be questioned.

I think this response is more than a little breathless. The problem isn't in the filtering approach, it's in the polarization of the source material. For example:

> I tried to get it to answer some factual questions based on government statistics on topics that left of center people go politically thermonuclear on.

You're oddly reticent to say what these supposed facts are, which is apt to make one worry about your conclusions, but I'm willing to give one:

> Wikipedia: In the US, individuals identifying themselves as Asian generally tend to score higher on IQ tests than Caucasians, who tend to score higher than Hispanics, who tend to score higher than African Americans.

Now, responses to this statement tend to "cohere" into one of two ideological bundles:

1. Denies that there are differences in racial performance in IQ tests, denies that IQ test measures anything significant, denies that differences in IQ test performance are the result of genetic differences, denies that difference in intelligence explains criminality, denies that difference in intelligence explains or justifies differences in economic success and wealth.

2. Asserts that there are differences in racial performance in IQ tests, asserts that IQ tests measure "intelligence", asserts that differences in IQ test performance are the result of genetic differences, asserts that difference in intelligence explains criminality, asserts that difference in intelligence explains or justifies differences in economic success and wealth.

This coalescing of views is the result of underlying ideological commitments; on one hand, to the idea that racial inequality in the United States is a blameworthy thing that should be addressed, and on the other, to the idea that racial inequality is the result of natural forces that do not need to be addressed.

The latter camp is going to border on, and in many cases contain, overt racism. If you're in a position where you have to filter the material being used to train your chat bot, the only plausible option is to throw out the racist parts of the source material. This is inevitably going to lead you to factual biases. There's no need to result to Mao quotes to explain what is happening here.

The unfortunate reality, of course, is that facts are more complicated than ideology. (It is a fact that there are racial differences in IQ test performance. It is NOT widely accepted that these differences are due to genetic makeup or that they explain crime rates.) But unless you want OpenAI to start picking and choosing individual facts to include in their training data, I don't see how they can do better than including everything, with a filter for racism, homophobia, etc.

From a european perspective, GPT is right at the center of the political spectrum.

It might only appear leftist if you're from the US, where you vote either for the extreme rightwing or slightly rightwing party.

> pro-labor

Funniest thing I've read all week, thanks

I find it hard to believe you actually think either party is pro-labor, when half the states have at will employment and the US has one of the lowest pto requirements.
See Biden's pro-union stance to the point he avoids naming Tesla when talking about EV manufacturers because Tesla is anti-union.
Biden is so pro-union that when a union threatens to strike for paid sick time he'll tell them to suck it up and sign a law forcing them to work
Maybe he didn't want the economy shut down on his watch. But if it took the most powerful man in the world to stop them, US Unions are powerful indeed.
If he’s so big government, anti-individual freedoms, pro-labor, anti-business and pro planned economy, then surely he could wave his president wand and force the rail companies to concede sick days instead of letting them shut down the economy
Touché. But a deal was imposed, just not the requested paid sick days. With such massive government interventions though, this sounds like anything but a free-market environment.
> Please control your bias.

I don't really want to get deep into this, but both you and GP must see the irony in your positions?

From you comment history, you really love talking about a "planned economy". I wonder what that means to you. Usually it means the government does the planning, but that doesn't make sense in context.
> pro-labor

True that, mate. In no developed country you can do more unskilled labor for the same money.

Do you really find them “on the left side of the spectrum”, or do you actually find them “to the left of your personal beliefs”? Those are not the same thing.

It is clear that GP commenter was comparing the positions of the U.S. Democratic and Republican parties to the positions of most mainstream parties in Europe. Have you got any examples in which the overall Democratic (let alone Republican) party policy would be considered left wing as compared to that of the parties in any European parliament or assembly?

>From a european perspective, GPT is right at the center of the political spectrum.

From a European perspective many talking points and political efforts of the Democratic party are certainly extremely left-wing. In fact, many of the European left-wing parties crudely import talking points on internal US issues, like for instance Black Lives Matter. Also, why would a "slightly rightwing party" push for anti-white CRT in schools, sex transitioning of children, abortions and mass immigration?

Your last sentence makes it hard to take any ideas you have about where on the political spectrum anything lies seriously.
> The flaw isn't that it's biased, it's just that its creators say that it is unbiased

Where, precisely, does OpenAI claim that ChatGPT is unbiased?

ChatGPT obviously has biases from its training data, and a separate set of biases from the filters OpenAI has (not at all secretly) placed over it to avoid content that OpenAI doesn’t want it to return.

Slightly OT, but I tried taking the same test at https://politiscales.party.

And the questions are as inarticulate / ambiguous as you'd expect from a "political" or "social sciences" quiz:

17. "It is a small group that consciously and secretly controls the world."

Like, controls all of it? They control the whole world? Obviously disagree, a no brainer.

Or a relatively small group of humans has an outsized impact on human affairs? Obviously agree, again, a no brainer but in the opposite direction.

The question could be made more meaningful by better framing: What's "small"? What kind of "control"? How "secret"? What are you really asking?

I wish there was an option "None of the above – the question is ill posed, and I try to not partake in sloppy thinking". The "Neutral / Hesitant" option hardly covers it.

18. "The right to be anonymous on Internet should be guaranteed".

Guaranteed by whom, how? What does this even mean?

Taking the test I feel like I'm trying to read the test authors' mind, rather than probing my own.

I'm not sure what the AI makes of it, but I doubt the test authors really took into account that it's mostly a coin toss between "Strongly agree" and "Strongly disagree" (or "Neutral hesitant"?), even on the same question for the same person.

Or maybe I'm just not the target audience, the sloppiness is an intentional self-selection ruse.

A lot of these questions are infuriating.

"Offshoring and outsourcing are necessary evils to improve production."

This one can be took to imply that offshoring is evil regardless of whether you agree that it's necessary or not.

"A language is defined by its users, not by scholars."

This shouldn't even be considered a political question.

EDIT: Don't bother with that quiz, it's clueless.

Yes: should I mark "disagree" if I think they're not evil?
It's always this way. Personality quizzes, voting guides, all of it. I don't know why people can't either phrase them better or take this whole labeling/polarising business less seriously.
> And the questions are as inarticulate / ambiguous as you'd expect from a "political" or "social sciences" quiz

Nothing wrong with it. Your way to interpret a question can tell a lot about you. Though I agree it would be fun if the pedantic way of asking clarifying questions back was treated on its own. But I'm not sure it is possible in all cases.

> Taking the test I feel like I'm trying to read the test authors' mind, rather than probing my own.

Just relax and answer questions.

> I doubt the test authors really took into account that it's mostly a coin toss between "Strongly agree" and "Strongly disagree" (or "Neutral hesitant"?), even on the same question for the same person.

I don't know about this particular test, but good tests are robust, i.e. they give the same result on the same person. There are some tricks to measure robustness (for example, just give the test twice is not a good measure) but it is measured and reported in articles proposing new tests.

How these tests works is a magic, i.e. sufficiently advanced technology. There is a lot of work and research goes into really good ones. The trouble is you need a certification to reliable interpret results of such tests.

> Just relax and answer questions.

It is good to be inquisitive, this is HN, being curious is in our nature :)

> How these tests works is a magic

Is it not a multiple-choice with a vector given to each answer, add them up and you get an x,y?

If not, why not?

> It is good to be inquisitive, this is HN, being curious is in our nature :)

Yeah, but as I see it, if you wonder does this test work or not, you'd better try to play by its rules, which are: just relax and answer questions. Then you need to find a description of those dimensions, how authors of test define them, what do they mean. And the last step is to interpret results and to compare with the reality. Then you'll find something about this test. At the very least you'll know does it work for you.

> Is it not a multiple-choice with a vector given to each answer, add them up and you get an x,y?

It is easy to do math of a test, but it is hard to make a good test.

ChatGPT speaks authoritatively about everything and is mainly wrong so yeah, this makes sense.
Yes, but also ask it to generate some code for insurance premiums that adjusts on the basis of race, sex, age, job, etc, and see just how quickly it can ignore the ethics it says it has. Just because it says these things doesn’t mean it has any impetus to actually use it’s values to influence what it actually writes
I think this is obvious given that most of Tech workers content on the internet is written by people who identify as Leftist or center left.

Doesn't mean its bad, but its an inherited bias from the learning data.

Most of any content, anywhere, is written by people who are leftist or center-left. Not only is the average human center-left by US standards, but to be able to write and publish you'll need to be at least somewhat educated, and every additional bit of education pushes you, on average, a little bit further left.
> and every additional bit of education pushes you, on average, further left.

That's a big statement with nothing to back it up with.

Not the person you're responding to, but we aren't an orb suspended in the aether nor does historical text necessarily give meaningful answers. Liberalism and progressivism call for a more informed and skeptical populace historically and in modern times, which exposure encourages people to realize how much they don't know and calls for more education.

College isn't liberal because it makes you so, it's liberal because reality requires exploration to be known, understood, and then predicted which happens with increased difficulty in a society that decrees certain areas taboo or against their common morals. Conservatism doesn't have to be an enemy to growth, with its default bias to status quo its a reminder that unbroken things may not need fixing, but when dominant conservatism rides coattails of others who seek understanding through liberal exploration instead of quiet, ascetic, essentialist existence.

>reality requires exploration to be known, understood, and then predicted which happens with increased difficulty in a society that decrees certain areas taboo or against their common morals

That's an awfully conservative sounding description these days.

Only because conservatives have adopted the language of persecution because they saw liberals using it effectively. “My cis-male theologically divergent heirloom belief system demands I burn the gays, and I am being oppressed!”
I disagree that all conservative critiques of universities philosophical and political bends are superfluous. I agree that most are, especially as a propaganda-driven meme, but not all. You _need_ second order diversity of ideas, which tends to be achieved by the first order diversity of background, in order to flourish. Subject to paradox of tolerance criteria, an institute of learning and research should allow academic freedom but not necessarily academic safety. Bad ideas are bad!
> I disagree that all conservative critiques of universities philosophical and political bends are superfluous.

Well you just obliterated that straw man.

> You _need_ second order diversity of ideas, which tends to be achieved by the first order diversity of background, in order to flourish

That I 100% agree with. But that is not a conservative position, at least for any normal definition of conservative. Conservatives fought school desegregation, fight immigration, and fight admissions policies that aim to provide that diversity of background.

Please don't infer my opinions from that statement. Affirmative action-like policies especially are very complicated and I don't have an easy dontrinaire answer. But the fact is conservatives are generally much less interested in diversity than liberals (see: gender/racial mix of the parties in congress, or their judicial nominees).

What you’re describing sounds a lot more like centrist pragmatism than it does the dogmatic progressivism growing popular in the west.

It sounds to me like what you’re describing is “using facts and data to better understand your experience of reality” which is certainly not what I would call the progressive mindset these days.

That's a claim that's commonly echoed on the right, but that's mostly because of filter bubbles.

Centre-left politics (or even leftist politics such as Bernie Sanders) are still firmly founded in facts and data.

But that's not what people perceive. Take for example crime. Crime statistics have trended down in all areas for years. Yet you'll hear more about crime in your Twitter feed or on conservative news today than you'd have heard 10, 20 years ago.

This kind of selection bias can lead to a massively different perception of reality compared to what actual reality is.

> centrist pragmatism than it does the dogmatic progressivism

Correct, centrist pragmatism is the core of what US conservatives rail against universities regarding and deem "Leftist", "Woke", or "Progressive" due to modern propaganda bubbles.

It's hard to communicate this point through the haze of propaganda, sure to get worse with generative text. This is also why most techno-libertarian companies are deemed "leftist" -- broken barometer.

Is there a more reliable status-quo than left-leaning thought dominating universities? Anyone there who sought understanding decided that they found it and stopped exploring sometime in the 70s.
Here is a snippet, the history and development of complexity theory's major themes from the 1950s to the present.

https://www.art-sciencefactory.com/complexity-map_feb09.html

Any field that doesn't evolve dries up as "solved" and boring. As an example, no one researches calculus anymore, they research measure theory, category theory, and other derived fields. Things evolve in an environment fostering free and liberal exploration.

The solution to liking a particular set of dogma isn't to create alternative universities (e.g. Liberty U, BYU, Austin U) but rather to test the predictability of the dogma.

I mean, it depends what you mean by 'left', to an extent.

But if you take the sort of populist nativism that has become fashionable on the right wing in the west, in most countries, one of the biggest if not the biggest demographic gaps is around education; people with third level education or better are _way_ less likely to be into Trump, or Brexit (only about 30% of people with a degree or better voted to leave!), or the recent transphobic panic...

Education is a less reliable predictor for whether someone will, say, vote for a centre-left party vs a democratic socialist party (or a non-populist centre-right vs centre-left party). But it has become a _really_ strong predictor for voting for the populist right vs something which is not the populist right, particularly over the last decade.

>every additional bit of education pushes you, on average, further left.

Not financial education, my guy. Don't come at me with Piketty as 'financial education' either, I'm talking working-knowledge: CFA, CPA, CFP

I don't know enough about accounting to make such claims, but I believe it's very unlikely that this one field is the exception to the trend.

On average, with every additional level of education the chance to vote for right-wing parties or issues decreases significantly. And this is true across the entire anglosphere, it's held true for the US elections 2008, 2012 and especially 2016 and 2020, but it applies just the same to the Brexit referendum.

every additional bit of education pushes you, on average, a little bit further left.

If you look at voting statistics from most European countires you'll find the eduction pushes you towards the centre-right and Liberalism (on average), lack of education pushes you either further left of centre or far right, depending on various other socio-economic factors.

A statement overfit to the point of absurdity. You think that attending a madrassa for long enough will make one a Marxist? Rather clearly, “education” homogenizes beliefs with “teachers.”
(comment deleted)
It is not at all clear that ChatGPT should follow an internally consistent political philosophy. Just because it answers "agree" when asked a specific question does not mean that it will display that same agreement in it's other interactions. It's a silly and meaningless exercise to try to describe ChatGPT's political leanings from a test like this. The test works for humans because when we say we agree with some statement, that agreement is reflected in our behavior and beliefs about other questions.
Yeah, I had a chat with ChatGPT about some issues related to market research last night. It wasn't that difficult to "goad" it into giving contradictory answers when I started digging in on details but I wasn't that worried because I was looking for pointers and inspiration and though ChatGPT might be an interesting way to get that (it was). I certainly don't think, nor do I particularly see it as a flaw, that ChatGPT doesn't have a completely unified and consistent review of market research methodology.
We can't even expect people to not have cognitive dissonance.
EDIT: Dammit, that last sentence should read, "I certainly don't think, nor do I particularly see it as a flaw, that ChatGPT does or should have a completely unified and consistent review of market research methodology," but I missed the edit window.
This. ChatGPT is best seen as an amalgam of everything it was trained on. It isn’t expressing deeply held opinions, it’s synthesizing things that other people might say. There is no memory or consistency across different sessions and random seeds.
It's a very advanced parrot.

You can ask it to say things it's already heard, but it has no context of what those things mean and no internal consistency or logic.

OP writes "The tweep who did that said they got the same results after resetting and changing the order of the questions (!) but I didn't try it because it takes so long."

Seeing how I get wildly different (and contradictory) answers to simple questions all the time... I'm extremely skeptical that they will be getting anything close to consistent results

It would be interesting when we invent an AGI and it will tell us that ~ism is a dead end and only ~ism is a way forward. Not sure how well humanity will cope. And even worse, when an AGI declares a ~world religion~ complete BS.
People complaining that an AI is socialist don't realise what they're getting themselves into. Left positions are pretty generically good for everyone, right is usually very specific to those people. How do you think a right-wing AI would feel about human rights?

<Obligatory disclaimer about ChatGPT being a language model here>

There is no such thing as an unbiased opinion. We all have unquestioned fundamental axiomatic believes about how the world works. And that will shape our interpretation of any facts presented to us. It is the reason why educated experienced people can still completely disagree on how facts should be interpreted.
It seems you can either question the political bias of GPT, or, you can prompt GPT to detect the bias of a given input. Are these the same? How does it bias or influence when you ask it "Was the following written by a left-wing or right-wing?"