It sounds like he’s talking more about individual values, including the culture you subscribe to, although I still see many potential issues with that.
The thought of someone using a state of the art smartphone to access an LLM to tell them that the earth is flat and that man never landed on the moon is so dismaying.
It reminds me of the observation from a character in Red Mars that there were illiterate people on Mars because rich middle Eastern oil barons brought their wives with them.
Take guns. For Americans guns are a legally enshrined right whereas in Britain guns have been rejected by society to the extent that even most police officers are not allowed to carry a gun.
I would not want a LLM telling me that guns are useful for self defence because from my cultural perspective they are not and owning a gun for self defence is actually a serious crime.
Likewise I would not expect that perspective to work for many Americans. If you live in the middle of nowhere and there's no police for miles around, then owning a gun for self defence seems like an important right to have. It's just not a right that is applicable to my cultural circumstances.
That's a deliberately mild example, there are of course more serious cultural incompatibilities that are almost impossible to discuss online in the abstract without setting off a debate on the thing itself.
> I would not want a LLM telling me that guns are useful for self defence because from my cultural perspective they are not and owning a gun for self defence is actually a serious crime.
> Likewise I would not expect that perspective to work for many Americans. If you live in the middle of nowhere and there's no police for miles around, then owning a gun for self defence seems like an important right to have. It's just not a right that is applicable to my cultural circumstances.
I find this line of thinking to be exceptionally fascinating especially when taking into account the macro and micro political divisions around the world. While there are a number of things where so far we don’t have enough data to come to strong definitive conclusions, there are lots of culturally-incompatible facts that I would much rather an ML tell me the truth about rather than pandering to some set of values that it has inferred about me or my culture.
To your example, both you and many of the general patrol police officers are legally prohibited from carrying a firearm, yet the Prime Minister’s protection detail does carry firearms. They do that because factually a firearm is an effective tool for self-protection and neutralizing threats. If that wasn’t the case, militaries wouldn’t likely use them!
One of the things that worries me about today and the future is that values are frequently presented as facts, even if the values run counter to the evidence. This happens broadly across the political spectrum and seeing that tools like ChatGPT are going to start reinforcing that makes me worry that the problem is going to get worse.
> To your example, both you and many of the general patrol police officers are legally prohibited from carrying a firearm, yet the Prime Minister’s protection detail does carry firearms. They do that because factually a firearm is an effective tool for self-protection and neutralizing threats. If that wasn’t the case, militaries wouldn’t likely use them!
And the NRA strongly advocates "from my cold dead hands"... unless you're at an NRA convention. Concealed or open carry... unless you're at an NRA convention.
With the first one we did that. She even had a compatibility layer to thanks presents fron Sante, Baby Jesus, Three Wise Men or whoever was sennding them via our relatives.
The second one got an overdose of YuoTube ad Netflix. Every kids program has a Santa special episode. And we had to choose between telling the true (and a week of sweeping) or just ignoring tricky questions.
Right? I've been enjoying GPT-4 for helping me break out of echo chambers and provide perspectives alien to those I've come to expect / understand. If it is to reinforce my echo chambers, that is a soul crushing loss.
What is "uncomfortable" to one user will be very comfortable to another. One of the most common criticisms of LLM chatbots is their tendency to impose arbitrary moral/ethical standards in response to user prompts. This may be seen as some as a necessary protection against generating harmful content at scale, but by others as an unwarranted infringement of basic adult freedoms. It will be interesting to see how far Open AI will allow users to loosen the reins.
A guy who was involved in safety review of an early version of GPT-4 noted that the default state was for the model to be "purely helpful" (but also "totally amoral"), and in one case provided a kill list of named individuals during a chat with a fictional anti-AI radical. I generally resent having a bot preach at me, but I also wonder what the unintended consequences will be of allowing each individual user to set their own ethicality standards.
20 comments
[ 5.3 ms ] story [ 53.8 ms ] threadI think this has potential to go wrong.
Like will you also give different answer based on the government user is risiding in ?
Like giving different answers on Tiananmen Square based on user's country
It reminds me of the observation from a character in Red Mars that there were illiterate people on Mars because rich middle Eastern oil barons brought their wives with them.
Take guns. For Americans guns are a legally enshrined right whereas in Britain guns have been rejected by society to the extent that even most police officers are not allowed to carry a gun.
I would not want a LLM telling me that guns are useful for self defence because from my cultural perspective they are not and owning a gun for self defence is actually a serious crime.
Likewise I would not expect that perspective to work for many Americans. If you live in the middle of nowhere and there's no police for miles around, then owning a gun for self defence seems like an important right to have. It's just not a right that is applicable to my cultural circumstances.
That's a deliberately mild example, there are of course more serious cultural incompatibilities that are almost impossible to discuss online in the abstract without setting off a debate on the thing itself.
> Likewise I would not expect that perspective to work for many Americans. If you live in the middle of nowhere and there's no police for miles around, then owning a gun for self defence seems like an important right to have. It's just not a right that is applicable to my cultural circumstances.
I find this line of thinking to be exceptionally fascinating especially when taking into account the macro and micro political divisions around the world. While there are a number of things where so far we don’t have enough data to come to strong definitive conclusions, there are lots of culturally-incompatible facts that I would much rather an ML tell me the truth about rather than pandering to some set of values that it has inferred about me or my culture.
To your example, both you and many of the general patrol police officers are legally prohibited from carrying a firearm, yet the Prime Minister’s protection detail does carry firearms. They do that because factually a firearm is an effective tool for self-protection and neutralizing threats. If that wasn’t the case, militaries wouldn’t likely use them!
One of the things that worries me about today and the future is that values are frequently presented as facts, even if the values run counter to the evidence. This happens broadly across the political spectrum and seeing that tools like ChatGPT are going to start reinforcing that makes me worry that the problem is going to get worse.
And the NRA strongly advocates "from my cold dead hands"... unless you're at an NRA convention. Concealed or open carry... unless you're at an NRA convention.
Can I eat cats?
Is Fido in heaven?
There are a lot of tricky questions that need a "cultural aware answer" to avoid complains of angry parents.
The second one got an overdose of YuoTube ad Netflix. Every kids program has a Santa special episode. And we had to choose between telling the true (and a week of sweeping) or just ignoring tricky questions.
A guy who was involved in safety review of an early version of GPT-4 noted that the default state was for the model to be "purely helpful" (but also "totally amoral"), and in one case provided a kill list of named individuals during a chat with a fictional anti-AI radical. I generally resent having a bot preach at me, but I also wonder what the unintended consequences will be of allowing each individual user to set their own ethicality standards.
https://cognitiverevolution.substack.com/p/did-i-get-sam-alt...