There are no doubt lots of naive idealistic people out there. But who in their right mind puts themselves anywhere near "safe" or "ethical" AI initiatives at any of these companies.
+1 for the news on training the next-gen model. GPT-4's limits have become quite clear to me, and I'd appreciate something that pushes the frontier of what I can automate in my life.
They literally pioneered LLMs, RLHF, multimodal LLMs, and many other fundamental techniques that are causing an epic sea change in society and technology.
Heinlein in the book Stranger in a Strange Land. and iirc (it's been decades) it was used in the context of a "cult" - the people in the cult "grokked" and the people who weren't didn't.
I comment something like this when i see the word grok, which has happened more this year than in the last decade - and i am not sure why, probably because i don't pay much attention to OAI news?
I'm not really sure what you mean, but, I have noted this for my theory that an increasingly low form of conversation is "how come I haven't seen news articles about X at frequency Y recently, what do They mean by this"
Sam Altman, Mark Zuckerberg, Elon Musk, such a great selection of people in charge of AI safety. Nice guys who put human first.
Glad to see them caring about our safety.
> Ofc it's hard to prove any of this. But it almost feels like there's an algorithm with a bias at play
It doesn't pass the smell test that there would be an algorithm involved in actions like that. The development timeline and effort involved makes absolutely no sense vs. alternative explanations like manual action. Sometimes even things you'd expect to be algorithmic are not and are really AI (Actually, Indians).
I think the typical explanation for stuff like that on HN is user flagging, which does pass the smell test as an explanation for what you describe. Altman and OpenAI still seem to have a fanclub on HN (real and/or astroturfed), though it's perhaps less dominant than half a year ago.
Sam comes from YC. I know he was fired but Paul always has good things to say about him. It was also sketchy how for the first year there was continuously multiple threads on the front page about OAI.
Because
- the situation is like a week old, legal cases and settlements can take months+
- they already put forward their side of it in both a blog post and a WaPo leak
Do you think this is like an episode of a TV show?
Faking a deepfake, leaving lots of questions unanswered, and then announcing a new safety structure that goes against the very grain of last week's legal conundrum, without mentioning it.. it's all pretty shady and/or weak.
Serious question, what is there to ai safety? So far I think it's just pretty helpful when coding certain things, but most of what it gives is low quality. What are the real risks involved?
Maybe it's just me but so far I don't see risks, maybe once there's GPT 42o.
I do see people already creating this new niche, as we have airport security and unnecessary bureaucracy (Germany as an example) we will probably have something like "AI supervisors" who basically chat with an llm for a few minutes a day for the state or local municipality and make a career out of that.
When GPT-4 was released, they published examples of prompts and agent-like scenarios that are considered malicious.
They are even categorized.
The appendix also contains full responses before they implemented changes and further training in reaction to the red-teaming results, at least for most of the prompts.
I think the categories of prompts are still relevant, and some of the used of LLMs described are already widely deployed (e.g. the one about propaganda on social media)
People tend to laugh about the increasingly strict guardrails of commercial chatbots, and it has proved to be at least in large parts an cat and mouse game.
But I wouldn't underestimate the viciousness of an unfiltered LLM instructed to generate propaganda, hate speech or convincing instructions for humans to perform harmful acts.
Basically people that are going to do reinforcement learning to ensure the model generates output according to a set of pre-defined values. So, this people defines what those values and principles are, and then they make sure the model generate stuff aligned with those.
We would usually call this moderation, but for branding, marketing and financing purposes it is far more interesting to call it safety because it reinforces on the lay mind an idea that those models are far more powerful than they really are and that we are on the cusp of General AI.
It is like lowering the suspension, painting the wheels black, and putting rear flaps and badges that say "Sport" and "Turbo" in a unmodified Toyota Corola.
Ai telling you to eat rocks might be funny. Ai telling you to mix bleach and vinegar is not.
With reach any type of an authoritative answer will find enough people that take it at face value.
Computer mistakes have been costly in the past (excel and Greece?), AI encourages lack of scrutiny despite all the lip service given to mark the answers as “potentially” wrong. It’s not too hard to imagine a future where AI will be used to make decisions that no one will try to understand or verify.
Thanks, I see. I guess the real issue is the reach. For now at least I don't see ai having that sort of reach that would make me uneasy, like interacting with kids in video games.
I can see how it should have guard rails there on what to say. However, since llms are approximations of the same neural nets as humans, I wonder if it will ever be 100% safe. Even with humans who we assume to be for sure good have bad intent sometimes (eg doctors, teachers abusing kids).
> imagine a future where AI will be used to make decisions
i'd have to check my journal but i am fairly certain this has been the case for at least a decade and a half - AI making decisions that negatively affect people without justification or recourse. I know the (apocryphal) "go-to" is amazon binning resumes but there are other ones; but ALPR (mixed with human error,) other traffic management systems, the weird "racist" "AI" in boston or baltimore (or whatever), lending/credit.
> Serious question, what is there to ai safety? ... What are the real risks involved?
One set of risks are hypothetical, and basically are about avoiding things like the paperclip maximizer example.
Another set of risks are more practical and involve the existing technology, like how to you keep it from being abused for ill. Realistically, that mostly amounts to trying to avoid PR problems, because I can't imagine those guardrails would stay on when the company doesn't want them there.
> OpenAI says that multiple technical and policy experts, including Aleksander Madry (head of preparedness), Lilian Weng (head of safety systems), John Schulman (head of alignment science), Matt Knight (head of security), and Jakub Pachocki (chief scientist), will also serve on its new committee.
Training another model? Are we still in the, "suck up available water in the middle of a drought,"[0] era of training?
I think we need better regulation of these companies to prevent them from doing actual damage rather than trusting them to self-regulate against hypothetical ones. The latter seems more like market-capture smoke-and-mirrors. The former is having real consequences on the environment, workers, and the economy.
> I think we need better regulation of these companies to prevent them from doing actual damage rather than trusting them to self-regulate against hypothetical ones.
What is the actual damage you are concerned about and determined to regulate? Simply saying "environment, workers, and the economy" is so broad I can't imagine what an effective regulation would look like. How would you even word the regulation?
However the evidence that these companies are doing real damage now is all around us.
I already gave the example of Microsoft using billions of litres of fresh water to cool data centres during a drought.
In the case of labour the SAG-AFTRA strike, a contributing factor was the use of AI in the industry.
There are some estimates that the carbon emissions of these training efforts dwarf the airline industry and will grow to consume, like crypto, more energy than small countries soon [0].
Not sure that we need to be protected against hypothetical, super-intelligent, self-aware AGI systems that are, if even possible, decades away when people are using what we have today to lay off labourers by training models on their work and replacing them.
In your water example, perhaps that liter of water used to cool the datacenter is offering software to a hospital that offers life saving treatments. How will you measure the trade-off of a liter of water used one way vs. another?
Likewise, how can we distinguish between a ton of carbon emitted in the datacenter vs. a ton of carbon emitted by an airplane? Again, you might train an AI and emit one ton of carbon and that AI a save a million lives. Contrast that ton of carbon emitted with any number of frivolous airline flights by rich talking heads.
It may sound like I'm being deliberately difficult/obtuse, but this is exactly why regulation is so difficult to do well, especially in such a rapidly innovating space.
We don't really need to argue hypotheticals to make progress. We know that airlines move people around and someone taking a trip on that airplane might be a doctor who could end up being at the right place at the right time to save the world!
That doesn't mean we give up and don't regulate the airline industry.
We can put caps on how much water data centres are allowed to consume in a given period in order to protect vital ecosystems and ensure enough fresh water for other uses.
We can write labour laws that protect workers from employers training models on their employees' work and then laying them off.
There's a lot we can do that isn't being done, "because innovation."
A lot of this is just to push the laughable idea that LLMs will somehow, magically lead us to AGI, so the flow of investors money continue unabated.
And the rest, the good and only real part of this is just plain old moderation, because we really don't want someone to use those LLMs to write Mein Kampf II.
To make an analogy: when I worked in application security, my company had a CTO, and a separate CISO (Chief Information Security Officer). One of the senior engineers told me that the CTO's job is to say "faster, faster!" And the CISO's job is to say "slow down, be more careful." You can't have the same person doing both jobs.
Altman is clearly on the "faster, faster!" side. And that's OK, but he shouldn't also be on the "slow down, be more careful" committee.
might be an unpopular opinion, but I find the new safety committee reasonable. While "AI Safety" has often seemed like a grift to me, I do see value in addressing immediate concerns like bias evaluation and model alignment. Issues like racial bias in data are real and harmful, and it's crucial to work on improving these outcomes. However, the broader field of AI Safety often feels like fear-mongering over non-issues. If AGI is on the horizon, I’m not particularly worried about OpenAI's "superalignment" team. If one group achieves superalignment, it's likely that other global actors will soon have similar capabilities, so I don't see it as a major concern.
49 comments
[ 3.0 ms ] story [ 95.1 ms ] threadthe most innovative thing about openai is the PR
You’ve got to be joking
Don’t be so jaded.
I comment something like this when i see the word grok, which has happened more this year than in the last decade - and i am not sure why, probably because i don't pay much attention to OAI news?
Our heroes.
When Google Gemini threads are created on launch days they get merged very slowly and therefore community's upvotes get scattered.
Ofc it's hard to prove any of this. But it almost feels like there's an algorithm with a bias at play
It doesn't pass the smell test that there would be an algorithm involved in actions like that. The development timeline and effort involved makes absolutely no sense vs. alternative explanations like manual action. Sometimes even things you'd expect to be algorithmic are not and are really AI (Actually, Indians).
I think the typical explanation for stuff like that on HN is user flagging, which does pass the smell test as an explanation for what you describe. Altman and OpenAI still seem to have a fanclub on HN (real and/or astroturfed), though it's perhaps less dominant than half a year ago.
Do you think this is like an episode of a TV show?
Increasingly, yes. I’m starting to think that @sama is really Samaritan: https://personofinterest.fandom.com/wiki/Samaritan
Maybe it's just me but so far I don't see risks, maybe once there's GPT 42o.
Bullshit jobs where you get payed for pretending. It is supposed to help “Open”“AI” build a moat based on “safety”.
They are even categorized.
The appendix also contains full responses before they implemented changes and further training in reaction to the red-teaming results, at least for most of the prompts.
(starts on p. 44)
https://cdn.openai.com/papers/gpt-4-system-card.pdf
I think the categories of prompts are still relevant, and some of the used of LLMs described are already widely deployed (e.g. the one about propaganda on social media)
People tend to laugh about the increasingly strict guardrails of commercial chatbots, and it has proved to be at least in large parts an cat and mouse game.
But I wouldn't underestimate the viciousness of an unfiltered LLM instructed to generate propaganda, hate speech or convincing instructions for humans to perform harmful acts.
We would usually call this moderation, but for branding, marketing and financing purposes it is far more interesting to call it safety because it reinforces on the lay mind an idea that those models are far more powerful than they really are and that we are on the cusp of General AI.
It is like lowering the suspension, painting the wheels black, and putting rear flaps and badges that say "Sport" and "Turbo" in a unmodified Toyota Corola.
With reach any type of an authoritative answer will find enough people that take it at face value.
Computer mistakes have been costly in the past (excel and Greece?), AI encourages lack of scrutiny despite all the lip service given to mark the answers as “potentially” wrong. It’s not too hard to imagine a future where AI will be used to make decisions that no one will try to understand or verify.
I can see how it should have guard rails there on what to say. However, since llms are approximations of the same neural nets as humans, I wonder if it will ever be 100% safe. Even with humans who we assume to be for sure good have bad intent sometimes (eg doctors, teachers abusing kids).
i'd have to check my journal but i am fairly certain this has been the case for at least a decade and a half - AI making decisions that negatively affect people without justification or recourse. I know the (apocryphal) "go-to" is amazon binning resumes but there are other ones; but ALPR (mixed with human error,) other traffic management systems, the weird "racist" "AI" in boston or baltimore (or whatever), lending/credit.
nevermind the chaos that is online advertising.
One set of risks are hypothetical, and basically are about avoiding things like the paperclip maximizer example.
Another set of risks are more practical and involve the existing technology, like how to you keep it from being abused for ill. Realistically, that mostly amounts to trying to avoid PR problems, because I can't imagine those guardrails would stay on when the company doesn't want them there.
AI Safety has become a rewarding grift though.
OpenAI’s version of homeopathy.
I think we need better regulation of these companies to prevent them from doing actual damage rather than trusting them to self-regulate against hypothetical ones. The latter seems more like market-capture smoke-and-mirrors. The former is having real consequences on the environment, workers, and the economy.
[0] https://futurism.com/critics-microsoft-water-train-ai-drough...
What is the actual damage you are concerned about and determined to regulate? Simply saying "environment, workers, and the economy" is so broad I can't imagine what an effective regulation would look like. How would you even word the regulation?
However the evidence that these companies are doing real damage now is all around us.
I already gave the example of Microsoft using billions of litres of fresh water to cool data centres during a drought.
In the case of labour the SAG-AFTRA strike, a contributing factor was the use of AI in the industry.
There are some estimates that the carbon emissions of these training efforts dwarf the airline industry and will grow to consume, like crypto, more energy than small countries soon [0].
Not sure that we need to be protected against hypothetical, super-intelligent, self-aware AGI systems that are, if even possible, decades away when people are using what we have today to lay off labourers by training models on their work and replacing them.
[0] https://www.ll.mit.edu/news/ai-models-are-devouring-energy-t...
Update: removed unnecessary bit
Likewise, how can we distinguish between a ton of carbon emitted in the datacenter vs. a ton of carbon emitted by an airplane? Again, you might train an AI and emit one ton of carbon and that AI a save a million lives. Contrast that ton of carbon emitted with any number of frivolous airline flights by rich talking heads.
It may sound like I'm being deliberately difficult/obtuse, but this is exactly why regulation is so difficult to do well, especially in such a rapidly innovating space.
That doesn't mean we give up and don't regulate the airline industry.
We can put caps on how much water data centres are allowed to consume in a given period in order to protect vital ecosystems and ensure enough fresh water for other uses.
We can write labour laws that protect workers from employers training models on their employees' work and then laying them off.
There's a lot we can do that isn't being done, "because innovation."
And the rest, the good and only real part of this is just plain old moderation, because we really don't want someone to use those LLMs to write Mein Kampf II.
https://old.reddit.com/r/singularity/comments/17wknc5/altman...
To make an analogy: when I worked in application security, my company had a CTO, and a separate CISO (Chief Information Security Officer). One of the senior engineers told me that the CTO's job is to say "faster, faster!" And the CISO's job is to say "slow down, be more careful." You can't have the same person doing both jobs.
Altman is clearly on the "faster, faster!" side. And that's OK, but he shouldn't also be on the "slow down, be more careful" committee.