Different Levels of “AI Safety”

2 points by golol 3y ago ↗ HN

I think the term "AI Safety" is so overloaded right now that large amounts of conversation on the topic are essentially meaningless. The reason for this is that what AI Safety means is highly dependent on the level of intelligence your AI has. This point and the rest of this post may be obvious to most of the readers, but even then, someone still has to say it.

I would like to present a loose definition of 5 levels of AI models and the corresponding important concepts in AI safety/ethics. This is not meant to be complete or definite in any way. Since I believe in example over explanation, I will not try to give an exact definition but instead just keywords and examples.

Level 1: Everything before ~2021ish

Keywords: non-agentic models, image classification, low quality generative models, clustering, "big data" usage, autonomous driving, ...

New safety concerns: mostly bias and racism, misunderstanding and misuse of the model leading to bad performance, "black box", ...

Level 2: Current era ~2021-now

Keywords: High quality generative AI, realistic image/audio/video generation, video generation, text models like GPT, multimodal generatibe models, agentic models

New safety concerns: Fake content on a large scale, fake videos of politicians and events, fake posts online, bots spamming everything, massive scam or misinformation operations, generation of harmful contemt, some possible harm by misaligned agentic models (you tell an auto-gpt to book a plane flight but it somehow ends up making a bomb threat at the airport)

Level 3: near AGI and AGI

Keywords: Imagine a multi-modal model with human level intelligence, or very close. Immediate improvements you might get are the intelligence of many humans in a group (parallel execution) and that of a human being sped up (run program faster). But that is not necessarily superintelligence! Perhaps embodiment in androids is available.

New Safety concerns: Classic stuff from sci-fi novels (without superintelligence), "Laws of Robotics", murdering androids, anything evil a human might do on a possibly much larger scale. Furthermore: AI ethics, is this thing conscious? is this slavery? Should we create AI rights?

Level 4: Slow takeoff superintelligence

Keywords: Superintelligence that arises from self-improvement over the course of at least several months, it can not magically escape any constraints

New safety concerns: Monitor for such self improvement takeoff processes, global AI regulation, bombing data centers, development on isolated islands bla bla

Level 5: Fast take-off superintelligence

Keywords: Extinction

New safety concerns: Back to monke or don't bother

I have never seen any mathematical or eyperimental evidence that Level 4 and 5 are possible, yet it often dominates the discussion. Similarly people bringing up Level 1 are living in the 2010s. I think discussion should focus on Level 2 and some aspects of Level 3.

tikkun 3y ago ↗

Nice.

I'm glad that you're defining it. There's definitely an issue with the vague term "AI safety" that so many people use to mean different things.

nh23423fefe 3y ago ↗

Do you imagine 4/5 are impossible because you haven't seen some proof? What would constitute such a proof?

superintelligence and intelligence aren't different in type. only in magnitude. intelligence is bounded only by physics and energy.

recursive self improvement is just a tighter loop than natural selection, biological reproduction and cellular growth.

golol 3y ago ↗

I don't claim they are impossible (although I personally believe 5 is extremely unrealistic and 4 is difficult), I just have not seen any mathematical or experimental evidence. Experimental evidence would be an example of n accelerated self-imporvement process in a simpler model. Of course this is unlikely to happen, as one expect this kind of self-improvement to happen once a critical amount of intelligence is reached, and such examples should be impossible nowadays. Nevertheless, no experimental evidence. Mathematical evidence would be some kind of mathematical argument that suggests that a rapid growth of intelligence is plausible. Again, that is extremely high of me to expect as we don't really have any idea how to define these things mathematically. So I am setting the bar very high again.

Okay, so while there is no experimental or mathematical evidence for rapid self-improvement, one may say that this is unrealistic to expect. But given the absence of positive evidence, let's look at negative evidence. Every growth pattern we see in machine learning is either some form of decaying growth, logarithmic or sigmoid shaped. The rapid self-improvement of superintelligence would be the first accelerating performance curve we see...

>recursive self improvement is just a tighter loop than natural selection, biological reproduction and cellular growth.

Just because some process is self-improving, doesn't mean it will grow at increasing speed. It may just converge to an asymptote. Or grow logarithmically. Also, the claimed self-improvement is so different from evolution that it's not great to think of it as "evolution but faster" imo.

nh23423fefe 3y ago ↗

I can't tell if this is just a matter of different assumptions.

I think superintelligence is obvious even in principle. I would analogize with some technology like nano-tech. The tech-tree starts with fire, metallurgy, chemistry, atomic physics, solid-state electronics and will eventually get to https://en.wikipedia.org/wiki/Atomically_precise_manufacturi...

There is nothing "in principle" to stop this trend, and even more than that, there is no limitation when constructing arrangements of matter outside physics and knowledge. You could accuse me of technological determinism if you wanted.

superintelligence isnt something different than intelligence to me. Its just a transition in the substrate. Intelligence is the emergent capacity of a system to alter its environment to achieve goals. superintelligent systems are those with more capability. the main difference is the magnitude of the ability to correlate the system's internal model with external reality and then act on that model to produce low error predictions of the world state. the act-perceive-optimize loop of synthetic minds is unbounded in ways that biological minds are. artificial minds will build larger models of the world that correlate better with reality and they will search-think-tinker with their internal models to better achieve their goals.

What could stop this? Sure you can posit an asymphtote. But what if i said the singularity creates minds that are 300 IQ? Smarter than any person would ever be even if trillions of humans were born. What if the number is 3000 IQ?

I do think of it as evolution but faster because evolution isn't directed, its just global search optimization. AI will be driven toward capability, not optimal fitness.

golol 3y ago ↗

You know I agree that superintelligence is totally possible, but what I disagree with is that it follows immediately after AGI. Suppose you reach the level of AGI. Then your AGI can not just write down the weights for a smarter AI. It has roughly the intelligence of humans, so to build a better AI it would have to do something similar to what humans do: Large scale training on large computers . This takes time and is easily supervisable. I don't see a risk of rapid singularity like intelligence blow up. There is a risk of the AGI escaping but since it needs aignificant amounts of compute, and does mot have the intelligence to just rewrite itself into a decentralized abstract form, I again don't see a big danger. When we are at this stage we can start worrying about superintelligence, as well as AI rights and digital citzenship obviously.