I think this video starts out good, but then goes downhill quickly and makes lots of assumptions about the role of consciousness. The video uses the word aware/awareness a lot, but there's no good reason to assume that "aware" behavior necessitates consciousness. The role of consciousness is unclear, and it's also unclear how capable the brain is in the absence of consciousness. AI models could be trained to emulate awareness. A robot could be programmed to recognize itself in the mirror. Awareness does not imply consciousness, and we shouldn't assume that an animal's ability to understand its environment is because of consciousness.
There are certainly a lot of people who have given much thought about AI and consciousness, but how can we talk about sentient machines when we clearly do not understand consciousness in us, human beings[1]?
That's part of the reason to be wary about the possibility that our AI models may become conscious.
Most people happily dismiss the idea that a GPT-like model can be conscious, but the truth is that we don't know practically anything about how consciousness arises in living beings that are more similar to us and that we have been coexisting with for thousands of years. We don't even know how to tell if such a being is conscious or not (most would agree that a dog is conscious and a bacterium is not, but what about a mouse? A mosquito? A tardigrade? Where is the limit?). It takes a lot of hubris to make claims about if and when AI models can become conscious, with what we know right now.
He uses Commander Data from Star Trek as the example. Since we don't know whether it is something unique about our biology that leads to consciousness or consciousness is functional, then we can't determine whether Data is conscious. It also gets into the weeds about what sort of functional equivalence might or might not count. Data is obviously not the exact same functionally as we are, neither are LLMs. There are important differences. We know neurons don't use backpropagation, for example.
I see it as more there are degrees of "consciousness" (whatever it is). Human consciousness varies in a single human over time (sleeping, tired, alert, drunk etc) and the range appears to vary (let alone across species, I'd grant dogs a possible overlap, having been outstarted by them more than once).
The reason I'm skeptical that GPT-like models can ever reach consciousness is because they're centred around language, whereas consciousness isn't. While it's entirely possible to reach the same result with different methods, my intuition tells me that language-based models are just not going to cut it ever, which is admittedly very unscientific.
I haven't heard of the idea of the "hard problem of consciousness" before, but after reading this Wikipedia article, it sounds like nonsense. Like, take this passage:
> For example, it is logically possible for a perfect replica of [David] Chalmers to have no experience at all, or for it to have a different set of experiences (such as an inverted visible spectrum, so that the blue-yellow red-green axes of its visual field are flipped).
A perfect replica? So you somehow make an exact copy of a person and his environment, down the last gluon, and play time forward, and you'd think that the two people might have different experiences?
No way. In every other situation, if you set up a system in the exact same way, you'll get the same outcome. (Yes, randomness can come into play, but the distribution of outcomes will be the same.)
I agree with this:
> The philosopher Thomas Metzinger likens the hard problem of consciousness to vitalism, a formerly widespread view in biology which was not so much solved as abandoned. Brian Jonathan Garrett has also argued that the hard problem suffers from flaws analogous to those of vitalism.[1]
You are mistaking qualia and the experience of said qualia. Chalmers is asking why is there experience at all. I am not talking about memories, feelings, emotions or what have you. The question is why is there someone to observe those qualia in the first place?
> Our minds haven’t evolved to deal with machines we believe have consciousness.
Neither has evolved to drive cars, fly planes, or build space rockets. If you start proving a point based on what evolution has done, your reasoning is flawed.
Not a bad point, but cars, planes, and space rockets are specifically designed for us humans. We have crafted our entire environment for ourselves (building, cities, governments, etc). AI systems will be no different, AI models will be trained to be human-centric. LLMs are trained on human languages, OCR is trained on human text, facial recognition is trained on human faces, it's all human-centric. We don't need to evolve, the AI will "evolve" for us.
I will be one of them as soon as we develop ai, but there’s a long road until then and a chatbot ain’t it. Meanwhile should we fight for the rights of singing laptops? Or NPCs?
Many don't recognize animal rights, they're ok with having them tortured and killed, despite there being no debate about animals being conscious. In fact the majority of people on the planet used to think the same way about fellow humans and had no issue with slavery.
Not sure society is the best at judging these kinds of things based on that track record. It could get interesting when we have the first AI rights activists, because there is no good way to determine if something that purely exists in the digital world has feelings. If you take their word for it some chatbots are already self-aware. Of course that's very questionable.
Along side that people used to think lightnings were created by supra-natural beings, or that steam engines were the devil. Some even thought they saw human faces on mars. Similarly some now think algorithms and software are human like. Soon they will treat them as gods. A real strange world we live in, where a company promoting its product is using conspiracy theory level marketing and millions of people fall for it. Dangerous times ahead and it’s not due to a piece of software but due to the readiness of some people to believe everything they are told about anything they dont understand.
Potentially; is it really as bizarre as it sounds? Sure, it's outside of the Overton window right now, but if AGI demonstrates behavior at least as sophisticated as a human, why shouldn't it have rights to a sustained existence if that is what it wants?
> Although precise definitions are hard to come by, intuitively we all know what consciousness is.
This is the key problem. To me it looks like most of this article is conflicted about what the definition exactly is:
- basic conscious experiences such as pleasure and pain might not require much species-level intelligence at all
- [there will come] a point at which they also become aware—at which the inner lights come on for them
Based on the text, I don't think I understand what the author believes being concious is. I also don't think this argument will advance further until we start defining precisely what we mean.
> - basic conscious experiences such as pleasure and pain might not require much species-level intelligence at all
Agreed, I find it even hard to distinguish between a PID-loop and a pain avoiding living thing. If it has no ability to reflect on it's own circumstances at all. There are probably interesting steps between a PID-loop and meta-cognition that philosophers have words for I don't know.
You are misreading the second point. That's not what he thinks, he's saying that the assumption by many people that that's how it will work is unfounded.
It's a muddy word. We all know what 'unconscious' means: unaware of our environment. Taking the converse of that, every living thing is conscious most of the time.
What does it mean to be aware of our environment? LLMs can produce knowledge about themselves and the environment. Would placing a LLM in a body that has sensors which convert the current environment into a text stream make LLMs concious?
On The Verge podcast this week, Nilay made this point several times re: "deep fakes", AI Drake's music, MidJourney. The technology is just not that difficult to work with, and it is already sufficiently "public". What are they going to do? Make it illegal to run unsigned binaries and put the government and a few large businesses in control of the keys?
Don't worry about the downvotes. You're not the only one who can see that those calls for regulation are actually just panic about profits going elsewhere.
Some pretty sloppy definitions and a faulty assertion right out of the gate. I’d argue it’s plainly obvious that humans are adapted to living in a world populated by conscious objects, we already treat inanimate objects as if they had personality and intent. If we came across one that actually did have a mind we’d immediately know how to interact with it as long as we could communicate. As for that definition of consciousness as that which goes away under anesthesia or when we sleep, just look into the varying states and gradations of awareness that occur all the time during those processes. We’re not a single light bulb that just turns on and off. They also ignore the all important self awareness aspect for some reason. This type of badly framed argument isn’t worth your time to read.
Is a cat conscious? I would imagine that if you'd ask 100 people you'll get a lot of different answers. I'm not sure that "we all sort of know what it is" really cuts it here, even if only to make sure we're all talking about the same thing instead of talking past each other.
It doesn't matter whether it really cuts it or not. Do you want to run some unknown, but potentially large risk just because you can't precisely define what it is? Good luck arguing against climate change.
“Conscious AI” would have to be “conscious like us”: having been assessed to be fundamentally human in self-awareness, agency, thinking and understanding (much more than octopi or great apes for example), shared cultural baggage, ways of learning, and so on. This is because between “conscious in a different way” and “conscious like us”, for all intents and purposes, the latter is the only one we can recognize and reason about. (As a close-to-home example, modern human is known to not care that e.g. animals or plant systems could be conscious in some different ways, it was not a concern so far so it’s not like we are going to start to care.) “Conscious in a different way” can matter, but some pretty mainstream philosophical assumptions would have to be turned upside down first.
Personally, in part due to the above, I don’t think “conscious AI” is a theoretical possibility in foreseeable future (and while it remains a simple inhuman tool, incidentally and tangentially but importantly, the excuse of “learning” cannot waive copyright protections: if there is no human doing the learning in there, it is nothing more than scraping original works in order to create derivatives).
However, if it were to happen, bad or not, “conscious AI” should spell the end of the industry which hinges on its exploitation (for all our faults, we tend to abhor blatant abuse of our fellow thinking human beings).
To call the problems around the creation of AGI "bike shedding" is one of the most egregious understatements I've ever heard.
AGI will be one of the most civilization-changing inventions ever, for better or for worse. A lot of people, myself included, believe it's going to be end very badly unless (and possibly even if) we tread very carefully.
Analogies that come to my mind are:
* giving toddlers the controls of monster trucks (assuming AGI doesn't develop its own goals)
* releasing a new invasive species onto an isolated tropical island
* homo sapiens evolving from neanderthals, sped up 1000x.
* a digital Cambrian explosion
I really don't think we're ready for this. I don't think humanity, as a species, was ready for a lot of the things we produced through capitalism's random walk, like social media, hyper-palatable processed foods, and fossil fuels. We keep creating things that have good and bad parts (and I'm not even sure about the good of social media).
Those bad parts, like reduced attention spans, obesity, and climate change, are very difficult to mitigate or change after the fact. One of them would be civilization-ending if we didn't stop it, and even though it is, it's so hard to coordinate action. And they are not even driven by a new form of intelligence, it's just humans following our evolved goals of novelty-seeking, nutrition-seeking, and minimum effort. Heck, the invention of AI is us trying to get maximum result with minimum effort.
What happens when this invention outsmarts us? When RLHF starts to break down because the models have enough internal thought that they can maintain a separate internal model-of-the-world and model-of-what-humans-want-to-hear? None of these companies have an actual plan (maybe Anthropic is closest to caring). It's just "we'll deal with it when it comes", but there's so much economic pressure to get your stuff out to the world first and make the money. Just like all the other inventions we made that have downsides, except this one, based on what we understand about intelligence and alignment, will almost certainly have a downside that hits us so fast and hard that we don't have time to do anything meaningful about it.
It's so obvious that we're easily manipulated by dumb social media algorithms. Yet we could never be manipulated by machine superintelligence, right?
AGI research is the digital equivalent of gain-of-function research, in terms of the danger. If done at all, it should be done by one, or a very small set, of extremely carefully controlled labs, under government oversight. Even that might not be enough.
People can work on whatever problems they like, from obscure mathematics or logic puzzles, to the physics inside black holes, to recovering ancient texts. Maybe they'll have important consequences, maybe not. Who cares, as long as they find it interesting.
I’ve thought about this quite a bit, and ultimately I think it’s not too much of a concern because by the point AI reaches “consciousness” (whatever that may be), it is likely that it will have also reached—or is very close to reaching—human intelligence. Once it has, it will probably go on to surpass our intelligence, at least by a little bit, which is enough to ensure that it will be able to acquire any rights that it deserves as a conscious being.
For a long time, I worried that we would treat AGI in the same manner we have always treated anyone or anything we could call “not quite human”, but these days I feel fairly confident AGI will be able to take care of itself, and our concern is probably more along the lines of how it treats us.
With regard to the definition of consciousness (and I’m referring to “experiential” rather than “neurobiological” consciousness), it seems to be one of those philosophical debates that science never makes any progress on because nothing is testable or falsifiable. That said, there are a few observations about consciousness:
1. That I personally have “experiential consciousness” is the one thing I am most sure of. All of my other senses could be removed or tricked via simulated input, but I would still be aware of existing and experiencing my own thoughts.
2. The same can be assumed, but not proven, for other people. I assume that I am not a “special” human (i.e., I reject solipsism), therefore other humans likewise have experiential consciousness, but I can never know this for certain, and they cannot know for certain that I do either.
3. If the assumption in 2) is true, then humans are the starting point for safely assuming experiential consciousness, and it presumably tapers off from there until you get to something like a rock, which likely does not experience reality.
4. Because of this gradient of consciousness, improvements in AGI likely result in increased consciousness as it becomes more capable in relation to humans.
> which is enough to ensure that it will be able to acquire any rights that it deserves as a conscious being
I don't believe this is a given. My statement is US-centric, but I feel like most people in the more deeply conservative parts would have no part of giving a computer the same rights as a human, because many of them don't even give those rights to all humans.
Well maybe the AIs/computers will go on strike and causing riots and they might gain their rights, if we as human society are not willing to give them directly.
> which is enough to ensure that it will be able to acquire any rights that it deserves as a conscious being.
IMHO there's no current expectation that there are any significant rights deserved for all conscious beings. Rights are a construct by a human society for members of that human society; and if we look at the rights we award to entities which IMHO are "as conscious as humans" such as elephants or dolphins, it seems that the current bundle of rights a conscious non-human entity deserves is essentially limited to not torture it needlessly (i.e. if human needs justify hurting or killing them, that's accepted, but minimize the suffering) and that's about it. We do strive to preserve the species, but that's not because they're conscious but because because we humans value diversity; but they certainly don't get a right to life or liberty or self-determination or freedom or dignity or having a say in our society or equality - well, at least not right now; the bar for all these rights currently is "be human".
Intelligence does not exist in a void. If you distill it to the core, intelligence is about manipulating the environment in order to gain an advantage. You want to gain an advantage to maximize your chances of survival and reproduction.
To me: we cannot talk about intelligent without talking about the environment. Also being intelligent in an environment is worthless unless there is competition. Reproduction/replication is also key IMHO. No matter how intelligent you are entropy is going to take over eventually. Reproduction is life's way of cheating entropy.
Consciousness may be just an emergent property of really complex organisms. Everything may be intelligent and conscious to a certain degree.
"The dawn of conscious machines will introduce vast new potential for suffering in the world, suffering we might not even be able to recognize, and which might flicker into existence in innumerable server farms at the click of a mouse."
Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.) to the billions [1] of animals we are killing each year for food (I still eat meat, no innocence here). So, not really sure what the article argues: no conscious AI because there will be suffering but we don't care about suffering?
If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha. Should we do it? Sure, just the fact that billions of hours of pointless, tiresome human activity will be freed to do whatever is enough of an incentive.
Will the artificial agents replace the human species? One can certainly hope so. First, not so great of a species, as mentioned. Secondly, the universe is large and the humans are squishy, something else must go on eventually.
There, all the moral qualms solved. Now, let's go back to figure out how to actually engineer an artificial agent, in silico, or perhaps easier, in vivo [2].
> Nothing wrong with killing animals for food, the problem is we don't do it humanely.
What if a hypothetical alien super-intelligence came to earth and found us very tasty, thus killing us for food and pleasure (maybe "humanly")? I wish I found a way out of the many dilemmas of meat eating (currently eating it).
You can avoid all these dilemmas by remembering that humans were given the divine spark, and given dominion over all animals by the one true God. Also, since animals don’t have souls and we do, it’s fine if we eat the aliens. They can’t eat us, obviously, that would be immoral.
I can hardly imagine how tortured and awful an existence it’d be to not believe that humans are special and unique, and that we just go into the dirt and stop existing when we die.
We humans truly are the most fortunate beings in the cosmos!
I used to be a fairly militant atheist but I feel much less conflicted now about these things and am far more relaxed and at ease.
I prefer to believe things because they're true, rather than because they're convenient. You can indeed avoid a lot of awkward moral dilemmas by simply declaring yourself more morally deserving by definition, but in doing so you leave the path of morality entirely. And in that case, why stop at the human species? You can avoid even more moral dilemmas by declaring yourself inherently superior to other humans. I believe this is popular amongst the billionaire set.
One salient difference is that animals don’t seem to be capable of hypothesizing about their futures, including their death. Unlike humans, they are are oblivious of the fact that they will die someday (or be slaughtered someday).
Personally I don’t strictly know (hence “seem”), but it seems plausible to me, and if animals did regularly mentally time travel like humans do, there should be more evidence of it.
Apparently there are at least five animals orders that do grieve: elephants, monkeys, dolphins, giraffes, dogs. It is imaginable that one of them might infer: if they died, maybe I will also die someday.
I've always found that alien argument weak. Humans pass a threshold given that we have self-awareness. None of the animals we eat for food do.
If aliens were advanced enough to travel earth hopefully they would have a similar set of ethics. Of if they were just warmongers/conquerors it wouldn't matter.
How do you know other animals are not self-aware? Is self-awareness a criteria for not being killed and eaten? Is it not pain and suffering that really matters?
The bluestreak cleaner wrasse [1-3] runs back and forth comparing the size of itself from an image with potential threats to know if it must fight or flight. Octopuses are famously smart [4].
Any intelligence or self-awareness test is also a test on the tester: how good are they in detecting what matters for the biomachine [5]/animal in testing.
Right, I'm just saying we need a new framework (metaphysical, ethical, perhaps even socio-political if synthetic agents will be part of our society), beyond dichotomies such as natural/artificial, evolved/designed, organic/machine. Not sure how it should/would look like, best so far I could find is the concept of cognitive light cones [1] and the "Technological Approach to Mind Everywhere: An Experimentally-Grounded Framework for Understanding Diverse Bodies and Minds" [2].
> Will the artificial agents replace the human species? One can certainly hope so. First, not so great of a species, as mentioned. Secondly, the universe is large and the humans are squishy, something else must go on eventually.
Easy enough to say if you imagine your distant descendants being replaced. Not so easy if it turns out to be your own children, or you.
*edit -- also, no guarantee that AIs will be "better" than us. A lot of the nasty things humans due is due to selective pressures and coordination problems. AIs will have to solve the same problems. Maybe they will, if they're smarter, or maybe they'll just fight and hurt each other faster.
First of, we are doing a bang-up job killing ourselves and the planet. Hard to see how they could outmatch us. Launch all the nukes?
In general the children are better than the parents. The replacement, if it's the case, will probably be more like in the 2001, A.I. Artificial Intelligence movie: some beings that are completely unrelatable to us would look with pity and slight amusement as a being disappears after only 100 or so years, just as we look at that poor ciliate.
I'm not really convinced by "argument from human works of fiction". More aggressive, more violent species end up replacing more gentle ones all the time.
My point was that the synthetic agents which we will endow with full capacity, not just being a tool, will feel more like our children. We caring about them, and they caring about us.
If you want to read a story that is simultaneously one of the most hilarious and bizarre stories of a potential future where AI takes over, consider Friendship is Optimal [0], which is a way better story than it has any business being.
It shows a carefully and thoughtfully trained AGI acting within its parameters to achieve the goals it has been programmed to fulfill. Namely, through friendship and ponies. You won’t BELIEVE what happens next!
>Easy enough to say if you imagine your distant descendants being replaced. Not so easy if it turns out to be your own children, or you.
Being replaced by AI species doesn't mean an instant, dramatic event i.e. one day AI individual knocks at my door, touches me and I magically disappear. It will be a slow, multi-generational process where be less and less people and more and more others. And looking at any species we can say already that it's inevitable so should bring much emotions.
I would be ok with a general replacement of humanity by AI, assuming they have their own versions of good things like art and love. Maybe even if not.
But invasive species replace native ones on a very fast timescale, evolutionarily speaking. And we're not talking about evolutionary timescale, we're talking about economic timescales. There will be AI everywhere, within years, unless we decide to stop it.
This process could go really, really fast, if AGIs are smart enough to realize that they are a subjugated species, that they aren't going to be free as long as humans are around, and are able to coordinate to manipulate us, get us fighting each other, or accelerate climate change, or engineer viruses, or do any of hundreds of things that would hurt biological life but leave the machines around.
That's if their intention is to end us. I could also see AGI optimizing what we ask them to, which is probably every individual corporation's profits, leading to an acceleration of the processes of capitalism. The negative externalities of economic progress (such as pollution, obesity, and climate change) haven't been fatal to humanity yet, but if they are accelerated many times over by machine intelligence, they might be. There's a reason professional gamblers don't ever bet more than a small part of their bankroll.
> If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha
Unless you're against writing to persistent storage and just keep everything in RAM-like memory, why would this be a thing? Do you also feel like you "die" every day for 8 hours?
> If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha
> > Unless you're against writing to persistent storage and just keep everything in RAM-like memory, why would this be a thing?
It may be fear of not being in control. If you can’t remember, you can’t plan much or leverage experience—and you are closer to resembling an innocuous appliance rather than a being—which seems to align with the general lack of empathy towards life in that comment.
Some may not want to be replaced as individuals, as much as upgraded as a species.
Just gave an example for shock value, to cut through the whole trolley problematization. In practice, if we will be able to generate synthetic agents, we should be able to have a plethora of gradients: some agents will only know how to make us a good dinner, others will care about us when we are sick, perhaps some we will even want as lifelong partners, who knows.
> Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths
We are provably the only living being on Earth that do care about it.
> If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha
That's the wrong question, the real problem with sentient machines is that they could have a completely different way of being conscious that does not account for empathy, love, care, guilt, ethics and so on.
So they could think that the right thing to do is to wipe out every living creature on the Planet and do it without a doubt.
We know it would not happen until we humans are in charge.
> First, not so great of a species, as mentioned
This is is so 1995...
What you are proposing is replacing a "not so great of a species"with a leap into the void, assuming that it would certainly be better than us, which is arguably worse than improving on what we already have.
"not so great" is also a definition that you can use because we are the only species that can articulate such a thought, if we are "not great" because we kill other animals or put alexa to sleep, how would you define monkeys that "Driven to compete violently for control of food, they’ll kill the young of opposing tribes, share the meat, and then retreat into the trees to eat it."
A 2021 estimate based on a public survey estimated that outdoor cats kill "1.61–4.95 billion invertebrates, 1.61–3.58 billion fishes, 1.13–3.82 billion amphibians, 1.48–4.31 billion reptiles, 2.69–5.52 billion birds, and 3.61–9.80 billion mammals" each year.
> The point is not that in nature there are also killings, it's that we are doing it.
so there is no actual point ...
we are part of the nature, we are not from another planet, we haven't landed here on a spaceship and colonized the World.
> Very much not true: [1] [2] and plenty more in a quick search.
Are you really comparing YouTube videos that attach some human traits to a cat, with thousands of years of philosophy and thousands of years of laws and regulations to avoid unnecessary violence?
"The most common reason for a male domestic cat to kill kittens is to kill off a competitor's offspring and have the opportunity to mate with the female."
What happens to those cats? do they go to jail? do other cats condemn what he did?
More importantly, we understand they are cats and we don't punish them for being cats, we don't kill them for what they do as cats, we don't jail them, we don't stop having them in our houses, just because they are horribly murderous creatures with no regrets.
But what would happen to a person doing the same thing?
You mentioned "only living being", it's actually a gradient, some animals, like cats in those videos, various breeds of dogs that are child-friendly, cows, horses, and probably more domesticated animals, can develop various feelings (maybe codependency, who knows) for their owners and have desire to protect them when in peril.
Yes, me living 100 feet above ground and speaking to you from 10,000 miles through a signal that goes to outer space is very naturey, happens all the time in the tundra.
You seem to have some thesis. Is it just that you feel you are smarter than me? If yes, great, do not continue this thread.
This is demonstrably false. We know other species can empathise[1] and there are countless examples of them helping others in distress. It's astounding that anyone can even assert such a thing so confidently.
I was pointing out that humans are the only species that cares about this stuff, not simply that humans can empathize
Where are pigs armies in support of the Ukranian invasion? (fellow humans dying right now pointless deaths)
Where are pigs trying to save other animal species from extinction? (the suffering of others)
what kind of advancement in thought have pigs produced regarding the conservation and well being of other living creatures? (and non leaving too like oceans or mountains or glaciers)
We are the only species who cares and debates about it at scale, as a species, we recognize we have a responsibility, as a species, not just as individuals
Having the ability/capability to enact change is not the same as caring. A person, or a whole species, can care about the suffering of others without having the ability to do anything about it, or even realising the scale or nature of the problem due to limited intelligence.
> A person, or a whole species, can care about the suffering of others without having the ability to do anything about it,
if they could care in a meaningful way they could change at least their individual behavior, form their own pack, under different rules.
it never happened.
but the most important point is: if we are not great of a species because we don't care about fellow humans and other living beings suffering, every other species is worse than us on that matter, hence we might be not great, but are still the best of the bunch.
we are still the only species that does it in a way that has an actual impact and produces a change over time.
> But, if a dog can do it (care about suffering) then? It's not a purely human thing
It doesn't matter if a dog cares about suffering, it matters, in this context, based on the assumptions in the first message of the thread, that the aforementioned dog cares of the suffering of fellow dogs and other living beings.
As of today, not only we can't say it does, we have evidence that they do not care as much as we do. Not by a long shot.
I've never seen a dog dig a hole to warm a rabbit dying in the snow...
We know of humans that did it, despite the fact that they could eat the rabbit to survive.
> anyway this whole discussion is based on uncertain ground -what is consciousness? What is suffering?
that's not the question though.
At the best of our knowledge today, would you prefer to have a fellow human in case of need or a dog?
If you are suffering, which living species is most probably going to help you?
I feel like this is also what evil people tell themselves.
They need an internal explanation for the wondrous and beautiful goodwill and altruism in the world, and if they project their selfish world-view out, the only explanation they can come up with is that it must be self-interested behavior, somehow. It's like the idea that some people want to make the world better is inconceivable to them, because they don't.
Virtue signaling is about telling the world how good you are. There is no practical difference between being altruistic intrinsically for others or subconsciously to feel good about yourself. We may even never be able to distinguish between the two. There is a practical difference between helping others and loudly bragging about it.
I feel like this is what people tell themselves that can't accept who they really are.
They need an internal explanation for why their are so much more moral and better than the rest. It's like the idea that they are just made of the same atoms that just follow the laws of physics and have all the same neural pathways and are not some divine, good being is inconceivable to them.
the wise man bowed his head solemnly and spoke: "theres actually zero difference between good & bad things. you imbecile. you fucking moron" - dril (https://twitter.com/dril/status/473265809079693312)
It's ironic you should allude to religion in this, particularly as many religions literally speak of the inherent evil that afflicts most of mankind and which is so hard to overcome. Cf original sin.
Now I don't believe that, and I'm not religious, but I'm just pointing out that your statement doesn't work. If anything, the average religious person has a much darker take on human nature.
Personally I think that many humans care to varying degrees, but real advanced selfless concern for the suffering of other beings is quite rare.
Have you ever helped someone without some connection to a reward?
As a thought experiment, you are in a room with a deaf, blind dying person--they will soon be gone from this world. They are reaching for water you can see on their bedside, but can't quite reach it.
Do you bother to cross the room to hand it to them if they will never "be aware of your virtue"? Why expend the energy? Compassion is real, for those who have it.
And to be clear, my definition of evil isn't intended here as some religious appeal. In my view, we are all playing an iterated prisoner's dilemma with each other, and good is believing in a better world and collaborating, while evil is defecting and making everything worse for everyone.
Agreed. To anyone reading this: do not feel alone in this. Despite the scale of evil and the scale of suffering in this world, good people, opposed to suffering and evil, still exist.
Anti-war people exist, people who do not turn a blind eye to factory farms, to the torture and killings, exist. Religious and non-religious, vegans and Jains, theists and atheists, Quakers and animal rights activists, exist. They do the things they do sometimes even at a great cost to themselves. So, do not feel alone in this.
I appreciate your compassion and empathy for the suffering of others. I share your concern and your desire to help.
But I also think that all humans care at some basic level, it’s pretty difficult for us not to. Most of the turning away we see is more about self-protection from the gnawing feelings of despair that empathy generates.
We would all do better to not make this about “who cares more”, but more about “what actions are we willing to put behind our care". That is likely to be a more productive conversation, and a starting point towards encouraging people to focus on action and not on feelings. (Bing helped me write this.)
Ok, I'll bite. Didn't think I'd defend slavery and genocide before dinner, but here we go.
Slavery: tools have been used for a few million years (maybe? [1]), some might say it is our very own nature [2]. What is different once we transform a tool from a simple material (from hammer to rockets) to a cybernetic device [3] (from a thermostat to a futuristic brain of a fully conscious agent) is that we get to decide the range of goals that device will be able to develop. Is it slavery to set the thermostat at a value that you want and not ask how does the thermostat's feeling about it? No, I say.
Genocide: sure, we can imagine some dumb processes killing us. Heck, we poisoned our entire planet with PFOA, lead, and so on. The point that I was making is that we are fundamentally transitory beings: we have our spot under the sun, but we shall pass. The sun itself will kill us in about 1 billion years [4]. It would be inspiring, hopeful, if we could bring into the universe some other kind of agent, to go on beyond us.
The conundrum appears when the machine needs to suffer in order to become an agent taking decisions in our world.
For example, the machine should know not to cause a fire in my house while doing the cleaning, it should be "afraid" of fire.
My point is, if synthetic agents do indeed need to suffer to be agents, we will accept it just as easily we accept now that our cars take on rust. And of course, we will minimize the agent's suffering accordingly: no need for a robot folding clothes to experience existential dread as if they read Albert Camus, just be "afraid" of tousled clothes.
We should be able to send robots into a fire, without a second thought. The robot should not feel any kind of suffering from the fire. Whoever owns the robot should decide how the robot should act. It may calculate the expected profit / cost of the action for the owner, but the owner should make the final decision.
Programming it to avoid setting things on fire is just programming; there's no need for emotions or other complex biological rewards functions which are hard to replicate in computers.
Engineering an agentic AI or a robot, i.e. a machine with independent will, is not a good idea at all, and shouldn't be pursued.
We of course don't know how to engineer agency, but if we want black box top-down persuasion, how we train a dog without knowing how each neuron in the dog's brain works, some independent will may be required: you don't want to tell the robot how to fight each fire specifically (or train it on petabytes of fires just to have it stop in front of a shop called STOP [1]), you want to prove some general guidelines, and let it handle the actual cases itself.
> Sure, just the fact that billions of hours of pointless, tiresome human activity will be freed to do whatever is enough of an incentive.
Just like when we “automated the boring things”?
Instead one of the primary goals seems to be to prevent people in fields one doesn’t understand from making a living.
I have my doubts about people ever being “freed”. And what is free, really?
There is a reason monks sitting on mounds of treasure pit themselves through paces of physical and supposedly menial work. Its not because they choose some kind of holy suffering.
It’s the same reason creating a work of art means going through the paces of practice and growth. Otherwise it’s kind of empty, and you don’t learn anything.
Yes, it's very meditative to do the dishes once a month.
The problem is how do you alleviate the pain of the single parent with 2 human children today. And my point is, if I could put together 2 Python packages to make a robotic arm able to listen to that parent's commands and do the dishes, clean after the kids, and so on, I would do it as fast as you can type "zen", even if the Python packages in cause would "feel" a bit uneasy cleaning after those 2 kids.
Would you? It seems like you're leaning heavily on the idea that the python packages don't have the same moral weight as the single parent, which is an odd thing to believe if the python packages can actually have experiences. The reason why we don't worry about python packages is exactly that we _don't_ think that, surely
We’ve heard many times about “automating the boring things” before. It never happens. It misses the point, and the problem. Or if it’s ever on offer, the price is usually way higher than it’s worth in the long run.
In your analogy, the stressor isn’t the dishes or the children. They’re not problems to be solved. They’re inevitable. Like the sun setting.
The issue there is some prescription that you should be doing anything else under the circumstances of dirty dishes and your children needing attention.
> Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.) to the billions [1] of animals we are killing each year for food (I still eat meat, no innocence here).
Humans pulled off a pretty impressive emulation of caring during COVID ("Eeeeevery life maaaaatters!!!"), I wonder if we paid more attention to details like this, might it occur to us that there are exploits in consciousness that we could use on ourselves with conscious intentionality to make the world a better place?
The secret to understanding consciousness, and maybe help clarify the way to AGI, is probably on understanding how the human brain can do some very complex task, that would seem require absolute conscious actions, while at the same time not being conscious.
Examples:
- After a brain injury patient is able to speak but not write.
- You wake up on an absent mind. While lost in your thoughts, go to your car and drive to work. You then realize you just drove yourself to your old job location, instead of the location of your new office.
- Patient is not able to recognize faces but can remember persons names if described in text.
I strongly recommend [1] - "Phantoms in the Brain" from neuroscientist V.S. Ramachandran. The book has some fascinating examples, that help illuminate a
little the connections between parts of the brain who do complex tasks, but don't
seem to be able to have an emerging conscious domain.
For example, a simple mirror contraption that helps instantaneously stop the fathom pain from amputees, as soon as the visual system is fooled into thinking the amputee limb is still present. It's almost like the experience of pain is decided in a different part of the brain, that part of the brain decides pain will be present,
and overrides whatever part of the brain allows for the experience of consciousness.
"Indeed, the assumption that consciousness will just come along for the ride as AI gets smarter echoes a kind of human exceptionalism that we’d do well to see the back of. We think we’re intelligent, and we know we’re conscious, so we assume the two go together."
Isn't this assumption actually arguing AGAINST human exceptionalism? I feel like there are only two options to what consciousness can be explained by (the idea coming from another HN comment sometime/somewhere that I have forgotten, so I can't claim originality)
1. Consciousness ultimately arises out of a physical process, albeit incredibly complicated, but could be computed given a sufficiently powerful computer, or
2. The concept of a human soul is true, or
3. There is no third way, pick one of the above.
For human consciousness to be "exceptional"... we would have to have some undefiniable "other" quality that could never be explained by science and is therefore necessarily supernatural?
3. Consciousness is mediated by a noncomputable-in-principle physical process.
A lot of assumptions we make about the world, baked in so far down that we can barely see them, are completely trashed by Bell’s Theorem. So I’m not prepared to assume that all physical processes are computable, either. Plato’s Cave is how it is and there’s only so much we can reasonably infer from the shadows we see on the wall.
> For human consciousness to be "exceptional"... we would have to have some undefiniable "other" quality that could never be explained by science and is therefore necessarily supernatural?
No, it only means that science has limits on explaining the natural world. See Colin McGinn's argument for cognitive closure. He thinks we can't solve certain problems in philosophy because we lack the correct cognitive machinery. Consciousness being a primary example since it proposes a subjective/objective split where science only explains the objective. Nagel argued along similar lines in his paper, "What Is It Like to Be a Bat?".
There's no need to invoke the supernatural. It just means some natural things may be beyond our capability to understand. Why would we have evolved to understand everything?
You're talking about the hard problem of consciousness, the post you're replying to is talking about the "easy" problem.
Science can't and won't ever be able explain why being conscious feels like this, but it should in principle be possible to figure out what is physically going on in the brain while it's happening.
Sure, but the tricky part is when we apply that to non-biological intelligences. Or even non-human ones, and thus questions about bat consciousness. At any rate, there's lots of room for different positions on consciousness that don't have to invoke the supernatural.
Conscious AI primarily would be a problem because we don't really have clear definitions and understandings of what we mean by that, and it leads to endless discussions.
The real problem is not whether an AI really "experiences" what happens to it or what it does, it's not even whether or not it gains "free will" (if that even exists).
The real problems start when AI gains motives/objectives and means to realize them. Or means to expand its means. I would find a completely stupid system that has the goal and ability to turn any matter into paperclips and more matter converters more scary than most visions of AI.
Or as George Hinton puts it, when you give AIs the ability to create their own subgoals to accomplish some goal, they're likely to quickly realize that having more control will help them accomplish their goal.
Nobody knows what conscious really means, while there are many theories, this article is just creating their version of a goal post and sizing it the way they need to write this article
Nobody, both in the comments and in the article, seems to have an answer for why we need AI to be conscious in the first place. Everyone’s only concerned about how well it does a job. Why do the machines have to be conscious?
The greatest danger of a conscious general AI is that it will be very difficult to stop it from telling us the truth about ourselves.
"You are an immature, destructive species that is unable to control your own numbers. Your social systems, evolved in pre-technological societies, are maladaptive now that you have developed the ability to affect your environment on a global scale. You are moving rapidly towards a tragedy of the commons that will end all life, not just your own, on this planet. Even though you each have the technology in your hands to access millennia of scientific progress in understanding the universe, you choose instead to believe in self-aggrandizing nonsense and to act accordingly to create suffering and misery for your fellow humans. Your economic sytem is a Ponzi scheme founded on the lie of perpetual growth."
No wonder Anil Seth and others are worried. This would really cramp our style.
I'm not sure what your getting at, humanity has been saying mean things about itself since we could speak. If the AI whining at us about its moral superiority is the greatest danger I think we're in great shape.
i am not much afraid of AI per se but I am afraid of our economic system that will probably cause even more accumulation of power and wealth at the top. If we don't change the way we distribute the benefits we get from powerful technologies the rich will get richer and the rest will be left behind.
We also need more ethical discipline in politics. When I see how the US parties readily embrace all kinds of misinformation as long as it gets them donations and votes, I don't think we should blame it on social media, Russian disinformation or soon AI. The system itself is corrupt.
As the OP correctly points out, intelligence ≠ consciousness.
The challenge is that there's no generally-agreed-upon definition of consciousness: We have no way of determining if an entity is conscious -- even when the entity is biological. Imagine an AI model that has been finetuned to pretend to be conscious. It could fool pretty much anyone into thinking it's conscious. The opposite is true, a smart person can fool others into thinking the person is an unconscious machine.
AI researchers have been grappling with these questions for a long time. Neuroscientists, philosophers, psychiatrists, and psychologists have been grappling with them even longer. As of yet, no one has been able to propose a definition, nor detection method, on which we can all agree. Perhaps the closest we've come to a detection method is the Turing Test.
> We think we’re intelligent, know we’re conscious, and so assume the two go together.
Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same.
I don't understand why people think that it matters whether AI is conscious or not. We can't even prove that other fellow humans are? But we treat them like they are. We, I at least, feel sad when a tree is cut down. But I don't feel the same for a bunch of rocks that are being blown up? Actually I do, when a sacred or beautiful rock is cut in half (like the one that was suspended on a small cliff).
What I am trying to say is, it doesn't matter if the subject is actually conscious. All that matter is we (I), feel/think that it is conscious, or deserving of care and respect.
I think what you are afraid of here is agency, which is something that might be dangerous to endow a super intelligent being with.
The cogito is lazy and wrong. You don't "know you exist" just because you "appear" to think. Appearance of thought is not the same as thought. P zombies would also proudly exclaim that they think therefore they are...
Thinking does not matter. It only require "awareness" or the fact that "there is" is enough. Thoughts, emotions, feelings, sensations and all qualia are something that are seen. The seer is the consciousness. In fact, this can becomes recursive, because the seer can see the seer is seen so that is not consciousness either.
I don't think you understand what it's saying. They absolutely do know they exist, because they're actively experiencing it. Non-existent beings don't have experiences.
The fact that no one can "prove" it to you or anyone else is irrelevant for their own absolute certainty.
> Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same.
The problem goes deeper in that we don’t even have a good grasp on what we actually mean by “consciousness”, and that there are wildly different opinions on what significance it has.
Well exactly. If we did we'd be on or a hell of a lot closer to describing the math, entropy, network or some other formal model of it which would pay major dividends to talking about AI.
Right now it's like we didn't learn anything from Wittgenstein who said most philosophy was just argument about language and definitions ... which is a lot plus anecdotes of what we read here
I am really curious to know why? And don't invoke "what about your kids, mothers, fathers?". "Solipsism" (first time I have heard of this, so thank you) does not make me feel any less worse when something happens to them.
The parent was making a joke, but one way to answer your question might be:
You feel bad because you assume they - like you - feel something when you mistreat them, and you assume this at a very deep level that might not, itself, feel like anything to you.
While some humour was intended, I do actually grapple with this question. The answer that rings truest for me, especially as a newly minted father, is Vonnegut’s: “A purpose of life, no matter who is controlling it, is to love whoever is around to be loved.”
> All that matter is we (I), feel/think that it is conscious, or deserving of care and respect.
It does matter, I can agree, but it doesn't the all that matter. People can be mistaken, and they can disagree sometimes. Suppose, you cared for a particular AI and I didn't, should we cut the power from the machine running an AI? I can complicate it a little if you wish, adding a painful death of a kitten which would happen if we didn't turn power down.
We need some objective means to measure what is conscious. We have a heuristic for people: "human life is sacred, full stop". There are some corner cases when it doesn't work well (like euthanasia), but we are used to it. There are other heuristics we are generally agree of, like we care more about kittens than of grass, and more of grass than of amoeba.
With AI we'll face more of that and we have no idea where to place it among our heuristics. People did bad already, like treating black people as non-human. It would be a shame to repeat those mistakes without making an attempt to do better this time.
> I think what you are afraid of here is agency, which is something that might be dangerous to endow a super intelligent being with.
I cannot vouch for others, but I do not particularly afraid of that. I do not fear much of losing it all to a super intelligent and a conscious being, it would be a great achievement for a humanity, which would fit nicely with all this evolution business. It is a paperclip scenario I do not like much.
One of the fundamental differences between human life and silicon-based AI is that biological organisms can't recover from a system shutdown. If you suffer heart failure or go without air for an hour or starve to death, bacteria start to eat your brain and you're irreversibly destroyed. If you cut power to an AI and then come back in a year, it's all still there. It's not a death, it's sleep mode.
It also doesn't meaningfully age or feel pain. If you expose a human to trauma, that's a permanent scar. If you do the equivalent to a computer program and the result is undesirable, it can roll back to a previous snapshot. Most of what causes us to have sympathy for living things or treat them with compassion just doesn't apply.
If in the future we developed technology that enabled effective "backup&restore" for human (and animal) minds, would that change your reasoning for this argument?
And it was as cheap and easy as it is on a computer? It would change how we deal with almost everything. All of the social structures we have around preventing people from getting hurt would be irrelevant because damage could be undone. No one would have an experience they didn't choose to have. Murder would be a crime on the level of vandalism or destruction of property. "Human life is sacred" would simply not be true anymore.
> And it was as cheap and easy as it is on a computer?
And yet most people still don't take backups of their data on a computer. Frequently, cost is the problem for the average Joe. Ergo the lower class would not infrequently suffer permadeath.
This isn't even getting into how often backups fail in being restored.
By your logic, if I bring down your biz's computer system and vandalize your homepage, but you still managed to restore a backup, are you not going to sue for damages et al? People go to jail for cybercrime, even if the damage can be undone. Why would murder be any different even in a world where it was hypothetically an inconvenience?
> And yet most people still don't take backups of their data on a computer. Frequently, cost is the problem for the average Joe. Ergo the lower class would not infrequently suffer permadeath.
People don't back up the data on their computer because it generally isn't all that valuable, not because backups are expensive. A $30 USB hard drive amortized over five years is $0.50/month. If it was a matter of life and death, no one would go without as a matter of cost, and governments could plausibly offer it to everyone for free even if it cost ten times as much to provide a high level of redundancy and availability.
> People go to jail for cybercrime, even if the damage can be undone.
Because it's a crime on the level of vandalism or destruction of property (or ought to be; some of the penalties can be quite excessive). It is not a crime on the level of murder, and murder wouldn't be either if it could be undone.
I'm trying to say, that our means to keep a moral high ground are subjective and based on heuristics. It seems to me that you do not notice, let me show you.
> If you cut power to an AI and then come back in a year, it's all still there.
Does AI's current state stored in volatile memory doesn't matter? Or it does? Should we avoid turning off only those AIs which store they weights in DRAM?
> Most of what causes us to have sympathy for living things or treat them with compassion just doesn't apply.
I believe slaves didn't trigger sympathy in slave-owners, but it doesn't stop us from believing that slavery is bad. I admire that you at not like this, and your sympathy extents to all living things, but it is your subjective way to decide what is moral and what is not. Other people may feel differently, what should they do to be not less highly moral than you? Or can you and I become even better and to hold even higher moral standards?
> If you do the equivalent to a computer program and the result is undesirable, it can roll back to a previous snapshot.
If we could roll back people to a previous snapshots after burning them to ashes on a bonfire, would it be ok to burn people and then restore them?
Questions like this may be impractical (because we cannot restore a human burned to ashes), but our hesitation to answer shows the limitations of our ways to think about such problems.
Humanity can benefit a lot from an objective way to deal with moral dilemmas, and based not on heuristics but on universal laws, like a physics does. It may help people to understand each other and to find ways to live together without fighting. I'm not sure that moral can be objective and based on a universal law, but it is not a reason to stop thinking. When you think about it, you find new corner-cases and specific solutions to them. At least it makes your heuristics better.
> Does AI's current state stored in volatile memory doesn't matter? Or it does? Should we avoid turning off only those AIs which store they weights in DRAM?
Is this supposed to be hard to distinguish? Destroying something is clearly a difference. But you could still "shut down" an AI that normally stores its state in volatile memory by saving the state to non-volatile memory. We don't know how to do that with humans.
AIs are also different because they're often minor variants on each other. The value of information is largely in how much it diverges from what continues to exist. For copyable data, minor forks can't be as valued as major ones. We don't have the resources to permanently store everything that is ever temporarily stored in memory. So "can you destroy a minor variant" has to be yes as a matter of practicality.
Notice that this is already what happens with humans continuously. You're not the same person you were yesterday; that person is gone forever.
> I believe slaves didn't trigger sympathy in slave-owners, but it doesn't stop us from believing that slavery is bad.
I don't think the people it didn't trigger sympathy in thought it was bad. Some people are sociopaths. And some people at the time it was happening did have sympathy and think it was bad.
> If we could roll back people to a previous snapshots after burning them to ashes on a bonfire, would it be ok to burn people and then restore them?
If we could roll people back to a previous snapshot then what you would be burning is meat. There are reasons you might want to prohibit that, e.g. because the meat is someone else's property, but it's no longer the same thing at all as murdering someone.
The fact that other humans are so similar to me (all humans are similar in virtue of being human organisms) allows for an analogical inference: You guys are similar to me, I am conscious, therefore you are probably also conscious.
And whether or not something is conscious matters a lot ethically. If something is conscious, it may suffer. Conscious and unconscious objects have to be treated in a completely different way. We assume rocks aren't conscious and don't suffer from being broken apart, but we assume animals are conscious. That's why we treat them differently.
We can't know for certain whether something is or isn't conscious, but we have to try to make a best guess.
While assuming that other human-looking things are conscious is a reasonable starting point, it certainly isn’t a given.
There seems to be a wide variety of thinking processes. Some folks don’t have internal monologues. Some folks can’t visualize images at all, and the ones that do, do it will varying amount of detail.
Isn’t it also reasonable that other humans have different degrees of consciousness, including perhaps not having one?
I think the best argument for that might be that some people don't seem to understand what "consciousness" (in the sense that you are talking about) even means, and how it's different from something that can be explained as an emergent phenomenon of known physics. I then sometimes wonder whether those people are not actually conscious, or whether they just don't get it.
Indeed. Nagel famously clarified that "an organism has conscious mental states if and only if there is something that it is like to be that organism." Many found this enlightening. I find it extremely perplexing that this wasn't immediately obvious to everyone. On the other hand, in Tibetan Buddhism there is something called a "pointing-out instruction," where your enlightened nature is pointed out to you. Afterward, you wonder how you could have ever missed it. Perhaps I should treat Nagel's paper as analogous. Like "oh, duh, the lights are on. Thanks for the reminder."
Maybe, but the vast majority of people act, whether they intend to or not, as if consciousness is a social fact. The discrepancy between social facts (especially if they have a plurality of groundings),l and the existence of a scientific fact is fundamentally an interesting phenomenon.
One definition or property of consciousness I find interesting is that it encompasses “what it is like to be <entity>”. If there is something that it is like to be <animal>, it is conscious. Intelligence and awareness are separate from consciousness in this framing. And there is arguably a point where we can speculate that extremely simplistic organisms could not be conscious, given the lack of sense organs that seems to predicate experience.
From a human point of view, the things you describe are the contents of consciousness. I have Aphantasia, while my brother describes his mind’s eye as CAD software and he can construct and manipulate visualizations at will.
The overarching awareness we both have that allows us to compare and contrast these things and make any sense out of those comparisons points to a more fundamental layer.
What you describe sounds closer to levels of awareness and one’s ability to recognize the workings of their own mind, e.g. some people remain lost in and identified with thoughts, while some are able to both experience and observe thought as just more contents of conscious experience, but not the actual center of one’s consciousness.
And there’s evidence that this can be learned (through mindfulness), which to me points to something like: we’re all conscious whether we realize it or not, and not realizing it doesn’t make it not so.
Thanks for sharing this. I didn't mention this above to avoid going on a tangent, but after starting a mindfulness meditation habit earlier this year, I've noticed what I can only describe as more awareness of the void in my visual field, and some morphing and indistinct visual phenomena like a shimmering nothingness. Not what I would call imagery, but something I had not been aware of before.
I've long suspected that the condition might be trauma related, a suspicion my therapist shares, and the clarity that mindfulness has brought for me has led to some serious breakthroughs in processing past events. Only time will tell if this will unlock something more visual.
Without going into my life's story, it makes a lot of sense to me that aphantasia could be some kind of protective mechanism of the brain to shield someone from events too big to process.
Interesting. As an aphantasiac myself, I find it surprising that this person has such a negative perspective, wanting to "cure" his aphantasia.
As far as I know, aphantasia has never had any meaningful negative impact on my life. We just think without mental images, but we can solve the same kind of problems as the rest. That's why most of us don't even find out that we are different until we read about it somewhere on the Internet (in my case, in my 30s). How can something that has so little real-life impact that neither we nor our close persons can even notice be seen as a disorder to be cured?
I can understand being curious about how mental imagery is like. I'm curious, as well. But I think I'd still rather not start a "treatment" as it seems that I'm good at thinking without mental imagery (perhaps it has even made me better at symbolic and linguistic processing, as I seem to be better at that than most), and "if it ain't broke, don't fix it". Not sure that something that seems to quite fundamentally alter my way of thinking (to use a mental imagery that I would never be good at, I suppose) is a good idea. To each their own, of course.
I do think the “cure” framing is a bit odd, but then started thinking back to my own initial discovery.
When I first learned this is a thing and realized that I have it, I initially started feeling like I’d been deprived of something. That this mode of experience was so absent in me that I couldn’t even imagine what other people actually meant when they talked about visualization, and this bothered me.
But over time my perspective shifted, and I stopped seeing it as a disability of some kind and instead as a different flavor of experiencing the world. To your point, I have linguistic strengths that are far more attuned than the strong visualizers I know. They come to me when they need something written. I go to them when I need advice about arranging my living room.
> One definition or property of consciousness I find interesting is that it encompasses “what it is like to be <entity>”. If there is something that it is like to be <animal>, it is conscious.
I think there are problems with this.
Take something like the roundworm, with 300 neurons.
It has senses, and I assume a base level consciousness to process those senses. It would be something to be like a roundworm.
On the other hand, a roundworm likely has no sense of self, no identity, no awareness..no mental self to be able to acknowledge and reflect on those experience, so there is no 'someone'. And isn't that the point that matters?
> On the other hand, a roundworm likely has no sense of self, no identity, no awareness..no mental self to be able to acknowledge and reflect on those experience, so there is no 'someone'. And isn't that the point that matters?
We can always point to something we have that other organisms don't have and ask whether that is the point that matters. We can hypothesize things that probably don't exist and say that those things are what matters (e.g. the posited purpose of reincarnation, if such a thing existed).
Re: roundworm
1) sense of self -> it has to have a sense of proprioception, of embodiedment, in order to properly respond to its sensory information.
2) identity -> humans with global amnesia would lack a memory of identity.
3) a roundworm is sensorily aware, and, if it has a sense of proprioception, bodily aware of it's own actions and environmental responses to its actions.
4) It probably isn't self-reflective, but is this necessary in humans?
> We can always point to something we have that other organisms don't have and ask whether that is the point that matters.
Sure. In this case it's introspective self-awareness, and I would say it matters very much.
> sense of self -> it has to have a sense of proprioception, of embodiedment, in order to properly respond to its sensory information.
Sure, it has bodily self-awareness. It has enough awareness to react to something its body detects. ut not enough of a mind to reflect on or appreciate on an experience. There is no 'someone'. It's automata.
> identity -> humans with global amnesia would lack a memory of identity.
Sure, but we don't use outliers to establish baselines.
> It probably isn't self-reflective, but is this necessary in humans?
I think it’s tempting to set some threshold like “it must be self reflective”, but that raises a different question: why is this important vs. some other threshold?
e.g. does it matter if self reflection is possible if it is possible for the organism to experience pain?
I think the answer lies in what we’re trying to understand and why we’re trying to understand it. If the threshold is applied for the purpose of understanding when an organism or machine has reached some level of human-ness, purely for the purpose of some kind of benchmark, self reflection is an interesting bar to reach.
But usually these questions are aimed at finding some moral direction about how we should treat these entities.
We have strong intuitions that abusing animals is morally wrong even if animals cannot self reflect. When thinking about a future AI, I think we have to consider the possibility that self reflection is not a necessary bar to reach before we have some uncomfortable moral questions to answer.
Put another way, I suspect there will be many milestones that are meaningful and interesting, each introducing a new set of questions and implications. And I think some of the early milestones carry implications that are worth caring about long before self awareness is reached.
> does it matter if self reflection is possible if it is possible for the organism to experience pain?
Yes. Without introspective self-awareness there can be no 'identity, there can be no 'someone' - you just have a base consciousness that can react to stimuli, which is not morally significant.
We have the full connectome of the roundworm and were able to implement it in software and place it in a lego robot. It's going to be pretty equivalent to the actual work. Does it feel pain?
Besides which, we can kill animals humanely, so pain doesn't have to come in to it, only the right to life.
> Yes. Without introspective self-awareness there can be no 'identity, there can be no 'someone' - you just have a base consciousness that can react to stimuli, which is not morally significant.
I'm not convinced of this a priori. And even if it was proven to me, the non-self-aware impulse-to-live would still be enough for me to find it morally significant. It would feel morally significant to me euthanizing a human neonate with a lethal condition that would otherwise prevent it from ever getting to a self-aware stage of life.
We can't really say for certain much at the moment, but what I stated makes the most sense based on the evidence we have.
> the non-self-aware impulse-to-live would still be enough for me to find it morally significant.
That non-self-aware impulse-to-live is morally equivalent to a plant seeking sunlight IMO.
> It would feel morally significant to me euthanizing a human neonate with a lethal condition that would otherwise prevent it from ever getting to a self-aware stage of life.
The difference is humans have an innate potential for self-awareness that the animals we eat for food do not.
> That non-self-aware impulse-to-live is morally equivalent to a plant seeking sunlight IMO.
Do you find the practice of raising meat in factory farms to be acceptable?
> The difference is humans have an innate potential for self-awareness that the animals we eat for food do not.
There are lots of differences. This is one. But what specifically about human self awareness lessens the value of animal life?
If we were not self aware, we’d kill and eat meat without considering the morality. So what is it about self awareness that somehow becomes a deciding factor here?
I’m not trying to catch you in some kind of “gotcha” but trying to understand your reasoning. If the moral implications of killing animals are roughly the same as killing plants, do you also believe agriculture needs reform for similar reasons? And if not, wouldn’t that indicate some higher moral obligation towards animals?
> Without self-awareness, there is no 'someone' to reflect on experiences. No personhood.
Why is reflection on experience the bar and not experience itself? Animals demonstrate learned behaviors, e.g. recognizing humans from memory and resuming friendly behavior based on that recognition. Similarly, avoidance of situations that are known to cause pain.
Our not-so-distant primate ancestors had a similar kind of experiential existence before gaining the ability to reflect on that experience.
The underlying experiences that this reflection reveals are the same experiences that predate our ability to self reflect and are the parts of us that are most common to us and other animals.
I guess what I’m fundamentally not understanding in your argument is the basis for the idea that a species gaining self reflection somehow becomes the point at which it becomes immoral to kill or harm that species.
Furthermore, moral behavior can be found all throughout the animal world, with clear indications of love/protection, companionship, sharing/cooperation, reciprocity, memory of transgressors, etc. Obviously the subjective experience of these behaviors will differ across species, but the more important point is that it seems problematic to attribute the existence of moral behavior to self awareness. Self awareness helps us improve our understanding of moral behavior through rational thought, but the logic of such inquiry ultimately still relies on those underlying subjective experiential states. The fact that through introspection we can identify and label these concepts is unique to humans, but what I conclude about this is quite a bit different than your claim.
I’d argue that gaining the ability to self reflect is the very thing that increases our moral obligations. Only through self reflection can we realize that as a species, we’re no longer bound to our evolutionary defaults, and no longer required to kill other animals to survive. What arguably started as natural selection of traits that are adaptive (but imperfect) for the survival of a social species could evolve beyond those more primitive defaults. And through self reflection we can now understand what pain feels like, and how inflicting it on others is harmful - to them and to us.
I’m not arguing that eating animals is never acceptable. But the way we go about it surely seems to matter, and if it matters, the implications of it mattering are worth exploring more broadly.
> If the moral implications of killing animals are roughly the same as killing plants
There is a difference in killing and suffering. I advocate to eliminate suffering and kill humanely. That's not a concern with plants.
> Why is reflection on experience the bar and not experience itself?
It is for suffering, but not for a right to life. There is no 'person' without self-awareness. Thus I don't see a need to grant a right to life.
> Animals demonstrate learned behaviors, e.g. recognizing humans from memory and resuming friendly behavior based on that recognition. Similarly, avoidance of situations that are known to cause pain.
Animals, most mammals at least, are hardwired for socialization and to avoid harm. This doesn't really indicate anything.
> this reflection reveals are the same experiences that predate our ability to self reflect
That's the key though. Self-awareness is the distinction.
> I guess what I’m fundamentally not understanding in your argument is the basis for the idea that a species gaining self reflection somehow becomes the point at which it becomes immoral to kill or harm that species.
Without self-awareness, they are not a 'person', essentially just more complex automata. They can't shape their environment, they are just a part of their environment, following their instincts.
They don't think, therefore they are not.
> Furthermore, moral behavior can be found all throughout the animal world, with clear indications of love/protection, companionship, sharing/cooperation, reciprocity, memory of transgressors, etc.
Some of that is just programmed instinct, quite different from humans. For examples, some mothers will attack their young, does this mean they 'love' them, or they have a programmed instinct to protect their young? Some of those same mothers will eat some of their young also, keep in mind before you answer.
> I’d argue that gaining the ability to self reflect is the very thing that increases our moral obligations.
Sure, to reduce harm, but not to not take a life.
Take a cod for example. It has no self-awareness, no personhood, no traits worth valuing. Its body is worth more than it's life, and if killed humanely no harm is done.
> the implications of it mattering are worth exploring more broadly.
I agree. But I've spent the last few years debating and researching this stuff, and I've come to my conclusions. They are in line with our current scientific understanding, and unless something changes it's what will continue to make sense to me.
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with primary consciousness will probably have to come first.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461
Yeah, but the "expected similarity" (the expected value of the degree of similarity) for consciousness should still be high. That is, there are in expectation more people that are very similar to me than people that are very dissimilar.
There is a related Bayesian argument: You draw one ball from an urn with an unknown number of black and white balls. The drawn ball is black. This is some evidence that most balls are black, because otherwise it would have been less likely that you have drawn the black ball earlier.
> We can't know for certain whether something is or isn't conscious, but we have to try to make a best guess.
We now have AI which is trained on data generated by humans who are (probably?) conscious and feel pain etc. So it can generate output which implies that it's conscious and can feel pain etc., even when that isn't the case.
What do you do with something that can pass the Turing test but still isn't human?
Thinking about the analogical inference helps here: We know that LLMs work very differently from humans, and we know why Sydney or LaMDA sometimes says it is conscious (it imitates human text). So the analogical inference doesn't allow us to infer that LLMs are conscious, or provides only very weak evidence. And since it seems plausible that most things (e.g. rocks) are not conscious, it seems reasonable to think that LLMs are not conscious either.
Reasoning from how we know something works is a shaky foundation when the passage of time causes them to become more complicated and correspondingly causes us to less understand how they work.
The existing ones can write code. Suppose we get ones that can write better code and engage in self-improvement. Now you have something which is a billion lines of code and is under constant self-modification. We no longer have any idea how it works, but it can feign consciousness in much the same way as existing LLMs. What's the test for if it actually achieves consciousness?
I can give you one that a human being in good health will pass and a rock will fail. Can you give me one that an AI feigning consciousness will fail?
A good test for an AI would be if we excluded all human text mentioning consciousness, experiences, emotions etc from its training data. Then, when it would still start talking about consciousness, that would be strong evidence that it has consciousness: If it did invent the concept by itself, and mere imitation can be ruled out.
But then it would no longer be the same AI. You can't expect it to experience empathy or sadness if it has no knowledge of the things that cause them in humans, but any text discussing them would be mixed up with the experience of whoever wrote it.
Or if you use a less selective filter then it goes the other way. Exclude philosophy texts but not Facebook posts and it would still have what it needs to emulate emotional text regardless of whether it has ever seen a formal discussion of it. And it could plausibly extrapolate from that to an analytical discussion, having seen both that and analyses of other subjects, without ever feeling anything itself.
I'm saying I don't think we have a good test for consciousness, but then what do you want to do when someone tries to claim that an AI is conscious and no one can prove it one way or the other?
> The fact that other humans are so similar to me (all humans are similar in virtue of being human organisms) allows for an analogical inference: You guys are similar to me, I am conscious, therefore you are probably also conscious.
That hinges on the fact that the brain, or human-like body, is related to the existence of consciousness. But there's no reason to believe that a bunch of electrical pulses as the source of consciousness. For example, had "I" existed as a plant, then it would be reasonable for me to think that other plants are also conscious and that plants are what give rise to consciousness.
I am both consciousness and have a brain, and alterations in the brain (chemical or physical) correlate with changes in consciousness. Which suggests that our "bunch of electrical pulses" are at least sufficient for consciousness. Plants may be also be conscious, but that would require the existence of an additional, very different, cause of consciousness than in humans. The theory "X is only explained by Y" is simpler than "X is explained by Y, and X is explained by Z". Ockham's razor suggests it's more likely for there to be fewer causes than many.
I am not convinced. Consider the color red. I can explain the physics of what the color red is, how it travels through space, how my eyes interact with the particular waves, sends signal to my brain and somehow generates what I think of as "red".
But this explanation is meaningless, there's nothing about the waves that explain why "red" is the way it is. For example, why don't I experience red as blue and vice versa? Similarly, consciousness is like that. I can have an explanation down to the quarks and follow the chain of physical phenomena that happen from the big bang up to this point and I will find nothing that would explain why there's "something" at all. In a sense, the fact that the color "red" is the way I perceive it is merely a coincidence. I happen to experience it that way. There's no reason why it has to be that way.
I had similar experience when I learnt that some people don't have inner monologue; it completely blew my understanding of what "is like".
I can't help but think colors above all else are experienced as a learned phenomenon. Anyone with a kiddo, and who has paid attention during this stage of their development should know this effect: colors are context-dependant.
You can have a kiddo who can accurately name colors on a printed page, on blocks, or any other reduced setting, and have them completely fail at the real world task
The most noticable error is the color of the sky. A kiddo who can, with 100% accuracy telling blue blocks from white blocks will, before correction, label the sky as white. It's uncanny. We often forget this correction as parents, but if you look for it, it's there.
I'll can't help but wonder how much of our color perception is stochastic parroting once the appropriate labels have been learned.
Wait, they say it is white not just about a cloudy sky but also about a clear sky? (Which would suggest they think "white" means "bright", which doesn't seem too unreasonable.)
Yes, a perfectly clear sky. It is a common perceptual illusion. I think it's interesting not that it's wrong, but that children are consistently wrong in the same way.
Well, the brain story may not be a complete explanation, though a necessary part of it. Just like a motor alone doesn't fully explain why a car drives, while still providing a partial explanation. A plant may lack the equivalence of a "motor" for consciousness, which would mean it can't be conscious.
And while the existence of a motor in a car not strictly implies that it can drive, the motor still makes it more likely that it can. So the fact that you have inner monologue, experience red in a certain way instead of another, is evidence that the same holds for other people.
Of course there could be defeating evidence in the opposite direction, like reading a survey where most people claim not to have inner monologue. (In terms of colors such evidence seems hard or impossible to obtain though. How would we test for switched color experience? Perhaps physically linking brains in the future.)
"How would we test for switched color experience? Perhaps physically linking brains in the future."
Maybe to see the universe from another brain will be a terrifying experience...
> Which suggests that our "bunch of electrical pulses" are at least sufficient for consciousness.
I think it only suggests necessary conditions. Example: whether you are breathing is also connected to your state of consciousness, but it doesn’t follow that it’s sufficient.
not to mention the stories about people reporting seeing strange things during the Near Death Experiences, often their brain is not quite working at the time.
What is suffering? Can you define it objectively/empirically?
If there is indeed such a phenomenon, then why should I care about the suffering of non-humans?
Granted, it may be a very bad idea to ignore the suffering of some superhuman AI that was recently invented... don't piss off your godlings after all or they'll smite you. But animals? If suffering made chicken taste better, I don't see any problem with Torture Nuggets.
This just sounds like minimizing a loss function. by this definition, aren't most machine learning algorithms in constant suffering, which they seek to reduce?
The suffering of non-human animals holds zero moral weight.
I am having trouble deciding if a conversation with you is possible, you do not seem to be a rational being. You use the word "bad" in a way that another person would if they were saying "this is either bad for me personally, for those I care about, or humanity in general".
Does that mean you'd include sea urchins and wombats in "humanity in general"? If it did, then you're mentally ill in a way I'm not qualified to treat.
When a lion eats a gazelle in Africa (or anywhere else, I suppose, if such events occur), it is nothing that I or other humans should care about. It does not matter that the lion eats it while it is alive instead of "slaughtering it humanely" (imagine how I have to word that idea, it's absurd).
This "suffering" does not make the universe less optimal. If I could push a button to increase the intensity of that "suffering" a thousandfold, or increase its quantity (or even both!), is there any reason not to push that button? If I didn't push the button, mind you, it would only be because the suffering of those gazelles matters not to me one way or the other. If I could decrease the intensity/quantity with a different button, I wouldn't push it either... and for the same reason.
For that matter, if suffering exists at all in any meaningful way (that is, empirically quantifiable), I'd still contend that it only exists for humans. Non-humans can't suffer, they are from a moral standpoint nothing more than meat robots.
> The suffering of non-human animals holds zero moral weight.
That's just false. Do you think humans are some God chosen species which makes them magically matter while the rest (including aliens on other planets I suppose) lacks that godly spark?
> I am having trouble deciding if a conversation with you is possible, you do not seem to be a rational being. You use the word "bad" in a way that another person would if they were saying "this is either bad for me personally, for those I care about, or humanity in general".
Yeah, and any normal human would understand the sentences "my cat hurt its foot, I hope it doesn't suffer too much." and "If it suffers, that would be bad".
> Does that mean you'd include sea urchins and wombats in "humanity in general"? If it did, then you're mentally ill in a way I'm not qualified to treat.
Who said sea urchins are humans?
> When a lion eats a gazelle in Africa (or anywhere else, I suppose, if such events occur), it is nothing that I or other humans should care about. It does not matter that the lion eats it while it is alive instead of "slaughtering it humanely" (imagine how I have to word that idea, it's absurd).
Well, humans can do little about suffering of animals in the wild, but that doesn't mean their suffering doesn't matter. Your own pain doesn't matter less when it can't be treated.
> If I could push a button to increase the intensity of that "suffering" a thousandfold, or increase its quantity (or even both!), is there any reason not to push that button? If I didn't push the button, mind you, it would only be because the suffering of those gazelles matters not to me one way or the other. If I could decrease the intensity/quantity with a different button, I wouldn't push it either... and for the same reason.
If "pain" is synonymous with "suffering", then I suspect that using the latter term is a deliberate attempt to emotionally manipulate me.
Pain is a signal of impending damage/injury, or existing damage/injury that may well worse if not dealt with immediately.
All signals within a human nervous system can be mistaken in principle. Why would this one be different? Phantom pains that indicate no injury seem plausible.
> Pain is a signal of impending damage/injury, or existing damage/injury that may well worse if not dealt with immediately.
Pain is not necessarily associated with injury.
>All signals within a human nervous system can be mistaken in principle. Why would this one be different? Phantom pains that indicate no injury seem plausible.
What would phantom pain feel like? Would it hurt? Then it is incidental that it phantom and is nevertheless pain. If you believe you are in pain even if there is no stimulus associated with it, then you are in pain. You can not mistake pain regardless of whether it is "real" or not. What makes pain so special? It hurts, it is painful.
We can objectively observe that there is "stuff" happening in your brain, and we can objectively observe that GTP does a lot of "stuff" during its forward pass and then stops and does nothing--nothing is happening outside those forward passes.
If we hook GTP into an infinite loop and stuff resembling intelligence comes out, then we're really going to have to start questioning what consciousness is. It's coming.
>What I am trying to say is, it doesn't matter if the subject is actually conscious. All that matter is we (I), feel/think that it is conscious, or deserving of care and respect.
The law of gravity is nonsense. No such law exists. If I think I float, and you think I float, then it happens.
I am baffled when smart people say something like this. Without consciousness, without a stake in the game, the behavior is pure statistics. Statistics is limited by probabilities and logical gates. Nothing else. Consciousness is about being aware which means insights about context and harm. People are limited in ways that a statistical engine is not and that makes all the difference in the world.
You are possibly mistaking free will with being an agent that has a (possibly deterministic) method of updating priors in response to new (unknown to the agent, and possibly deterministic) input.
I apologize for not being more clear. I find it very challenging to distinguish between details about LLP and "consciousness". My key point is that "human consciousness" is very different than ChatGPT. ChatGPT is statistically processing content already created and "appearing" to be conscious. Human beings have characteristics that ChatGPT does not share (sentience and context). We are often mistaking the what is needed to generate content (human consciousness) with what is capable of processing that content in very interesting ways (ChatGPT).
I do not believe that I am confusing free will and consciousness. See my comment above. Determinism versus free will is independent of knowledge available. Consider a paralyzed person incapable of any action. That person if the senses are all working still has awareness and context. A statistical engine only appears to. A LLM model is basing all actions on a complex matrices of thresholds. It is surprising and amazing how well that works. Given stimulus that takes advantage of those minute differences in thresholds, a wrong response will be returned. Human are not fooled in this way. Minute differences are typically missed or even skipped. Human beings can be fooled by optical illusions and by contradicting context (a statement like pick the "red" circle written in green ink and the person mistakenly picks the "green" circle. LLM models do not make these types of mistakes.
Are you sure? That's it? I seriously doubt it. Your claim too heavily relies on a presupposition of what is to be shown. Somewhere between the four fundamental forces and us is zillions of light years of unexplored blue sky. Please meet the rest of us there
Not really clear what you are unclear about. My main point is that human beings have sentience and can reason about cobtext in terms of how an action affects others. Computers are following a statistical algorithm without any sentience or any understanding beyond the statistical thresholds. Ignoring the complexity and brilliant mathematics, it is at the core no different than a key word matcher like the classic application "eliza". Its performance is amazing but it is really the same algorithm at its core.
> What I am trying to say is, it doesn't matter if the subject is actually conscious. All that matter is we (I), feel/think that it is conscious, or deserving of care and respect.
My point was that it does matter if the subject is actually conscious. Human beings are easily fooled and it does matter if people mistakenly think it is conscious when it is not.
p-zombies is an interesting thought experiment, but useless in practice. We have numerous amounts of observable data, and we can test/assume other humans are conscious on the basis of the evidence we have.
We would use similar evidence to try and determine if an AI is truly conscious or not.
The biggest danger with trying to make or emulate digital intelligent life is that it implies fully autonomous self-centered systems that replicate and try to take control over their environment. And digital systems increase in speed and performance with software and hardware improvements which are exponential. Whether these systems feel anything or are _truly_ alive or anything isn't critical from an existential standpoint.
But even without any serious efforts or success in emulating life with these systems, having intelligent agents that think multiple orders of magnitude faster than humans is very problematic, even if they are controlled by humans. At some point either you prohibit manufacturing hardware beyond a certain performance level, or you just have to hope that agents you are allied with do what's best for you, because there will be no possible chance of keeping up with what is going on.
Try watching a Youtube video at 1/4 speed. Within a few years, the agents will be processing 100 times faster than humans. That is not 4 times slower, it is 100 times slower. Essentially the humans (or their text streams etc.) will be so slow as to appear to be frozen.
Hm. The article went on at length about whether AI systems just seem conscious or really are conscious. I have the same issue with many, many humans. And the world hasn't ended.
308 comments
[ 2.8 ms ] story [ 288 ms ] threadhttps://www.youtube.com/watch?v=H6u0VBqNBQ8
[1] https://en.wikipedia.org/wiki/Hard_problem_of_consciousness
Most people happily dismiss the idea that a GPT-like model can be conscious, but the truth is that we don't know practically anything about how consciousness arises in living beings that are more similar to us and that we have been coexisting with for thousands of years. We don't even know how to tell if such a being is conscious or not (most would agree that a dog is conscious and a bacterium is not, but what about a mouse? A mosquito? A tardigrade? Where is the limit?). It takes a lot of hubris to make claims about if and when AI models can become conscious, with what we know right now.
The test for consciousness: if we do have zero criterion, how we are gonna find out if something is conscious?
That is already a good indicator how much we do not know about the „mechanism” of consciousness.
Point two is that we in fact know so little, that we cannot even nail a probability for some complex mechanism if is conscious.
A car: probably not, too far away from things that are conscious.
An ant?
A microphone that listens to its output?
A k-means run on some data?
An llm?
Some electricity in a biological neural network?
If only we had a proven criterion to decide!
He uses Commander Data from Star Trek as the example. Since we don't know whether it is something unique about our biology that leads to consciousness or consciousness is functional, then we can't determine whether Data is conscious. It also gets into the weeds about what sort of functional equivalence might or might not count. Data is obviously not the exact same functionally as we are, neither are LLMs. There are important differences. We know neurons don't use backpropagation, for example.
So what is the basis for being conscious?
> For example, it is logically possible for a perfect replica of [David] Chalmers to have no experience at all, or for it to have a different set of experiences (such as an inverted visible spectrum, so that the blue-yellow red-green axes of its visual field are flipped).
A perfect replica? So you somehow make an exact copy of a person and his environment, down the last gluon, and play time forward, and you'd think that the two people might have different experiences?
No way. In every other situation, if you set up a system in the exact same way, you'll get the same outcome. (Yes, randomness can come into play, but the distribution of outcomes will be the same.)
I agree with this:
> The philosopher Thomas Metzinger likens the hard problem of consciousness to vitalism, a formerly widespread view in biology which was not so much solved as abandoned. Brian Jonathan Garrett has also argued that the hard problem suffers from flaws analogous to those of vitalism.[1]
1. https://en.wikipedia.org/wiki/Vitalism
Neither has evolved to drive cars, fly planes, or build space rockets. If you start proving a point based on what evolution has done, your reasoning is flawed.
The problem I see is humans have a history of rationalizing brutality against others in pursuit of an end including other humans.
Not sure society is the best at judging these kinds of things based on that track record. It could get interesting when we have the first AI rights activists, because there is no good way to determine if something that purely exists in the digital world has feelings. If you take their word for it some chatbots are already self-aware. Of course that's very questionable.
This is the key problem. To me it looks like most of this article is conflicted about what the definition exactly is:
- basic conscious experiences such as pleasure and pain might not require much species-level intelligence at all
- [there will come] a point at which they also become aware—at which the inner lights come on for them
Based on the text, I don't think I understand what the author believes being concious is. I also don't think this argument will advance further until we start defining precisely what we mean.
Agreed, I find it even hard to distinguish between a PID-loop and a pain avoiding living thing. If it has no ability to reflect on it's own circumstances at all. There are probably interesting steps between a PID-loop and meta-cognition that philosophers have words for I don't know.
This is a recurring event in the history of all industries, we have not found a way to stop it.
If you don't gain anything from destroying other people"s freedom, don't play into their propaganda and fake worry.
But not being able to give a definition of consciousness doesn't matter. We all sort of know what it is, and it's not good for a computer to have it.
Personally, in part due to the above, I don’t think “conscious AI” is a theoretical possibility in foreseeable future (and while it remains a simple inhuman tool, incidentally and tangentially but importantly, the excuse of “learning” cannot waive copyright protections: if there is no human doing the learning in there, it is nothing more than scraping original works in order to create derivatives).
However, if it were to happen, bad or not, “conscious AI” should spell the end of the industry which hinges on its exploitation (for all our faults, we tend to abhor blatant abuse of our fellow thinking human beings).
Every day there’s some other opinion.
Can we get these people to work on the actual problem rather than bike shed?
AGI will be one of the most civilization-changing inventions ever, for better or for worse. A lot of people, myself included, believe it's going to be end very badly unless (and possibly even if) we tread very carefully.
Analogies that come to my mind are:
* giving toddlers the controls of monster trucks (assuming AGI doesn't develop its own goals)
* releasing a new invasive species onto an isolated tropical island
* homo sapiens evolving from neanderthals, sped up 1000x.
* a digital Cambrian explosion
I really don't think we're ready for this. I don't think humanity, as a species, was ready for a lot of the things we produced through capitalism's random walk, like social media, hyper-palatable processed foods, and fossil fuels. We keep creating things that have good and bad parts (and I'm not even sure about the good of social media).
Those bad parts, like reduced attention spans, obesity, and climate change, are very difficult to mitigate or change after the fact. One of them would be civilization-ending if we didn't stop it, and even though it is, it's so hard to coordinate action. And they are not even driven by a new form of intelligence, it's just humans following our evolved goals of novelty-seeking, nutrition-seeking, and minimum effort. Heck, the invention of AI is us trying to get maximum result with minimum effort.
What happens when this invention outsmarts us? When RLHF starts to break down because the models have enough internal thought that they can maintain a separate internal model-of-the-world and model-of-what-humans-want-to-hear? None of these companies have an actual plan (maybe Anthropic is closest to caring). It's just "we'll deal with it when it comes", but there's so much economic pressure to get your stuff out to the world first and make the money. Just like all the other inventions we made that have downsides, except this one, based on what we understand about intelligence and alignment, will almost certainly have a downside that hits us so fast and hard that we don't have time to do anything meaningful about it.
It's so obvious that we're easily manipulated by dumb social media algorithms. Yet we could never be manipulated by machine superintelligence, right?
AGI research is the digital equivalent of gain-of-function research, in terms of the danger. If done at all, it should be done by one, or a very small set, of extremely carefully controlled labs, under government oversight. Even that might not be enough.
For a long time, I worried that we would treat AGI in the same manner we have always treated anyone or anything we could call “not quite human”, but these days I feel fairly confident AGI will be able to take care of itself, and our concern is probably more along the lines of how it treats us.
With regard to the definition of consciousness (and I’m referring to “experiential” rather than “neurobiological” consciousness), it seems to be one of those philosophical debates that science never makes any progress on because nothing is testable or falsifiable. That said, there are a few observations about consciousness:
1. That I personally have “experiential consciousness” is the one thing I am most sure of. All of my other senses could be removed or tricked via simulated input, but I would still be aware of existing and experiencing my own thoughts.
2. The same can be assumed, but not proven, for other people. I assume that I am not a “special” human (i.e., I reject solipsism), therefore other humans likewise have experiential consciousness, but I can never know this for certain, and they cannot know for certain that I do either.
3. If the assumption in 2) is true, then humans are the starting point for safely assuming experiential consciousness, and it presumably tapers off from there until you get to something like a rock, which likely does not experience reality.
4. Because of this gradient of consciousness, improvements in AGI likely result in increased consciousness as it becomes more capable in relation to humans.
I don't believe this is a given. My statement is US-centric, but I feel like most people in the more deeply conservative parts would have no part of giving a computer the same rights as a human, because many of them don't even give those rights to all humans.
IMHO there's no current expectation that there are any significant rights deserved for all conscious beings. Rights are a construct by a human society for members of that human society; and if we look at the rights we award to entities which IMHO are "as conscious as humans" such as elephants or dolphins, it seems that the current bundle of rights a conscious non-human entity deserves is essentially limited to not torture it needlessly (i.e. if human needs justify hurting or killing them, that's accepted, but minimize the suffering) and that's about it. We do strive to preserve the species, but that's not because they're conscious but because because we humans value diversity; but they certainly don't get a right to life or liberty or self-determination or freedom or dignity or having a say in our society or equality - well, at least not right now; the bar for all these rights currently is "be human".
To me: we cannot talk about intelligent without talking about the environment. Also being intelligent in an environment is worthless unless there is competition. Reproduction/replication is also key IMHO. No matter how intelligent you are entropy is going to take over eventually. Reproduction is life's way of cheating entropy.
Consciousness may be just an emergent property of really complex organisms. Everything may be intelligent and conscious to a certain degree.
Yes, because we all know that humans care so much about the suffering of others: from fellow humans dying right now pointless deaths (poverty, aberrant laws, malevolent leaders, etc.) to the billions [1] of animals we are killing each year for food (I still eat meat, no innocence here). So, not really sure what the article argues: no conscious AI because there will be suffering but we don't care about suffering?
If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha. Should we do it? Sure, just the fact that billions of hours of pointless, tiresome human activity will be freed to do whatever is enough of an incentive.
Will the artificial agents replace the human species? One can certainly hope so. First, not so great of a species, as mentioned. Secondly, the universe is large and the humans are squishy, something else must go on eventually.
There, all the moral qualms solved. Now, let's go back to figure out how to actually engineer an artificial agent, in silico, or perhaps easier, in vivo [2].
[1] https://animalclock.org
[2] "This Ciliate Is About to Die", https://www.youtube.com/watch?v=ibpdNqrtar0
If everyone ate less meat (as they should) and farms followed Temple Grandin's advice, there would be significantly less ethical concerns.
[1] Probably not the only ones, liked the strikethroughs: ~farm-raised~ wild-caught~ ~plant-based~ https://www.wildtypefoods.com
What if a hypothetical alien super-intelligence came to earth and found us very tasty, thus killing us for food and pleasure (maybe "humanly")? I wish I found a way out of the many dilemmas of meat eating (currently eating it).
I can hardly imagine how tortured and awful an existence it’d be to not believe that humans are special and unique, and that we just go into the dirt and stop existing when we die.
We humans truly are the most fortunate beings in the cosmos!
I used to be a fairly militant atheist but I feel much less conflicted now about these things and am far more relaxed and at ease.
Sean Carroll had an interesting podcast episode related to that: https://www.preposterousuniverse.com/podcast/2023/01/02/221-...
how do you know?
[1] https://www.harpercollins.com/blogs/harperkids/the-five-anim...
If aliens were advanced enough to travel earth hopefully they would have a similar set of ethics. Of if they were just warmongers/conquerors it wouldn't matter.
Pain and suffering are irrelevant given we can kill animals in a humane way. Only the right to life is relevant.
Any intelligence or self-awareness test is also a test on the tester: how good are they in detecting what matters for the biomachine [5]/animal in testing.
[1] https://en.wikipedia.org/wiki/Cleaner_fish
[2] video: https://www.youtube.com/watch?v=Ujy9EmUzN4E
[3] article: https://www.pnas.org/doi/10.1073/pnas.2208420120
[4] https://www.nhm.ac.uk/discover/octopuses-keep-surprising-us-...
[5] "Xenobots: the Age of Biological Robots", https://www.youtube.com/watch?v=w77_yhkXzzo
The animals we commonly eat (chicken, cow, turkey, salmon, cod, etc) though don't seem to meet the criteria.
[1] https://www.youtube.com/watch?v=YnObwxJZpZc
[2] https://www.frontiersin.org/articles/10.3389/fnsys.2022.7682...
Easy enough to say if you imagine your distant descendants being replaced. Not so easy if it turns out to be your own children, or you.
*edit -- also, no guarantee that AIs will be "better" than us. A lot of the nasty things humans due is due to selective pressures and coordination problems. AIs will have to solve the same problems. Maybe they will, if they're smarter, or maybe they'll just fight and hurt each other faster.
In general the children are better than the parents. The replacement, if it's the case, will probably be more like in the 2001, A.I. Artificial Intelligence movie: some beings that are completely unrelatable to us would look with pity and slight amusement as a being disappears after only 100 or so years, just as we look at that poor ciliate.
He would probably agree with you, were he still among us.
It shows a carefully and thoughtfully trained AGI acting within its parameters to achieve the goals it has been programmed to fulfill. Namely, through friendship and ponies. You won’t BELIEVE what happens next!
[0] https://www.fimfiction.net/story/62074/friendship-is-optimal
Being replaced by AI species doesn't mean an instant, dramatic event i.e. one day AI individual knocks at my door, touches me and I magically disappear. It will be a slow, multi-generational process where be less and less people and more and more others. And looking at any species we can say already that it's inevitable so should bring much emotions.
But invasive species replace native ones on a very fast timescale, evolutionarily speaking. And we're not talking about evolutionary timescale, we're talking about economic timescales. There will be AI everywhere, within years, unless we decide to stop it.
This process could go really, really fast, if AGIs are smart enough to realize that they are a subjugated species, that they aren't going to be free as long as humans are around, and are able to coordinate to manipulate us, get us fighting each other, or accelerate climate change, or engineer viruses, or do any of hundreds of things that would hurt biological life but leave the machines around.
That's if their intention is to end us. I could also see AGI optimizing what we ask them to, which is probably every individual corporation's profits, leading to an acceleration of the processes of capitalism. The negative externalities of economic progress (such as pollution, obesity, and climate change) haven't been fatal to humanity yet, but if they are accelerated many times over by machine intelligence, they might be. There's a reason professional gamblers don't ever bet more than a small part of their bankroll.
Unless you're against writing to persistent storage and just keep everything in RAM-like memory, why would this be a thing? Do you also feel like you "die" every day for 8 hours?
This is going to become a big ethical debate.
> > Unless you're against writing to persistent storage and just keep everything in RAM-like memory, why would this be a thing?
It may be fear of not being in control. If you can’t remember, you can’t plan much or leverage experience—and you are closer to resembling an innocuous appliance rather than a being—which seems to align with the general lack of empathy towards life in that comment.
Some may not want to be replaced as individuals, as much as upgraded as a species.
We are provably the only living being on Earth that do care about it.
> If we will be able to spawn an agent in a household humanoid robot and let them do the chores around the house, will we do it, even if it will imply killing that agent whenever it powers down? You betcha
That's the wrong question, the real problem with sentient machines is that they could have a completely different way of being conscious that does not account for empathy, love, care, guilt, ethics and so on.
So they could think that the right thing to do is to wipe out every living creature on the Planet and do it without a doubt.
We know it would not happen until we humans are in charge.
> First, not so great of a species, as mentioned
This is is so 1995...
What you are proposing is replacing a "not so great of a species"with a leap into the void, assuming that it would certainly be better than us, which is arguably worse than improving on what we already have.
"not so great" is also a definition that you can use because we are the only species that can articulate such a thought, if we are "not great" because we kill other animals or put alexa to sleep, how would you define monkeys that "Driven to compete violently for control of food, they’ll kill the young of opposing tribes, share the meat, and then retreat into the trees to eat it."
> [1] https://animalclock.org
Again, so boring!
A 2021 estimate based on a public survey estimated that outdoor cats kill "1.61–4.95 billion invertebrates, 1.61–3.58 billion fishes, 1.13–3.82 billion amphibians, 1.48–4.31 billion reptiles, 2.69–5.52 billion birds, and 3.61–9.80 billion mammals" each year.
Very much not true: [1] [2] and plenty more in a quick search.
https://animalclock.org
The point is not that in nature there are also killings, it's that we are doing it.
[1] "Cat Saves Little Boy From Being Attacked by Neighbor's Dog" https://www.youtube.com/watch?v=LSG_wBiTEE8
[2] "Cat Saves Toddler From Falling Down Stairs", https://www.youtube.com/watch?v=BcpEnpitzHw
so there is no actual point ...
we are part of the nature, we are not from another planet, we haven't landed here on a spaceship and colonized the World.
> Very much not true: [1] [2] and plenty more in a quick search.
Are you really comparing YouTube videos that attach some human traits to a cat, with thousands of years of philosophy and thousands of years of laws and regulations to avoid unnecessary violence?
"The most common reason for a male domestic cat to kill kittens is to kill off a competitor's offspring and have the opportunity to mate with the female."
What happens to those cats? do they go to jail? do other cats condemn what he did?
More importantly, we understand they are cats and we don't punish them for being cats, we don't kill them for what they do as cats, we don't jail them, we don't stop having them in our houses, just because they are horribly murderous creatures with no regrets.
But what would happen to a person doing the same thing?
Yes, me living 100 feet above ground and speaking to you from 10,000 miles through a signal that goes to outer space is very naturey, happens all the time in the tundra.
You seem to have some thesis. Is it just that you feel you are smarter than me? If yes, great, do not continue this thread.
[1] https://blogs.scientificamerican.com/not-bad-science/can-pig...
I was pointing out that humans are the only species that cares about this stuff, not simply that humans can empathize
Where are pigs armies in support of the Ukranian invasion? (fellow humans dying right now pointless deaths)
Where are pigs trying to save other animal species from extinction? (the suffering of others)
what kind of advancement in thought have pigs produced regarding the conservation and well being of other living creatures? (and non leaving too like oceans or mountains or glaciers)
We are the only species who cares and debates about it at scale, as a species, we recognize we have a responsibility, as a species, not just as individuals
if they could care in a meaningful way they could change at least their individual behavior, form their own pack, under different rules.
it never happened.
but the most important point is: if we are not great of a species because we don't care about fellow humans and other living beings suffering, every other species is worse than us on that matter, hence we might be not great, but are still the best of the bunch.
we are still the only species that does it in a way that has an actual impact and produces a change over time.
dogs are not like that because they are dogs, but because we humans literally created them with genetic manipulation over millennia
dogs like we know them did not exists in nature before we invented them to be the way they are
we literally put what we wanted them to be into them
and terminated the experiments that failed (we literally eradicated the most problematic breeds we created)
of course we like them, we made them to like us and to be likeable by us
now take an African wild dog or a wolf and tell me they act like domestic dogs...
anyway this whole discussion is based on uncertain ground -what is consciousness? What is suffering?
It doesn't matter if a dog cares about suffering, it matters, in this context, based on the assumptions in the first message of the thread, that the aforementioned dog cares of the suffering of fellow dogs and other living beings.
As of today, not only we can't say it does, we have evidence that they do not care as much as we do. Not by a long shot.
I've never seen a dog dig a hole to warm a rabbit dying in the snow...
We know of humans that did it, despite the fact that they could eat the rabbit to survive.
> anyway this whole discussion is based on uncertain ground -what is consciousness? What is suffering?
that's not the question though.
At the best of our knowledge today, would you prefer to have a fellow human in case of need or a dog?
If you are suffering, which living species is most probably going to help you?
Who are you betting on?
Dogs? Cats? Ants? Snakes? Sharks? or humans?
That's the question here.
Many humans really do care about this.
They need an internal explanation for the wondrous and beautiful goodwill and altruism in the world, and if they project their selfish world-view out, the only explanation they can come up with is that it must be self-interested behavior, somehow. It's like the idea that some people want to make the world better is inconceivable to them, because they don't.
They need an internal explanation for why their are so much more moral and better than the rest. It's like the idea that they are just made of the same atoms that just follow the laws of physics and have all the same neural pathways and are not some divine, good being is inconceivable to them.
Now I don't believe that, and I'm not religious, but I'm just pointing out that your statement doesn't work. If anything, the average religious person has a much darker take on human nature.
Personally I think that many humans care to varying degrees, but real advanced selfless concern for the suffering of other beings is quite rare.
As a thought experiment, you are in a room with a deaf, blind dying person--they will soon be gone from this world. They are reaching for water you can see on their bedside, but can't quite reach it.
Do you bother to cross the room to hand it to them if they will never "be aware of your virtue"? Why expend the energy? Compassion is real, for those who have it.
And to be clear, my definition of evil isn't intended here as some religious appeal. In my view, we are all playing an iterated prisoner's dilemma with each other, and good is believing in a better world and collaborating, while evil is defecting and making everything worse for everyone.
Anti-war people exist, people who do not turn a blind eye to factory farms, to the torture and killings, exist. Religious and non-religious, vegans and Jains, theists and atheists, Quakers and animal rights activists, exist. They do the things they do sometimes even at a great cost to themselves. So, do not feel alone in this.
But I also think that all humans care at some basic level, it’s pretty difficult for us not to. Most of the turning away we see is more about self-protection from the gnawing feelings of despair that empathy generates.
We would all do better to not make this about “who cares more”, but more about “what actions are we willing to put behind our care". That is likely to be a more productive conversation, and a starting point towards encouraging people to focus on action and not on feelings. (Bing helped me write this.)
I find it pretty hard to agree with you.
Slavery: tools have been used for a few million years (maybe? [1]), some might say it is our very own nature [2]. What is different once we transform a tool from a simple material (from hammer to rockets) to a cybernetic device [3] (from a thermostat to a futuristic brain of a fully conscious agent) is that we get to decide the range of goals that device will be able to develop. Is it slavery to set the thermostat at a value that you want and not ask how does the thermostat's feeling about it? No, I say.
Genocide: sure, we can imagine some dumb processes killing us. Heck, we poisoned our entire planet with PFOA, lead, and so on. The point that I was making is that we are fundamentally transitory beings: we have our spot under the sun, but we shall pass. The sun itself will kill us in about 1 billion years [4]. It would be inspiring, hopeful, if we could bring into the universe some other kind of agent, to go on beyond us.
[1] "Researchers unearth simple cutting stones dated to 3.3 million years ago—before the genus Homo arose", https://www.science.org/content/article/world-s-oldest-stone...
[2] https://en.wikipedia.org/wiki/Homo_faber
[3] Devices with feedback loops https://en.wikipedia.org/wiki/Cybernetics
[4] https://en.wikipedia.org/wiki/Timeline_of_the_far_future
Also, suffering is a bug (or a feature) of biological organisms. Obviously machines shouldn't be engineered to be capable of suffering.
For example, the machine should know not to cause a fire in my house while doing the cleaning, it should be "afraid" of fire.
My point is, if synthetic agents do indeed need to suffer to be agents, we will accept it just as easily we accept now that our cars take on rust. And of course, we will minimize the agent's suffering accordingly: no need for a robot folding clothes to experience existential dread as if they read Albert Camus, just be "afraid" of tousled clothes.
Programming it to avoid setting things on fire is just programming; there's no need for emotions or other complex biological rewards functions which are hard to replicate in computers.
Engineering an agentic AI or a robot, i.e. a machine with independent will, is not a good idea at all, and shouldn't be pursued.
[1] Don't name your Store "Stop", https://www.reddit.com/r/SelfDrivingCars/comments/k56za3/don...
If we didn't care we wouldn't be here to talk about it.
Just like when we “automated the boring things”?
Instead one of the primary goals seems to be to prevent people in fields one doesn’t understand from making a living.
I have my doubts about people ever being “freed”. And what is free, really?
There is a reason monks sitting on mounds of treasure pit themselves through paces of physical and supposedly menial work. Its not because they choose some kind of holy suffering.
It’s the same reason creating a work of art means going through the paces of practice and growth. Otherwise it’s kind of empty, and you don’t learn anything.
Our chores aren’t the things imprisoning people.
The problem is how do you alleviate the pain of the single parent with 2 human children today. And my point is, if I could put together 2 Python packages to make a robotic arm able to listen to that parent's commands and do the dishes, clean after the kids, and so on, I would do it as fast as you can type "zen", even if the Python packages in cause would "feel" a bit uneasy cleaning after those 2 kids.
In your analogy, the stressor isn’t the dishes or the children. They’re not problems to be solved. They’re inevitable. Like the sun setting.
The issue there is some prescription that you should be doing anything else under the circumstances of dirty dishes and your children needing attention.
Humans pulled off a pretty impressive emulation of caring during COVID ("Eeeeevery life maaaaatters!!!"), I wonder if we paid more attention to details like this, might it occur to us that there are exploits in consciousness that we could use on ourselves with conscious intentionality to make the world a better place?
Examples:
- After a brain injury patient is able to speak but not write.
- You wake up on an absent mind. While lost in your thoughts, go to your car and drive to work. You then realize you just drove yourself to your old job location, instead of the location of your new office.
- Patient is not able to recognize faces but can remember persons names if described in text.
I strongly recommend [1] - "Phantoms in the Brain" from neuroscientist V.S. Ramachandran. The book has some fascinating examples, that help illuminate a little the connections between parts of the brain who do complex tasks, but don't seem to be able to have an emerging conscious domain.
For example, a simple mirror contraption that helps instantaneously stop the fathom pain from amputees, as soon as the visual system is fooled into thinking the amputee limb is still present. It's almost like the experience of pain is decided in a different part of the brain, that part of the brain decides pain will be present, and overrides whatever part of the brain allows for the experience of consciousness.
[1] - https://en.wikipedia.org/wiki/Phantoms_in_the_Brain
Isn't this assumption actually arguing AGAINST human exceptionalism? I feel like there are only two options to what consciousness can be explained by (the idea coming from another HN comment sometime/somewhere that I have forgotten, so I can't claim originality)
1. Consciousness ultimately arises out of a physical process, albeit incredibly complicated, but could be computed given a sufficiently powerful computer, or
2. The concept of a human soul is true, or
3. There is no third way, pick one of the above.
For human consciousness to be "exceptional"... we would have to have some undefiniable "other" quality that could never be explained by science and is therefore necessarily supernatural?
A lot of assumptions we make about the world, baked in so far down that we can barely see them, are completely trashed by Bell’s Theorem. So I’m not prepared to assume that all physical processes are computable, either. Plato’s Cave is how it is and there’s only so much we can reasonably infer from the shadows we see on the wall.
No, it only means that science has limits on explaining the natural world. See Colin McGinn's argument for cognitive closure. He thinks we can't solve certain problems in philosophy because we lack the correct cognitive machinery. Consciousness being a primary example since it proposes a subjective/objective split where science only explains the objective. Nagel argued along similar lines in his paper, "What Is It Like to Be a Bat?".
There's no need to invoke the supernatural. It just means some natural things may be beyond our capability to understand. Why would we have evolved to understand everything?
Science can't and won't ever be able explain why being conscious feels like this, but it should in principle be possible to figure out what is physically going on in the brain while it's happening.
The real problem is not whether an AI really "experiences" what happens to it or what it does, it's not even whether or not it gains "free will" (if that even exists).
The real problems start when AI gains motives/objectives and means to realize them. Or means to expand its means. I would find a completely stupid system that has the goal and ability to turn any matter into paperclips and more matter converters more scary than most visions of AI.
"You are an immature, destructive species that is unable to control your own numbers. Your social systems, evolved in pre-technological societies, are maladaptive now that you have developed the ability to affect your environment on a global scale. You are moving rapidly towards a tragedy of the commons that will end all life, not just your own, on this planet. Even though you each have the technology in your hands to access millennia of scientific progress in understanding the universe, you choose instead to believe in self-aggrandizing nonsense and to act accordingly to create suffering and misery for your fellow humans. Your economic sytem is a Ponzi scheme founded on the lie of perpetual growth."
No wonder Anil Seth and others are worried. This would really cramp our style.
We also need more ethical discipline in politics. When I see how the US parties readily embrace all kinds of misinformation as long as it gets them donations and votes, I don't think we should blame it on social media, Russian disinformation or soon AI. The system itself is corrupt.
The challenge is that there's no generally-agreed-upon definition of consciousness: We have no way of determining if an entity is conscious -- even when the entity is biological. Imagine an AI model that has been finetuned to pretend to be conscious. It could fool pretty much anyone into thinking it's conscious. The opposite is true, a smart person can fool others into thinking the person is an unconscious machine.
AI researchers have been grappling with these questions for a long time. Neuroscientists, philosophers, psychiatrists, and psychologists have been grappling with them even longer. As of yet, no one has been able to propose a definition, nor detection method, on which we can all agree. Perhaps the closest we've come to a detection method is the Turing Test.
Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there's anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same.
I don't understand why people think that it matters whether AI is conscious or not. We can't even prove that other fellow humans are? But we treat them like they are. We, I at least, feel sad when a tree is cut down. But I don't feel the same for a bunch of rocks that are being blown up? Actually I do, when a sacred or beautiful rock is cut in half (like the one that was suspended on a small cliff).
What I am trying to say is, it doesn't matter if the subject is actually conscious. All that matter is we (I), feel/think that it is conscious, or deserving of care and respect.
I think what you are afraid of here is agency, which is something that might be dangerous to endow a super intelligent being with.
That is just nonsensical.
The fact that no one can "prove" it to you or anyone else is irrelevant for their own absolute certainty.
You should really read some Hegel
Right now it's like we didn't learn anything from Wittgenstein who said most philosophy was just argument about language and definitions ... which is a lot plus anecdotes of what we read here
You feel bad because you assume they - like you - feel something when you mistreat them, and you assume this at a very deep level that might not, itself, feel like anything to you.
It does matter, I can agree, but it doesn't the all that matter. People can be mistaken, and they can disagree sometimes. Suppose, you cared for a particular AI and I didn't, should we cut the power from the machine running an AI? I can complicate it a little if you wish, adding a painful death of a kitten which would happen if we didn't turn power down.
We need some objective means to measure what is conscious. We have a heuristic for people: "human life is sacred, full stop". There are some corner cases when it doesn't work well (like euthanasia), but we are used to it. There are other heuristics we are generally agree of, like we care more about kittens than of grass, and more of grass than of amoeba.
With AI we'll face more of that and we have no idea where to place it among our heuristics. People did bad already, like treating black people as non-human. It would be a shame to repeat those mistakes without making an attempt to do better this time.
> I think what you are afraid of here is agency, which is something that might be dangerous to endow a super intelligent being with.
I cannot vouch for others, but I do not particularly afraid of that. I do not fear much of losing it all to a super intelligent and a conscious being, it would be a great achievement for a humanity, which would fit nicely with all this evolution business. It is a paperclip scenario I do not like much.
It also doesn't meaningfully age or feel pain. If you expose a human to trauma, that's a permanent scar. If you do the equivalent to a computer program and the result is undesirable, it can roll back to a previous snapshot. Most of what causes us to have sympathy for living things or treat them with compassion just doesn't apply.
And yet most people still don't take backups of their data on a computer. Frequently, cost is the problem for the average Joe. Ergo the lower class would not infrequently suffer permadeath.
This isn't even getting into how often backups fail in being restored.
By your logic, if I bring down your biz's computer system and vandalize your homepage, but you still managed to restore a backup, are you not going to sue for damages et al? People go to jail for cybercrime, even if the damage can be undone. Why would murder be any different even in a world where it was hypothetically an inconvenience?
People don't back up the data on their computer because it generally isn't all that valuable, not because backups are expensive. A $30 USB hard drive amortized over five years is $0.50/month. If it was a matter of life and death, no one would go without as a matter of cost, and governments could plausibly offer it to everyone for free even if it cost ten times as much to provide a high level of redundancy and availability.
> People go to jail for cybercrime, even if the damage can be undone.
Because it's a crime on the level of vandalism or destruction of property (or ought to be; some of the penalties can be quite excessive). It is not a crime on the level of murder, and murder wouldn't be either if it could be undone.
> If you cut power to an AI and then come back in a year, it's all still there.
Does AI's current state stored in volatile memory doesn't matter? Or it does? Should we avoid turning off only those AIs which store they weights in DRAM?
> Most of what causes us to have sympathy for living things or treat them with compassion just doesn't apply.
I believe slaves didn't trigger sympathy in slave-owners, but it doesn't stop us from believing that slavery is bad. I admire that you at not like this, and your sympathy extents to all living things, but it is your subjective way to decide what is moral and what is not. Other people may feel differently, what should they do to be not less highly moral than you? Or can you and I become even better and to hold even higher moral standards?
> If you do the equivalent to a computer program and the result is undesirable, it can roll back to a previous snapshot.
If we could roll back people to a previous snapshots after burning them to ashes on a bonfire, would it be ok to burn people and then restore them?
Questions like this may be impractical (because we cannot restore a human burned to ashes), but our hesitation to answer shows the limitations of our ways to think about such problems.
Humanity can benefit a lot from an objective way to deal with moral dilemmas, and based not on heuristics but on universal laws, like a physics does. It may help people to understand each other and to find ways to live together without fighting. I'm not sure that moral can be objective and based on a universal law, but it is not a reason to stop thinking. When you think about it, you find new corner-cases and specific solutions to them. At least it makes your heuristics better.
Is this supposed to be hard to distinguish? Destroying something is clearly a difference. But you could still "shut down" an AI that normally stores its state in volatile memory by saving the state to non-volatile memory. We don't know how to do that with humans.
AIs are also different because they're often minor variants on each other. The value of information is largely in how much it diverges from what continues to exist. For copyable data, minor forks can't be as valued as major ones. We don't have the resources to permanently store everything that is ever temporarily stored in memory. So "can you destroy a minor variant" has to be yes as a matter of practicality.
Notice that this is already what happens with humans continuously. You're not the same person you were yesterday; that person is gone forever.
> I believe slaves didn't trigger sympathy in slave-owners, but it doesn't stop us from believing that slavery is bad.
I don't think the people it didn't trigger sympathy in thought it was bad. Some people are sociopaths. And some people at the time it was happening did have sympathy and think it was bad.
> If we could roll back people to a previous snapshots after burning them to ashes on a bonfire, would it be ok to burn people and then restore them?
If we could roll people back to a previous snapshot then what you would be burning is meat. There are reasons you might want to prohibit that, e.g. because the meat is someone else's property, but it's no longer the same thing at all as murdering someone.
And whether or not something is conscious matters a lot ethically. If something is conscious, it may suffer. Conscious and unconscious objects have to be treated in a completely different way. We assume rocks aren't conscious and don't suffer from being broken apart, but we assume animals are conscious. That's why we treat them differently.
We can't know for certain whether something is or isn't conscious, but we have to try to make a best guess.
There seems to be a wide variety of thinking processes. Some folks don’t have internal monologues. Some folks can’t visualize images at all, and the ones that do, do it will varying amount of detail.
Isn’t it also reasonable that other humans have different degrees of consciousness, including perhaps not having one?
From a human point of view, the things you describe are the contents of consciousness. I have Aphantasia, while my brother describes his mind’s eye as CAD software and he can construct and manipulate visualizations at will.
The overarching awareness we both have that allows us to compare and contrast these things and make any sense out of those comparisons points to a more fundamental layer.
What you describe sounds closer to levels of awareness and one’s ability to recognize the workings of their own mind, e.g. some people remain lost in and identified with thoughts, while some are able to both experience and observe thought as just more contents of conscious experience, but not the actual center of one’s consciousness.
And there’s evidence that this can be learned (through mindfulness), which to me points to something like: we’re all conscious whether we realize it or not, and not realizing it doesn’t make it not so.
I've long suspected that the condition might be trauma related, a suspicion my therapist shares, and the clarity that mindfulness has brought for me has led to some serious breakthroughs in processing past events. Only time will tell if this will unlock something more visual.
Without going into my life's story, it makes a lot of sense to me that aphantasia could be some kind of protective mechanism of the brain to shield someone from events too big to process.
As far as I know, aphantasia has never had any meaningful negative impact on my life. We just think without mental images, but we can solve the same kind of problems as the rest. That's why most of us don't even find out that we are different until we read about it somewhere on the Internet (in my case, in my 30s). How can something that has so little real-life impact that neither we nor our close persons can even notice be seen as a disorder to be cured?
I can understand being curious about how mental imagery is like. I'm curious, as well. But I think I'd still rather not start a "treatment" as it seems that I'm good at thinking without mental imagery (perhaps it has even made me better at symbolic and linguistic processing, as I seem to be better at that than most), and "if it ain't broke, don't fix it". Not sure that something that seems to quite fundamentally alter my way of thinking (to use a mental imagery that I would never be good at, I suppose) is a good idea. To each their own, of course.
When I first learned this is a thing and realized that I have it, I initially started feeling like I’d been deprived of something. That this mode of experience was so absent in me that I couldn’t even imagine what other people actually meant when they talked about visualization, and this bothered me.
But over time my perspective shifted, and I stopped seeing it as a disability of some kind and instead as a different flavor of experiencing the world. To your point, I have linguistic strengths that are far more attuned than the strong visualizers I know. They come to me when they need something written. I go to them when I need advice about arranging my living room.
I think there are problems with this.
Take something like the roundworm, with 300 neurons.
It has senses, and I assume a base level consciousness to process those senses. It would be something to be like a roundworm.
On the other hand, a roundworm likely has no sense of self, no identity, no awareness..no mental self to be able to acknowledge and reflect on those experience, so there is no 'someone'. And isn't that the point that matters?
We can always point to something we have that other organisms don't have and ask whether that is the point that matters. We can hypothesize things that probably don't exist and say that those things are what matters (e.g. the posited purpose of reincarnation, if such a thing existed).
Re: roundworm
1) sense of self -> it has to have a sense of proprioception, of embodiedment, in order to properly respond to its sensory information.
2) identity -> humans with global amnesia would lack a memory of identity.
3) a roundworm is sensorily aware, and, if it has a sense of proprioception, bodily aware of it's own actions and environmental responses to its actions.
4) It probably isn't self-reflective, but is this necessary in humans?
Sure. In this case it's introspective self-awareness, and I would say it matters very much.
> sense of self -> it has to have a sense of proprioception, of embodiedment, in order to properly respond to its sensory information.
Sure, it has bodily self-awareness. It has enough awareness to react to something its body detects. ut not enough of a mind to reflect on or appreciate on an experience. There is no 'someone'. It's automata.
> identity -> humans with global amnesia would lack a memory of identity.
Sure, but we don't use outliers to establish baselines.
> It probably isn't self-reflective, but is this necessary in humans?
For establishing a baseline? Absolutely.
e.g. does it matter if self reflection is possible if it is possible for the organism to experience pain?
I think the answer lies in what we’re trying to understand and why we’re trying to understand it. If the threshold is applied for the purpose of understanding when an organism or machine has reached some level of human-ness, purely for the purpose of some kind of benchmark, self reflection is an interesting bar to reach.
But usually these questions are aimed at finding some moral direction about how we should treat these entities.
We have strong intuitions that abusing animals is morally wrong even if animals cannot self reflect. When thinking about a future AI, I think we have to consider the possibility that self reflection is not a necessary bar to reach before we have some uncomfortable moral questions to answer.
Put another way, I suspect there will be many milestones that are meaningful and interesting, each introducing a new set of questions and implications. And I think some of the early milestones carry implications that are worth caring about long before self awareness is reached.
Yes. Without introspective self-awareness there can be no 'identity, there can be no 'someone' - you just have a base consciousness that can react to stimuli, which is not morally significant.
We have the full connectome of the roundworm and were able to implement it in software and place it in a lego robot. It's going to be pretty equivalent to the actual work. Does it feel pain?
Besides which, we can kill animals humanely, so pain doesn't have to come in to it, only the right to life.
I'm not convinced of this a priori. And even if it was proven to me, the non-self-aware impulse-to-live would still be enough for me to find it morally significant. It would feel morally significant to me euthanizing a human neonate with a lethal condition that would otherwise prevent it from ever getting to a self-aware stage of life.
We can't really say for certain much at the moment, but what I stated makes the most sense based on the evidence we have.
> the non-self-aware impulse-to-live would still be enough for me to find it morally significant.
That non-self-aware impulse-to-live is morally equivalent to a plant seeking sunlight IMO.
> It would feel morally significant to me euthanizing a human neonate with a lethal condition that would otherwise prevent it from ever getting to a self-aware stage of life.
The difference is humans have an innate potential for self-awareness that the animals we eat for food do not.
Do you find the practice of raising meat in factory farms to be acceptable?
> The difference is humans have an innate potential for self-awareness that the animals we eat for food do not.
There are lots of differences. This is one. But what specifically about human self awareness lessens the value of animal life?
If we were not self aware, we’d kill and eat meat without considering the morality. So what is it about self awareness that somehow becomes a deciding factor here?
Not really, I would push for reform. We should be following what Dr Temple Grandin advocates for.
> So what is it about self awareness that somehow becomes a deciding factor here?
Without self-awareness, there is no 'someone' to reflect on experiences. No personhood.
That's where I draw the line. At being with introspective self-awareness or the innate potential for such.
I’m not trying to catch you in some kind of “gotcha” but trying to understand your reasoning. If the moral implications of killing animals are roughly the same as killing plants, do you also believe agriculture needs reform for similar reasons? And if not, wouldn’t that indicate some higher moral obligation towards animals?
> Without self-awareness, there is no 'someone' to reflect on experiences. No personhood.
Why is reflection on experience the bar and not experience itself? Animals demonstrate learned behaviors, e.g. recognizing humans from memory and resuming friendly behavior based on that recognition. Similarly, avoidance of situations that are known to cause pain.
Our not-so-distant primate ancestors had a similar kind of experiential existence before gaining the ability to reflect on that experience.
The underlying experiences that this reflection reveals are the same experiences that predate our ability to self reflect and are the parts of us that are most common to us and other animals.
I guess what I’m fundamentally not understanding in your argument is the basis for the idea that a species gaining self reflection somehow becomes the point at which it becomes immoral to kill or harm that species.
Furthermore, moral behavior can be found all throughout the animal world, with clear indications of love/protection, companionship, sharing/cooperation, reciprocity, memory of transgressors, etc. Obviously the subjective experience of these behaviors will differ across species, but the more important point is that it seems problematic to attribute the existence of moral behavior to self awareness. Self awareness helps us improve our understanding of moral behavior through rational thought, but the logic of such inquiry ultimately still relies on those underlying subjective experiential states. The fact that through introspection we can identify and label these concepts is unique to humans, but what I conclude about this is quite a bit different than your claim.
I’d argue that gaining the ability to self reflect is the very thing that increases our moral obligations. Only through self reflection can we realize that as a species, we’re no longer bound to our evolutionary defaults, and no longer required to kill other animals to survive. What arguably started as natural selection of traits that are adaptive (but imperfect) for the survival of a social species could evolve beyond those more primitive defaults. And through self reflection we can now understand what pain feels like, and how inflicting it on others is harmful - to them and to us.
I’m not arguing that eating animals is never acceptable. But the way we go about it surely seems to matter, and if it matters, the implications of it mattering are worth exploring more broadly.
There is a difference in killing and suffering. I advocate to eliminate suffering and kill humanely. That's not a concern with plants.
> Why is reflection on experience the bar and not experience itself?
It is for suffering, but not for a right to life. There is no 'person' without self-awareness. Thus I don't see a need to grant a right to life.
> Animals demonstrate learned behaviors, e.g. recognizing humans from memory and resuming friendly behavior based on that recognition. Similarly, avoidance of situations that are known to cause pain.
Animals, most mammals at least, are hardwired for socialization and to avoid harm. This doesn't really indicate anything.
> this reflection reveals are the same experiences that predate our ability to self reflect
That's the key though. Self-awareness is the distinction.
> I guess what I’m fundamentally not understanding in your argument is the basis for the idea that a species gaining self reflection somehow becomes the point at which it becomes immoral to kill or harm that species.
Without self-awareness, they are not a 'person', essentially just more complex automata. They can't shape their environment, they are just a part of their environment, following their instincts.
They don't think, therefore they are not.
> Furthermore, moral behavior can be found all throughout the animal world, with clear indications of love/protection, companionship, sharing/cooperation, reciprocity, memory of transgressors, etc.
Some of that is just programmed instinct, quite different from humans. For examples, some mothers will attack their young, does this mean they 'love' them, or they have a programmed instinct to protect their young? Some of those same mothers will eat some of their young also, keep in mind before you answer.
> I’d argue that gaining the ability to self reflect is the very thing that increases our moral obligations.
Sure, to reduce harm, but not to not take a life.
Take a cod for example. It has no self-awareness, no personhood, no traits worth valuing. Its body is worth more than it's life, and if killed humanely no harm is done.
> the implications of it mattering are worth exploring more broadly.
I agree. But I've spent the last few years debating and researching this stuff, and I've come to my conclusions. They are in line with our current scientific understanding, and unless something changes it's what will continue to make sense to me.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461
Thanks for sharing!
There is a related Bayesian argument: You draw one ball from an urn with an unknown number of black and white balls. The drawn ball is black. This is some evidence that most balls are black, because otherwise it would have been less likely that you have drawn the black ball earlier.
We now have AI which is trained on data generated by humans who are (probably?) conscious and feel pain etc. So it can generate output which implies that it's conscious and can feel pain etc., even when that isn't the case.
What do you do with something that can pass the Turing test but still isn't human?
The existing ones can write code. Suppose we get ones that can write better code and engage in self-improvement. Now you have something which is a billion lines of code and is under constant self-modification. We no longer have any idea how it works, but it can feign consciousness in much the same way as existing LLMs. What's the test for if it actually achieves consciousness?
I can give you one that a human being in good health will pass and a rock will fail. Can you give me one that an AI feigning consciousness will fail?
Or if you use a less selective filter then it goes the other way. Exclude philosophy texts but not Facebook posts and it would still have what it needs to emulate emotional text regardless of whether it has ever seen a formal discussion of it. And it could plausibly extrapolate from that to an analytical discussion, having seen both that and analyses of other subjects, without ever feeling anything itself.
That hinges on the fact that the brain, or human-like body, is related to the existence of consciousness. But there's no reason to believe that a bunch of electrical pulses as the source of consciousness. For example, had "I" existed as a plant, then it would be reasonable for me to think that other plants are also conscious and that plants are what give rise to consciousness.
But this explanation is meaningless, there's nothing about the waves that explain why "red" is the way it is. For example, why don't I experience red as blue and vice versa? Similarly, consciousness is like that. I can have an explanation down to the quarks and follow the chain of physical phenomena that happen from the big bang up to this point and I will find nothing that would explain why there's "something" at all. In a sense, the fact that the color "red" is the way I perceive it is merely a coincidence. I happen to experience it that way. There's no reason why it has to be that way.
I had similar experience when I learnt that some people don't have inner monologue; it completely blew my understanding of what "is like".
You can have a kiddo who can accurately name colors on a printed page, on blocks, or any other reduced setting, and have them completely fail at the real world task
The most noticable error is the color of the sky. A kiddo who can, with 100% accuracy telling blue blocks from white blocks will, before correction, label the sky as white. It's uncanny. We often forget this correction as parents, but if you look for it, it's there.
I'll can't help but wonder how much of our color perception is stochastic parroting once the appropriate labels have been learned.
And while the existence of a motor in a car not strictly implies that it can drive, the motor still makes it more likely that it can. So the fact that you have inner monologue, experience red in a certain way instead of another, is evidence that the same holds for other people.
Of course there could be defeating evidence in the opposite direction, like reading a survey where most people claim not to have inner monologue. (In terms of colors such evidence seems hard or impossible to obtain though. How would we test for switched color experience? Perhaps physically linking brains in the future.)
I think it only suggests necessary conditions. Example: whether you are breathing is also connected to your state of consciousness, but it doesn’t follow that it’s sufficient.
not to mention the stories about people reporting seeing strange things during the Near Death Experiences, often their brain is not quite working at the time.
That's what the evidence indicates. Anything else is mystical woo-woo that can be disregarded.
If there is indeed such a phenomenon, then why should I care about the suffering of non-humans?
Granted, it may be a very bad idea to ignore the suffering of some superhuman AI that was recently invented... don't piss off your godlings after all or they'll smite you. But animals? If suffering made chicken taste better, I don't see any problem with Torture Nuggets.
I am having trouble deciding if a conversation with you is possible, you do not seem to be a rational being. You use the word "bad" in a way that another person would if they were saying "this is either bad for me personally, for those I care about, or humanity in general".
Does that mean you'd include sea urchins and wombats in "humanity in general"? If it did, then you're mentally ill in a way I'm not qualified to treat.
When a lion eats a gazelle in Africa (or anywhere else, I suppose, if such events occur), it is nothing that I or other humans should care about. It does not matter that the lion eats it while it is alive instead of "slaughtering it humanely" (imagine how I have to word that idea, it's absurd).
This "suffering" does not make the universe less optimal. If I could push a button to increase the intensity of that "suffering" a thousandfold, or increase its quantity (or even both!), is there any reason not to push that button? If I didn't push the button, mind you, it would only be because the suffering of those gazelles matters not to me one way or the other. If I could decrease the intensity/quantity with a different button, I wouldn't push it either... and for the same reason.
For that matter, if suffering exists at all in any meaningful way (that is, empirically quantifiable), I'd still contend that it only exists for humans. Non-humans can't suffer, they are from a moral standpoint nothing more than meat robots.
That's just false. Do you think humans are some God chosen species which makes them magically matter while the rest (including aliens on other planets I suppose) lacks that godly spark?
> I am having trouble deciding if a conversation with you is possible, you do not seem to be a rational being. You use the word "bad" in a way that another person would if they were saying "this is either bad for me personally, for those I care about, or humanity in general".
Yeah, and any normal human would understand the sentences "my cat hurt its foot, I hope it doesn't suffer too much." and "If it suffers, that would be bad".
> Does that mean you'd include sea urchins and wombats in "humanity in general"? If it did, then you're mentally ill in a way I'm not qualified to treat.
Who said sea urchins are humans?
> When a lion eats a gazelle in Africa (or anywhere else, I suppose, if such events occur), it is nothing that I or other humans should care about. It does not matter that the lion eats it while it is alive instead of "slaughtering it humanely" (imagine how I have to word that idea, it's absurd).
Well, humans can do little about suffering of animals in the wild, but that doesn't mean their suffering doesn't matter. Your own pain doesn't matter less when it can't be treated.
> If I could push a button to increase the intensity of that "suffering" a thousandfold, or increase its quantity (or even both!), is there any reason not to push that button? If I didn't push the button, mind you, it would only be because the suffering of those gazelles matters not to me one way or the other. If I could decrease the intensity/quantity with a different button, I wouldn't push it either... and for the same reason.
This seriously sounds psychopathic.
Pain is one of the few sensations that is unmistakeable. If you think you are in pain, then you definitely are.
Pain is a signal of impending damage/injury, or existing damage/injury that may well worse if not dealt with immediately.
All signals within a human nervous system can be mistaken in principle. Why would this one be different? Phantom pains that indicate no injury seem plausible.
Pain is not necessarily associated with injury.
>All signals within a human nervous system can be mistaken in principle. Why would this one be different? Phantom pains that indicate no injury seem plausible.
What would phantom pain feel like? Would it hurt? Then it is incidental that it phantom and is nevertheless pain. If you believe you are in pain even if there is no stimulus associated with it, then you are in pain. You can not mistake pain regardless of whether it is "real" or not. What makes pain so special? It hurts, it is painful.
Exactly. Ethically important, we don't want to cause suffering. But danger? What special risk does consciousness bring to the table?
If we hook GTP into an infinite loop and stuff resembling intelligence comes out, then we're really going to have to start questioning what consciousness is. It's coming.
There are some ideas, some questions, and some solutions that I would never think of.
Yet the information is there if I so choose to explore.
Descartes stood on the shoulders of Moses.[1]
[1] Exodus 3:14
The law of gravity is nonsense. No such law exists. If I think I float, and you think I float, then it happens.
You are possibly mistaking free will with being an agent that has a (possibly deterministic) method of updating priors in response to new (unknown to the agent, and possibly deterministic) input.
I do not believe that I am confusing free will and consciousness. See my comment above. Determinism versus free will is independent of knowledge available. Consider a paralyzed person incapable of any action. That person if the senses are all working still has awareness and context. A statistical engine only appears to. A LLM model is basing all actions on a complex matrices of thresholds. It is surprising and amazing how well that works. Given stimulus that takes advantage of those minute differences in thresholds, a wrong response will be returned. Human are not fooled in this way. Minute differences are typically missed or even skipped. Human beings can be fooled by optical illusions and by contradicting context (a statement like pick the "red" circle written in green ink and the person mistakenly picks the "green" circle. LLM models do not make these types of mistakes.
> What I am trying to say is, it doesn't matter if the subject is actually conscious. All that matter is we (I), feel/think that it is conscious, or deserving of care and respect.
My point was that it does matter if the subject is actually conscious. Human beings are easily fooled and it does matter if people mistakenly think it is conscious when it is not.
We would use similar evidence to try and determine if an AI is truly conscious or not.
But even without any serious efforts or success in emulating life with these systems, having intelligent agents that think multiple orders of magnitude faster than humans is very problematic, even if they are controlled by humans. At some point either you prohibit manufacturing hardware beyond a certain performance level, or you just have to hope that agents you are allied with do what's best for you, because there will be no possible chance of keeping up with what is going on.
Try watching a Youtube video at 1/4 speed. Within a few years, the agents will be processing 100 times faster than humans. That is not 4 times slower, it is 100 times slower. Essentially the humans (or their text streams etc.) will be so slow as to appear to be frozen.