Computers exist inside the universe so the universe must be able to compute things. Likewise you can look for certain hallmarks of information manipulation that mean you are computing something.
Usually philosophers talking about these things either haven't read or are just discovering complexity theory.
I've done some more work - apparently nonperiodic tiling is uncomputable. This is used to make toilet roll more compressed (that's why the sheets don't line up) and that must count as a manifestation of the uncomputable in our physical universe. I think this is a reasonable proof that the physical universe is uncomputable.
all of these discussions eventually reveal themselves to be special framing of the old Parmenides question about determinism, whether we live in a block universe where choice and change are illusions, and thought and being are the same. I am increasingly convinced he is right, and that arguments such as the OP (and Searle-ism generally) present end up refuting not the existence of artificial intelligence, but intelligence itself. "Artificial" smuggles in naturalistic fallacy and privileges the dualism hypothesis.
This appears to be a series of arguments from incredulity.
In particular, it is equally incredible that intelligent life should evolve from a single-cell organism. But we have that as a counter-argument.
It is entirely reasonable to suspect that none of the current approaches will yield success, but claiming that no machine intelligences can possibly arise is... incredible.
The main claim being made is that “since AI is a logic system, and living humans are complex systems, AI cannot replicate human intelligence”.
That claim rests on some unfounded, and implicit, assumptions. In particular, the author assumes that neural networks are not complex systems (and as an even deeper, implicit assumption, that no complex neural network could ever exist).
Agreed. I am also extremely skeptical of AI, but while the paper does a good job at highlighting the problems with AI, the eventual conclusion is not at all well-supported.
There's an hidden assumption that complex systems cannot be modeled mathematically at all, but while that can be true right now, there is no fundamental reason why satisfactory models can't be produced at all.
There is a question as to are the systems of mathematics that human cognition can conceive adequate to represent the processes and mechanisms of human cognition or equivalent systems. Basically - can we write 'ourselves' down and if we can, can we read what we have written.
We don’t know how they work exactly, but we do know the mathematics that create them.
The question is if the systems that generate complex intelligence are too much for humans to create, not just the phenomenon that emerge from those systems.
I see no reason to suspect there is any upper limit, in principle, to the complexity or sophistication of things humans can create. The reason is that we are tool users. We create tools, including mathematical and information processing tools that magnify our abilities. These then enable us to create even more powerful tools, which magnify our abilities even more. Rinse, repeat. There doesn't seem to be any obvious reason to me that this has any theoretical limit.
Practical limits maybe, but if dumb evolution can crack the AGI problem clearly it's not beyond practical physical limits.
The paper is full-on nonsense. I’m surprised someone wasted their time writing it and you probably shouldn’t waste your time reading it.
In the part I read it claims we can’t develop AI because we can’t accurately model full reality. There’s no argument about what the connection there is, it’s just stated.
Kind of obviously, if we assume engaging with reality is necessary to develop intelligence, an artificial intelligence could do so in a similar way we non-artificial ones do, right?
I agree that it is nonsense. To save people the click, here, for example, is how the paper argues that a software system couldn't gain intelligence by simulating an evolutionary process:
"But we neither know how to engineer the drive that is built into all animate complex systems, nor do we know how to mimic evolutionary pressure, which we do not understand and cannot model (outside highly artificial conditions such as a Petri dish). In fact, if we already knew how to emulate evolution, we would in any case not need to do this in order to create intelligent life, because the complexity level of intelligent life is lower than that of evolution."
They wrote another paper on this topic, the summary of which is that an AI capable of human level conversations is impossible because:
"This is (1) because there are no traditional explicitly designed mathematical models that could be used as a starting point for creating such programs; and (2) because even the sorts of automated models generated by using machine learning, which have been used successfully in areas such as machine translation, cannot be extended to cope with human dialogue. If this is so, then we can conclude that a Turing machine also cannot possess AGI, because it fails to fulfil a necessary condition thereof."
In other words it can't ever be done because we haven't done it yet. QED. How stuff like this gets to come out of U Buffalo is beyond me. At first I suspected it might have come out of a religious think tank, but no.
What is the mathematical model used for designing linux or chromium or any kind of hyper-complex software?
None, there is no need for a specific mathematical models in general and it's very cringe to claim otherwise.
Sure mathematical models can help in the specific and in the general but there is no set of open problems in mathematics that lead to an impossibility of making AGI.
Sure p!= np but it would be ridiculous to think that brains can bypass algorithmic complexity.
The real problem is that we haven't found the winning lottery ticket software and this software spoiler alert is not at all only made of neural networks
By definition the computable functions are those that can be computed by a Turing machine or equivalent apparatus, so talking about "computation beyond Turing machines" is incoherent.
However, uncomputable functions not only exist, but are the overwhelming majority of functions. This suggests that at least some aspects of reality might only be accurately described by uncomputable functions. How could uncomputable functions exist in an entirely computable reality?
You need to start with the assumption that an infinite set exists (Axiom of Infinity) for uncomputable functions (and other weird mathematical things).
You are asking a question that does not have an answer, because "exist" is not well-defined. But if you want to know whether an arbitrary JavaScript in your browser will stop or not, well you can not know that for all scripts people can throw at you. In this sense uncomputable functions do exist.
BTW - The field trying to deal with defining what exist means is called ontology:
The JavaScript argument doesn't really get at the point I was trying to make (poorly, probably). The parent was arguing that since almost all functions are uncomputable, "some aspects of reality might only be accurately described by uncomputable functions."
This seems unlikely to me. Functions are a mathematical abstraction. Almost all of those functions are uncomputable, but what does the universe care? In what manner could the universe be organized in such a way as to depend on an uncomputable function?
Perhaps a better question for my above post would have been:
Can we imagine any empirical experiment to distinguish between a universe that depends on uncomputable functions and one that does not?
Edit: And if not, in what sense can the universe actually depend on an uncomputable function if we can't distinguish?
Universe depending on all real numbers would be uncomputable, because most real numbers are uncomputable.
This follows from the fact that set of all Turing machines is countable (you can write an algorithm that lists all Turing machines), but set of real numbers is uncountable. So, by Cantor's diagonal argument you can prove that there are real numbers that can not be computed.
My guess is that universe can not depend on real numbers (i.e. real numbers are not real) because real numbers introduce infinity related breakdowns in physical models. For example, if there exist a physical state can encode arbitrary real number then that would allow to store infinite amount of information. But it is not possible to prove that something does not exist, see arguments related to Russell's teapot.
I have similar feelings though I would allow all computable numbers, which would still exclude almost all the reals (while including all rationals and important real numbers like e and pi).
Turing designed his machines to capture "computation" as it is intuitionally understood: as something a mathematician, equipped with an unlimited supply of scratch paper and pencils, could do. (The other equivalent apparatuses are provably equivalent, but it is harder to argue that they capture the correct intuition.)
If you can show something a mathematician can do to compute a function, that cannot be emulated by a Turing machine, then you have demonstrated that Turing's definition does not capture the intuition and we get to start over with the theory of computation. So far, no one has been able to do that.
The existence of uncomputable functions is not itself a problem. It only becomes so if you can show that something computes them.
Not if the definition of computing and uncomputable are from two different models of computation. For example, there are functions uncomputable by finite state machine, but that can be computed by a Turing machine.
One can imagine that someone will find a model of computation that allows to compute more functions than a Turing machine. It's extremely unlikely, but nobody has proven that it is not possible. It may be that our universe is one such model.
On the contrary, uncomputable functions are uncomputable relative to a given model of computation. Many functions cannot be computed by finite state machines but can be by stack machines, and many that cannot be computed by FSMs or stack machines can be computed by Turing machines. (The interesting fact here is that Turing machines are equivalent to a large number of other models of computation, and no "fair" model of computation is known which is more powerful than Turing machines or those other equivalent models. ("Fair" means that the model of computation does not do something that a mathematician with pencil and paper could not also do---"unfair" models do exist but they do things like assuming you can do an unbounded amount of work in a finite number of steps.))
The fact that the halting problem (or the nonperiodic tiling problem mentioned by someone else) is undecidable by Turing machines (and all of the other equivalent models of computation) provides an interesting philosophical tri-lemma:
1. There is a "fair" model of computation that is stronger than TMs. No such are known, and the evidence against them is that a bunch of other models turn out to be equivalent than TMs. There's a considerable amount of mathematical fame if you can find one, though.
2. TMs, etc., are the most powerful "fair" model of computation and there is no physically realizable "unfair" model[1,2]. This has some interesting implications for the "superintelligent AI" stuff. This is also the general consensus as far as I know.
3. Something out there is capable of performing "unfair" computations. (The Oracle of Delphi? Something woo-woo?) I think there are information theoretic/thermodynamic reasons for discarding this option, but what do I know?
[1] Note that a finite state machine is actually the most powerful known physically realizable form of computation. Your laptop is actually a FSM---it has a finite amount of memory and if you converted all of the mass in the universe into RAM it would still only have a finite amount of memory. All of the stronger models are mathematical abstractions.
There are computation models that are not obviously similar to computers; using DNA segments in a broth to solve instances of the Traveling Salesman problem, or soap bubbles (https://www.americanscientist.org/article/the-soap-film-an-a...). These are also equivalent to FSMs---there is only a finite amount of DNA or soap involved.
[2] The fact that functions exist (like the Halting Problem) that cannot be computed by TMs is not in itself terribly important beyond the implications of that tri-lemma above. Nor does the fact that essentially all functions are uncomputable by TMs (proof sketch: continuous numbers are transcendental with probability 1.0). I mean, the halting problem is a thing, so what? It only becomes important if something in the universe actually computes one of those functions that cannot be computed by TMs---that would rule out #2 and leave everyone scrambling for #1 or #3.
> Note that a finite state machine is actually the most powerful known physically realizable form of computation. Your laptop is actually a FSM
I’m under the impression that our computing machines are LBAs, not FSMs. Linear bounded automata are practically as powerful as Turing machines. Sipser gives a nice introductory treatment of the subject.
> It only becomes important if something in the universe actually computes one of those functions that cannot be computed by TMs
Or the universe isn’t a computer and doesn’t compute anything. In that case computation is just a human concept that reduces to moving pebbles (calculi) between various jars.
I'm a little uncomfortable
with LBAs there because, while there is a limit to the storage based on the input size, there is no limit on the input size. It's a fine distinction, but that's the fun of theory.
I assume you know more about this than me, so can you help me understand this or direct me somewhere? The universe is constantly taking its current state (or infinity of relativistic frames and/or multiverses) and outputting its next state/states. Is that not definitionally a specialized computer? Are my assumptions wrong?
Suppose we have some intelligence that cannot be simulated in a Turing machine because it invokes an uncomputable function. That intelligence as it exists in reality is computing something that cannot be computed by a Turing machine. It's not impossible that that can exist (quantum computers may someday demonstrate that, for example), but it would be a huge discovery that would shake the foundations of information theory, especially if it was happening at neuron-scale.
I believe whether there is a deterministic system underlying quantum behavior is still an open question. My suspicion is that nothing can produce true randomness, in whioh case Turing machines are incomplete only for nonexistent computations. If I'm wrong, Turing machines still feel the effects of true randomness at a greater degree than the brain (compare the size of an axon with silicon), though my assumption that everything in the universe can be modeled in a computer falls through.
I think any argument against the possibility of developing AGI is going to have it. The argument has to be either:
1) there’s something non-material about human intelligence (basically, there’s a soul), or
2) something about the processes that created a completely material human intelligence is impossible in principle to reproduce, either implicitly or explicitly.
(1) has the obvious religious angle, but (2) tends to be what’s trotted out when (1) is too overly religious.
With (2), the usual supporting reason is that the conditions are too complex. The problem is that the fundamental rules are just those of physics, which are “simple”. And we have to remember that the initial conditions of the universe were also “simple” and not intelligently set up in a way that could be predicted to create intelligence. It was just a bunch of initially formless matter evolving over time.
By closing the door on even implicit use of physics (which created our own intelligence), which we don’t know enough to rule out completely, there’s the feeling that there’s some kind of magic dust that has to be part of the process or the initial conditions. That would disagree with our current understanding of the laws of physics and early development of the universe.
Ultimately, the real motivation is the desire to maintain the feeling that humans are somehow “special” in the universe.
Is it though? A creature living in a 4 dimensional world might argue that an AI that was brought up in a 3d world would never truly grasp 4 dimensions. This could be true. So why wouldn't it hold for other aspects where the model is incomplete?
It's not obvious at all. The following isn't a proof and if I had one I'd be publishing elsewhere. However, I believe it suffices to show the lack of obviousness.
All of our models of reality are restricted to computable functions. However, we know that uncomputable functions not only exist, but that nearly all functions are in fact uncomputable. Therefore, it's well within the realm of plausibility that the actual behavior of the universe is governed by uncomputable functions, and we are forever stuck modeling those behaviors with computable approximations.
To claim that reality is entirely computable, one has to show how uncomputable functions can exist therein. I wouldn't call this proof, but it strongly suggests to me that the behavior of the system we call reality is uncomputable, and that subsystems thereof also may not be. If what we call human intelligence is one of those uncomputable subsystems, then it's true that computational AI will never achieve it. Nonetheless, we've gotten pretty far with computable approximations, so machine "intelligence" that's close enough for practical purposes doesn't strike me as impossible even if we inhabit an uncomputable reality.
why not make AI on the same platform as the human brain? what is so exceptional about it, and even if it is an exceptional material, why not just use it?
I don’t see why we have to compute all of reality to create an AI anyway. I’m intelligent, to a point, and I’m pretty sure I don’t compute all reality. My neurological processes simply model enough of reality for me to function more or less effectively, that’s all.
The article works from the absurd premise that an AI would have to perfectly model its environment, but no living creatures do this. It also decides that we can’t create general AI because we don’t know how to do it. Therefore it’s impossible. Seriously, it’s right there in the conclusion.
One thing I think is interesting about the paper is how complex systems (like the brain) change over _time_ rather than at fixed points, and if those changes are computable in any meaningful way.
Like if we created an AI 100 years ago, could the AI 100 years later learn how to use an ipad or understand what twitter is or what a meme is?
What if brain changes from cultural (environmental) change are both complex and thus creating mathematical models that would change the intelligence of the AI in the way the brain changes is impossible. Like physical changes in the circuits in the brain that are so distributed, interconnected and complicated and subtle yet very specifically "tailored" to the complex system so as to make them virtually impossible to abstract or model in any way, and thus changes the "mathematical model" of the brain that is sort of "virtualized" at a fixed point in time.
Edit:
Well to put it a little more explicitly:
What if the real reason brains are intelligent is not because of the brain alone, but also because of the underlying physical systems like molecules, maybe even going all the way down to quantum mechanics and that those lower levels cause changes over time that fundamentally alters the function of the brain but still has the evolutionary potential of the lower level physical stuff.
If you have 2 levels:
1) the brain
2) the underlying physical stuff below neurons
1 is a virtualized fixed point in time that we can model and 2 is part of a complex system that alters 1 but importantly in a way that cannot be computed without simulating that stuff at the lower level. I feel like this is sort of implied in the article because either intelligence can be abstracted completely accurately or there will (as the paper claims) always be lower level physical changes that alters the intelligence in a way that cannot be computed at the brain/intelligence level. I don't know if this is true though tbh
^ This is really the point. Human intelligence is based on limited and filtered input, rough analog approximations in processing, and incomplete and interpolated mental representations and internal simulations, and yet nobody seriously denies that we possess some degree of intelligence. I am skeptical that current methods will get us to AGI, but the idea that machines must achieve some level of computational perfection far above and beyond humans is not reasonable.
"Therefore, it's well within the realm of plausibility that the actual behavior of the universe is governed by uncomputable functions..."
It's also well within the realm of plausibility that the behavior of the universe is governed by invisible, intangible unicorns. :-)
If you can provide an example of something in the universe actually computing an theoretically uncomputable function, then there is a gigantic problem somewhere and everyone is going to have to do some re-thinking. If.
By definition nothing will ever compute an uncomputable function. You’re only stating a trivial truth that by itself fails to refute the conjecture that we live in a reality that has behaviors that cannot be accurately described by computable functions and that we are forever stuck with merely approximate computable models.
In fact, so far we’re not able to completely computationally predict any behavior of reality. Even the marvelous theory of quantum electrodynamics is only shown to be accurate to, last I knew, about a dozen places.
Due to an unfortunately widespread misunderstanding of the Church-Turing thesis, far too many otherwise intelligent persons with some CS knowledge are completely blinkered to the possibility that the universe could have behaviors that are real, but not describable with computable functions beyond approximately. The practicing laboratory scientists I’ve spoken with don’t generally share that defect since they’re used to everything being approximate.
I think you have a valid point here, but it does not do anything to save this paper.
The paper claims to have refuted the possibility of what Searle calls Strong AI, but speculating that the universe has behaviors not describable with computable functions does not justify that conclusion. As Simonh pointed out above [1], Strong AI does not imply or depend on everything about the universe being computable, and even if it did, mere speculation that some unknown thing about the universe might not be computable would not justify inferring that therefore Strong AI must be impossible.
This paper is combination of an unargued-for tacit premise with a rather blatant case of burden shifting.
Whether certain behaviors in the universe are uncomputable or not is irrelevant. If we classify "humans" as intelligent and assume no spooky immaterial aspects of consciousness, you already have an existence proof that intelligence is computable.
That doesn't follow from the given premises. I assume the hidden leg of your rhetorical enthymeme is that the universe is a Turing machine. The burden of proof for such a claim, if you want to introduce it as an axiom, is on you. And no, the Church-Turing thesis does not prove that claim.
As a postulate, I'm happy to accept it for the sake of argument. And if we assume the universe is a Turing machine then sure the existence of human intelligence proves that a Turing Machine can simulate human intelligence and therefore it's computable.
Unfortunately, as far as I can tell nothing further follows. Perhaps it's correct, but it's rather dull too.
> I assume the hidden leg of your rhetorical enthymeme is that the universe is a Turing machine
No, I am not making that claim. Not sure how you inferred that.
... by any chance, are you the author of the paper linked in this post? I am asking because I wonder if you are a philosopher by training, rather than being an actual AI researcher. If you are, I wonder if you'd heard of the essay "Newton's Flaming Laser Sword" (https://philosophynow.org/issues/46/Newtons_Flaming_Laser_Sw...)
I’m not. I’m just an interested layman who likes to read and think. And if I’m really fortunate, to think clearly and distinctly.
Evidently then whatever claim it was you were making was not clear to me. I’d be obliged if you made it so. In case it’s not obvious I supposed you implied the universe is a TM on account of that was the first way it occurred to me to make sense of your claim.
Incidentally, that Newton’s laser sword monograph was interesting but philosophically rather immature. I think the author would greatly benefit from reading some pragmaticist philosophy, in particular that of C.S. Peirce. Who, by the way, was a mathematician and logician in the Frege tier. Frankly though I do share the author’s disdain for the more navel-gazing pursuits that occupy many contemporary philosophy departments.
> Therefore, it's well within the realm of plausibility that the actual behavior of the universe is governed by uncomputable functions, and we are forever stuck modeling those behaviors with computable approximations.
Even if the set of uncomputable functions outnumbers the set of of computable functions, I still don't see how your conclusion follows. The rules that govern a coherent universe are not randomly sampled from the set of all functions.
You’re making the contradictory claims that human intelligence is incomputable and AI is limited something computable (all by a certain definition of computable).
You need to at least try to propose something that explains the premise and the contradiction.
Does human intelligence arise out of processes within the human brain? If not, then how else? If yes, then why are those processes somehow out of reach of human science to investigate and manipulate?
How can intentional human actions be limited to a certain definition of computability while human intelligence is not?
The most obvious counter-argument is that the amount of things we can do with AI keeps expanding. People were incredulous that computer chess programs could beat humans in the 1980s. Now they can beat us at basically any board game including Go, do image classification, and we have some early prototypes of self-driving cars.
AI hasn't mastered common-sense reasoning yet. That's likely going to come last, but the amount of things AI can understand is set to only expand IMO.
Well I am not defending the paper thesis but no it's time to realize that we are in a new AI winter where progress has stopped.
Sure we can make accuracy progress on tasks that were underesearched before, moreover we do make extremely slow (and with increasingly diminishing returns) accuracy gains on core tasks.
But the diminishing returns are diminishing fast to the point that progress in terms of applications has stopped for core AI tasks such as NLU.
However there is still some hope as the vast majority of papers bring an innovation but almost never attempt to merge/synergize with other papers innovations.
If human resources where allocated at merging the top 10 papers on a given task, I'm sure it would lead to a major accuracy improvement.
One thing I’ve long noticed is that “common sense” is analogous to a stopped clock, in that it’s its only correct when it happens to also be a different form of reasoning such as deductive, inductive, or abductive reasoning. Things called “common sense” but which are not also a different kind of knowledge are mere cultural shibboleths, and vary from wrong (fan death) to opinion (Shakespeare is good).
The traditional examples of common sense knowledge given when introducing the topic of A.I. are sufficiently imprecise to only be true given further common sense interpretation. For example: “things fall when you let go of them” unless they’re buoyant, or they fly, or they’re already on the ground, or you were in free-fall when you let go — these exceptions won’t really surprise anyone, and yet it’s both more compact and more accurate to say Σf=ma, f_g=G(m_1)(m_2)/r^2 etc.
I think you may be confusing automated processing with communicable abstracted insight.
If this isn't obvious consider the difference between producing an AI that can play chess, producing an AI that learns to play chess, and producing a research program that produces an AI that can play chess and summarises all the resulting developments and insights.
It doesn't seem to be that far off from other (usually unconvincing) philosophical arguments that ultimately boil down to axioms or definitions.
For example, John Searle (of the infamous Chinese Room "argument"/paradox) posits that a simulation of a mind - however realistic and convincing it may be - is not the same as an actual mind any more than virtual reality is the same as actual reality. For example, you could have a chatbot that passes the Turing test but inside it is just smoke and mirrors (e.g. ML models.)
Which is to say if we could somehow replace Searle with a robot whose appearance and behavior were indistinguishable from that of the actual Searle then it would only confirm what we already know.
(Though it is fun to imagine the actual Searle secretly watching his robotic replacement in anger as it does things that he would never do but which are still completely convincing to his duped students, while faculty colleagues take a liking to robo-Searle in a way that they never did to the original.)
"In recent years, a number of prominent computer scientists, along with academics in fields such as philosophy and physics, have lent credence to the notion that machines may one day become as large as humans. Many have further argued that machines could even come to exceed human size by a significant margin. However, there are at least seven distinct arguments that preclude this outcome. We show that it is not only implausible that machines will ever exceed human size, but in fact impossible."
"Numerous entries in the Godzilla film series feature machines so large that they can crush portions of the Tokyo skyline with a single step (Honda, 1975)"
> But we neither know how to engineer the drive that is built into all animate complex
systems, nor do we know how to mimic evolutionary pressure, which we do not under-
stand and cannot model (outside highly artificial conditions such as a Petri dish). In
fact, if we already knew how to emulate evolution, we would in any case not need to
do this in order to create intelligent life, because the complexity level of intelligent life
is lower than that of evolution. This means that emulating intelligence would be much
easier than emulating evolution en bloc.
Chalmers is, therefore, wrong. We cannot engineer the conditions for a spontaneous
evolution of intelligence.
this is the thing i've always sort of loved about philosophy. they just kinda make shit up, provide their own definitions that are rooted in a bamboozling by use of flowery language, and then once they've stated all their definitions with their conclusions baked in, they hop, skip and jump down the path which now obviously leads to the conclusion they started with.
it's kind of like a form of mathematics where they define their own first principles in each argument with the express purpose of trying to build the most beautiful path to their conclusions. it really is a beautiful form of art, like architecture for ideas.
Funny, that's the exact reason I hate philosophy. And I say this as someone with a BA in it.
I thought of it initially as a useful way to model the abstract, the hypothetical, and the integrity our own ideas and perceptions.
But so many philosophers tried to use their arguments to prove things about the world. Like a less powerful form of economics, which itself is based on the "if we model X this way, Y" mindset.
I like your conceptualization of philosophy as art. I'll probably to refer to it that way hereon.
I think of it a bit like science vs scientific thinking (that isn't constrained to actual science and logic) that one encounters on the internet. The problem is not with philosophy, it is with humans.
The excerpt you quoted is perfectly meaningful. They are saying an evolutionary process that produces intelligent life is more complicated ("has more moving parts") than the intelligent life forms thus produced. How could this not be true? An intelligent life form may have hundreds of billions of neurons and trillions of cells, but the evolution that produced said life form involved untold zillions of complex life forms over billions of years. There are over 10-to-the-power-of-30 microbes on planet earth right now, evolving in ways currently beyond the understanding of any computational biologist.
Although computer scientists can use genetic algorithms inspired by evolution to "breed" better backgammon algorithms, this is quite a few orders of magnitude simpler than emulating a true evolution of intelligent biological life.
The point is intelligent biological life forms are less complicated than the "factory" that produced them.
> They are saying an evolutionary process that produces intelligent life is more complicated ("has more moving parts") than the intelligent life forms thus produced. How could this not be true?
because we don't know enough about evolutionary processes nor intelligent life to make statements like that, and "more complicated" is completely ill-defined.
how do we know that evolution isn't simply a few basic rules, a lot of randomness and a lot of time?
Yup, you nailed it. We have a lot of examples of complexity arising from a very simple set of initial conditions and rules. Why should evolution be any different?
i suppose, giving the authors the benefit of the doubt, one could make a statement like:
if one were to parameterize an entire line of evolution over time, and one were to parameterize a single intelligent being over time, then it is likely that the number of bits required to describe that evolutionary line (and the space of all evolutionary lines) is greater than the number of bits required to completely describe a single intelligent life form over time.
this still tells us nothing about the rules behind evolution, how an intelligence actually works, how evolution actually works and what would be necessary to manifest an intelligence.
> They are saying an evolutionary process that produces intelligent life is more complicated ("has more moving parts") than the intelligent life forms thus produced. How could this not be true?
The whole field of emergent complexity exists to answer questions about this. Questions which only exist because there are many situations where "this" is evidently not true.
You are right that the quoted passage is not without meaning; what is missing here (and by "here", I mean the whole paper) is any remotely good argument from the relatively trivial factual claim in this passage, to the conclusion that true artificial intelligence is impossible.
The factual claim is about the history of evolution: if we take that history to include everything produced during that history, then it is trivially true that the whole is greater than any subset of the things it produced - but so what? It is true for the creation of a microprocessor as well. There is no argument here that rules out the creation of artificial intelligence that does not also apply to the creation of microprocessors.
It seems problematic to disentangle the complexity of an entity from the complexity of the process which produced it. If we define complexity as the Kolmogorov complexity, the two are equivalent (https://en.wikipedia.org/wiki/Kolmogorov_complexity?wprov=sf...)
Rather, I interpret Wolfram's idea as:
Surprisingly complex patterns can be produced by simple/concise rules.
In my interpretation, the ultimate example of this would be the unfolding of everything that has ever happened as the consequence of the laws of physics, and some initial condition of the universe.
This was the thesis of Thales of Miletus approximately 2600 years ago, but for the sake of this discussion let's credit Wolfram with the idea since he seems to love credit. ;-)
The human experience of computer programming shows a problem with Wolfram's thought. Wolframism in a nutshell says,
1. The complicated processes of the universe reduce
to simple rules (a la Conway's Life).
2. Therefore, simple rules *produce* the more complicated processes.
(Or the complicated *emerges* from the simple.)
Consider now that the complexity of computer software "reduces" to simple rules for manipulating 0's and 1's executed on processors, rules like
- MOVE the sequence of 32 bits from memory address A1 to register R1
- ADD the sequence of 32 bits in register R1 to register R2 ...
However, these simple rules don't make complicated software---otherwise, many of us here on HN would have to find other work.
Rather, complicated humans make complicated software using these simple rules. (Digression: It's of course too hard to make modern software using these simple rules, so we use complicated toolchains instead to deal with the simple rules.)
Even with Conway's Game of Life, it is often overlooked that in order to actually run a game simulation, you either need to build a very complicated electronic computer to host the "simple" simulation, or you need a complex intelligent life form with a pencil and a great deal of both paper and patience to carry out the rules step-by-step.
Moreover, the simple rules for Conway's Game of Life were created (or discovered?) in the complicated brain of mathematician John Conway. It seems only the most intelligent and complex minds come up with these "simple" mathematical rules like Einstein's "E=mc^2" or Newton's "f=ma".
Also, I know from experience that randomly seeding Conway's Life almost never produces interesting results, such as long lived stable colonies with "gliders" frolicking to and fro. An intelligent human needs to fine tune the initial conditions for the game to get interesting universes that don't quickly become lifeless or stagnant. Interesting universes are sufficiently rare that, when found, are worthy of publishing and nerding out with your friends over.
So Wolframism in its supreme form says the Universe began with simple rules and these simple rules created stars and planets, humans and their brains. But simple rules need a machine to run the rules, a machine more complicated than the rules themselves. And even given a machine to run the rules, most rule sets and most initial configurations produce dead universes.
I appreciate your response for making me think more deeply about the idea and seeing the merits in the point of the person I replied to.
The task of understanding the entire evolutionary process step by step that led to intelligent life is indeed more complex than any single thing that arose from it, as a necessity. One can't know every stage by which the brain developed and the purpose of that without also being able to understand the method by which it accomplishes those tasks.
As an AI engineer though, the main question is not if I can understand what I created and how I created it, but rather can I engineer an environment that itself will produce the desired outcome. This is often the experience of modern AI engineers, try to find any who could explain exactly how any large model like GPT-3 works, or how it even came to work as it trained. It would be impossible to sufficiently describe and grasp the complexity of the entire process in a human lifetime with a human brain. Yet, one can with not too much trouble understand how to create an environment that will give rise to the same model again.
Humans seem to be able to do many things, like run businesses or govern nations, without truly comprehending exactly all the moving pieces that accomplished it. I suppose this could be thought of as finding "leverage", where certain, more easily understood and controlled processes, can allow one to accomplish much harder and more complex things.
While the points about the enormous complexity of evolution and intelligence are taken and agreed with, I'm not convinced that they mean we couldn't recreate them by leveraging simple processes like genetics and natural selection to recreate them artificially.
It begs the question though. The argument is: Assume you can't make artificial intelligence without the stimuli of the real world's processes. Since a single human is less complicated than the real world, therefore you can't make artificial intelligence.
I think the devil is in one important detail, namely - how do we define complexity?
Let's define the problem as Evolution(inputs) = Intelligence. The claim is that complexity(function) + complexity(inputs) > complexity(outputs). Now to show that the parent claim is not necessarily true (which is not the same as proving that it is false), we just need to show that there exists a combination of complexity function and a system that does not satisfy the above constraints.
1. Let's examine information compressibility as a complexity function. There are a few examples of where a simple set of rules and inputs could produce basically an infinite stream of incompressible information. Examples include Conway game of life, double pendulum, fractals, all irrational numbers, etc...
2. Now to tie that back to Evolution, the authors avoid defining evolution or its inputs, which means they could be quite simple yet produce mind boggling complexity. Therefore the argument that the evolution must be more complex than intelligent life is backwards (if you buy my complexity definition anyhow :P).
3. Of course this kind breaks down if we discretize evolution because at that point all of the existing life is an input into the evolution. So complexity(evolution) + complexity(life[t] + environment[t]) > complexity(life[t+1]) is obviously true for some t and t + 1. For example if t is right before a mass extinction event and t+1 is right after.
This is somewhat unrelated, but I am quite partial of theory that life in general and intelligence in particular is driven by entropy. Or maybe less confusingly (because who the hell knows what entropy is) is driven by macro tendency of everything towards lowest energy states. Life in this case is smart matter that bridges activation energy gap to extract available energy gradients as fast as possible. Here is the concept explained by people who put a lot more thought into it: https://www.quantamagazine.org/a-new-thermodynamics-theory-o...
> They are saying an evolutionary process that produces intelligent life is more complicated ("has more moving parts") than the intelligent life forms thus produced. How could this not be true?
Assuming the universe itself is described by something like the standard model, or perhaps something even conceptually simpler like vibrating modes of ether, string theory, etc, then clearly conceptual complexity does indeed arise from simplicity. In fact entropy suggests that is the only direction possible.
You can’t pick your favourite bad argument and ridicule an entire field. You are incidentally using ideas from several different old, influential philosophies to even formulate Luther comment
Philosophy includes questions like “how do we decide whether something is true or trustworthy,” or “what constitutes a good or a bad way to make a case for something.” If you’re going to throw philosophy out, you can’t question anything any more
Philosophy may refer to the specific branch in academia and its current practice, as opposed to any philosophical inquiry. Every field already pursues their own philosophical inquiry, and yet philosophers and mathematicians are in separate departments. Such is the current practice and organization of academics.
If we were to consider mathematics and computer science as part of philosophy, then we might say that as a mode of inquiry, philosophy has had great success in achieving multidisciplinary consensus and international impact. But if we were to consider philosophy as a specific branch of academic organization, then we might be disappointed at the fruits emerging from that field.
to bring it full circle and tear apart the original argument above, one could argue that the relatively simple laws of logic from philosophy give rise to all of digital computing (like... what is directly expressed in digital logic design). yet the emergent complexity of all computerdom is far beyond the complexity of the basic rules of logic coming from philosophy.
more to the point here, computer science and mathematics are very similar to philosophy in that authors invent a set of abstractions and then construct rules for how they interact in a self-consistent manner.
Relating computer science and mathematics to philosophy is already a fair and understandable argument. Yet, as separate divisions of professional academic labor, why do the fruits they bear look so different in terms of their ability to generate frameworks for multi-disciplinary consensus and international impact?
computer science, mathematics, science and many other technical fields borrow logic and formalized reasoning from philosophy and subsets of those fields have direct applications in the system of production, therefore i think that's why they're elevated in terms of perceived impact.
i've always sort of seen pure philosophy as an art for exercising rhetorical and general reasoning skills, which then seem to have the most natural applications in fields like law and politics.
i think where it can go sideways is cases where philosophers will attempt to apply reasoning to premises or fields that they don't fully understand.
that said, as we march into the post truth era, perhaps philosophy will have another moment to shine as a framework for training reasoning skills in an uncertain world.
fair, it's the first step towards trying to construct formalism around reasoning... but then it often jumps off into the abstract based on synthetic premises... that's when it makes the leap to art.
and who said anything about ridicule? art is important! perceptive exercises and exploration of ideas strengthen our skills for reasoning.
Philosophy has never even been accidentally right about anything regarding artificial intelligence, which is impressive since it seems like every philosopher and his dog felt the need to opine on AI since the inception of the idea.
Should we ever attain hardware, software and understanding of the human brain good enough to emulate a human brain, we have done it.
There is absolutely no reason why this shouldn‘t be possible. Actually, we could already do it if we understood the brain enough and could model it good enough, even if the emulation might not be real time.
"Though the infinitesimal definition of utility in (1) and the penalisation of complexity
in the definition of Υ provide a statistically robust measure of the kind of surrogate
intelligence those working in the general artificial intelligence (AGI) field have decided to focus on, the definition is too weak to describe or specify the behaviour even of an
arthropod. This is not only obvious from the issues already mentioned above, but also
from the fact that algorithms which realise the reward-schemes proposed in (1) and (2)
(for example, neural networks optimised with reinforcement learning) fail to display the
type of generalisable adaptive behaviour to natural environments that arthropods are
capable of, for example when ants or termites colonise a house."
Ok, I don't like the mathematical definitions of intelligence either (although I might be convincable and they do have some advantages over other definitions I've seen), but this refutation seems to be a prime example of proof-by-assertion.
"Brooks defines an AI agent, again, as an artefact that is able ‘to move around in
dynamic environments, sensing the surroundings to a degree sufficient to achieve the
necessary maintenance of life and reproduction'."
And this definition implies many things we know to be intelligent (i.e. people) are not. So there's that.
"There are three additional properties of logic systems of importance for our argument
here:
1. Their phase space is fixed.
2. Their behaviour is ergodic with regard to their main functional properties.
3. Their behavior is to a large extent context-independent."
Aaaaand here we go...
"As we learn from we standard mathematical theory of complex systems [23], all such
systems, including the systems of complex systems resulting from their interaction,
1. have a variable phase space,
2. are non-ergodic, and
3. are context-dependent."
Ok, to the extent that the first statement is true about "logic systems", it is also true about any physically realizable, material system. On the other hand, the "complex system", to that same extent, is not physically realizable. (Consider "a variable phase space means that the variables which define the elements of
a complex system can change over time" or "a non-ergodic system produces erratic distributions of its elements. No matter
how long the system is observed, no laws can be deduced from observing its elements." and question how much information is required for this in the authors' sense.)
And there we have the intrusion of the immortal soul into the argument that artificial intelligence is impossible.
It's not likely to be a popular opinion with technologists as AI's potential has lit the technopopular imagination, however this question has bothered me for a long time. I think strong emergent AI suffers philosophical problem that won't go away, and to the extent that the conversation revolves around evolution and consciousness rather than logic and intelligence, then we are having the right conversation.
I'll put my argument out there and let the flames come as they will.
Strong AI is about as likely to emerge from our current state of the art AI machinery as it is to emerge suddenly out of moon rocks. That's to say the fear of machines becoming self-conscious and posing an existential threat to us, especially replacing us in the evolutionarily sense, is completely unfounded.
This isn't to say that building machines capable of doing exactly that isn't possible - we and all living things are proof that it's possible - it's to say that achieving this level of engineering is on par with intergalactic mass transit or Dyson spheres - way out of our league for the foreseeable. And, even if we had the technology, it would be so entirely foolish to undertake that no sentient species would do it.
That said, there's a substantial argument to make that we will augment ourselves with our own machinery so throughoughly that we will become unrecognizable and in effect, accomplish the same task through merging with the machine. This is likely, but not at all to be like the experience of the singularity in that all of humanity is suddenly arrested and deposed by autonomous AI.
An interesting scenario in this vein is if a few powerful individuals can wield autonomous systems, modify themselves and simply wipe out all the competition, then in effect the rest of us wouldn't know the difference. This outcome is actually I think on the more likely side, albeit a good ways away in the future.
Less likely but still totally legitimate as a concern is the idea that AI could be very easily weaponized. This is a real problem and is I think behind the more substantive warnings by good thinkers on the topic. Like bioweapons, we might be wiped out by an machine that's been intentionally programmed and mechanically empowered to cause real harm. This kind of danger could also be emergent, in that a machine might be capable of deciding that it ought to take certain actions as well as have the capacity to take them, and then, voila, mass murder.
However it seems unlikely that such a mistake would be made, or that a bad actor would be capable to commit such an intentional crime. I think this is on par with nuclear MAD: even total madmen dictators hit the pause on the push-the-button instinct. And an AI MAD or similar would surely take as much resource to produce as a nuke arsenal. In other words, the resources required to build such a machinery are on the order of a nation-state, and perhaps more complicated to achieve than a nuclear arsenal, so probably more likely to be stopped or fail in-process rather than succeed.
So there are dangers from AI but I would say they are lesser than the accumulated danger of industrial society rendering they planet uninhabitable, which should of course occupy our primary concern these days.
The idea that the biological evolutionary 'machine' whose motive for existence is accumulated over billions of years of entropic adaptation can be out engineered, or accidently replicated by modern computational AI is silly - the two aren't in the same league and it's hubris to suppose otherwise. There's more intelligence in the toe of a lady bug than in an the computing power ever made.
In sum the danger from emergent AI is overstated, however the concern is most welcome to the extent that it informs wisdom and care in consideration for our techno-industrial impact on the biosphere.
Their premises about logical systems are wrong so their conclusion is not valid. In short, of course, there are logical systems with potentially infinite state space. For example, a Turing machine. A digital circuit is no different. Turing completeness is abundant, it is everywhere.
The Turing machine was designed by imagining a human-operator. Our Mind has also only a finite state, and no matter if quantum effects are involved, the information in it is always finite, describable in a finite state. Thus, all turing machines are capable do do exactly what we do with information. This argument is incredible.
This is Russell's teapot. You can not prove that infinite states do not exist. In fact, common models of physics assume possibility of infinite number of states by depending on formalism axiomatically assuming infinite sets (e.g. axiom of infinity in Zermelo–Fraenkel set theory).
I take for granted, the world exists, therefore it is.
You may call it Borrible's first tautology.
Or perhaps bias.
Yes, Borrible' bias sounds clever. At least to me.
And that is what counts, doesn't it?
I don't really know what that fucking world really is, but nonetheless it exists.
With temporarily stable local dynamics, some parts of the world began to copy themselves.
Albeit with errors and quirks.
The recurring processes of the surrounding builded the mold for the debris that collects in the swirls.
Some of those copies developed representations of their surroundings.
First in form of simple notes sticking on themselves, being themselves.
Which was an advantage, when they bumped into another.
They could navigate that thing I called world.
Which made their copy process stable.
With a lot of time and try and error, some parts of those parts of parts of that thing I called world even developed some really fancy little dollhouse worlds in this part of the world that will later call itself the brain.
And the most advanced ham actors in that dollhouse put more tiny little dolls in that house, the most precious one, ego.
It represented that part of the world that started the whole shebang, the body.
And it equipped that tiny little dollhouse with a lot of woundrous and a lot of silly things, some animated , some not.
And it took great delight in it, it even fancied itself a god and pushed the tiny little ego around doing his biddings.
But for the most part it just tried to please itself and learn about the world and itself, based on all that input it got somehow from the world.
And the drama that ham actor and his Muppet Friends acted.
Exactly like all those good little boys and girls do on their playgrounds since time immemorable.
When I was young, something happend.
My dolls started to become 'Little Computer People'.
And people my generation and that before developed fancy models about this Matrioshka Doll World, about Worlds in Worlds in World in Worlds.
Infinite regress, sometimes recursive, sometimes not.
A calaidoscopic mirror, sometimes dark, sometimes shiny.
Simulacron 1, 2, 3 and so on until there is no energetic process in that thing I call world, that can be harvested.
And every time those models became more complex, they gave more agency to that part of the world that is now mumbling about building a new Ghost in the Machine.
Apart from the possibly insurmountable practical problems, I see no reason in principle why it should become more complex in the form of artificial intelligence.
As an aside, it's great to be that part of the world, but beware.
It may all end the moment that ham actor in that dollhouse cuts the strings to that world he is living of.
A risk deeply embedded in this structure.
Of an agent acting in a model of the world.
The agent is subject to the risk of his striving to make himself independent of the world.
119 comments
[ 3.4 ms ] story [ 180 ms ] threadSpeaking specifically of neural networks as they exist now the answer is no because there is no obvious way to learn.
Usually philosophers talking about these things either haven't read or are just discovering complexity theory.
Yes - but there may be other mechanisms of translating information into outcomes that are possible in the universe as well. I don't know any though.
In particular, it is equally incredible that intelligent life should evolve from a single-cell organism. But we have that as a counter-argument.
It is entirely reasonable to suspect that none of the current approaches will yield success, but claiming that no machine intelligences can possibly arise is... incredible.
The main claim being made is that “since AI is a logic system, and living humans are complex systems, AI cannot replicate human intelligence”.
That claim rests on some unfounded, and implicit, assumptions. In particular, the author assumes that neural networks are not complex systems (and as an even deeper, implicit assumption, that no complex neural network could ever exist).
[0] https://en.wikipedia.org/wiki/2-satisfiability#Algorithms
[1] https://en.wikipedia.org/wiki/Boolean_satisfiability_problem...
There's an hidden assumption that complex systems cannot be modeled mathematically at all, but while that can be true right now, there is no fundamental reason why satisfactory models can't be produced at all.
The question is if the systems that generate complex intelligence are too much for humans to create, not just the phenomenon that emerge from those systems.
Practical limits maybe, but if dumb evolution can crack the AGI problem clearly it's not beyond practical physical limits.
In the part I read it claims we can’t develop AI because we can’t accurately model full reality. There’s no argument about what the connection there is, it’s just stated.
Kind of obviously, if we assume engaging with reality is necessary to develop intelligence, an artificial intelligence could do so in a similar way we non-artificial ones do, right?
"But we neither know how to engineer the drive that is built into all animate complex systems, nor do we know how to mimic evolutionary pressure, which we do not understand and cannot model (outside highly artificial conditions such as a Petri dish). In fact, if we already knew how to emulate evolution, we would in any case not need to do this in order to create intelligent life, because the complexity level of intelligent life is lower than that of evolution."
"This is (1) because there are no traditional explicitly designed mathematical models that could be used as a starting point for creating such programs; and (2) because even the sorts of automated models generated by using machine learning, which have been used successfully in areas such as machine translation, cannot be extended to cope with human dialogue. If this is so, then we can conclude that a Turing machine also cannot possess AGI, because it fails to fulfil a necessary condition thereof."
https://arxiv.org/abs/1906.05833
In other words it can't ever be done because we haven't done it yet. QED. How stuff like this gets to come out of U Buffalo is beyond me. At first I suspected it might have come out of a religious think tank, but no.
If your conclusion implies the existence of computation beyond Turing machines, you should probably find an example or check your assumptions.
However, uncomputable functions not only exist, but are the overwhelming majority of functions. This suggests that at least some aspects of reality might only be accurately described by uncomputable functions. How could uncomputable functions exist in an entirely computable reality?
Such concepts cannot appear in a finite universe.
BTW - The field trying to deal with defining what exist means is called ontology:
https://en.wikipedia.org/wiki/Ontology
This seems unlikely to me. Functions are a mathematical abstraction. Almost all of those functions are uncomputable, but what does the universe care? In what manner could the universe be organized in such a way as to depend on an uncomputable function?
Perhaps a better question for my above post would have been:
Can we imagine any empirical experiment to distinguish between a universe that depends on uncomputable functions and one that does not?
Edit: And if not, in what sense can the universe actually depend on an uncomputable function if we can't distinguish?
This follows from the fact that set of all Turing machines is countable (you can write an algorithm that lists all Turing machines), but set of real numbers is uncountable. So, by Cantor's diagonal argument you can prove that there are real numbers that can not be computed.
My guess is that universe can not depend on real numbers (i.e. real numbers are not real) because real numbers introduce infinity related breakdowns in physical models. For example, if there exist a physical state can encode arbitrary real number then that would allow to store infinite amount of information. But it is not possible to prove that something does not exist, see arguments related to Russell's teapot.
https://en.wikipedia.org/wiki/Computable_number
https://en.wikipedia.org/wiki/Cantor%27s_diagonal_argument
https://en.wikipedia.org/wiki/Russell%27s_teapot
Thanks for the interesting discussion!
If you can show something a mathematician can do to compute a function, that cannot be emulated by a Turing machine, then you have demonstrated that Turing's definition does not capture the intuition and we get to start over with the theory of computation. So far, no one has been able to do that.
The existence of uncomputable functions is not itself a problem. It only becomes so if you can show that something computes them.
By definition nothing will ever compute an uncomputable function. That’s completely irrelevant to what I wrote.
One can imagine that someone will find a model of computation that allows to compute more functions than a Turing machine. It's extremely unlikely, but nobody has proven that it is not possible. It may be that our universe is one such model.
The fact that the halting problem (or the nonperiodic tiling problem mentioned by someone else) is undecidable by Turing machines (and all of the other equivalent models of computation) provides an interesting philosophical tri-lemma:
1. There is a "fair" model of computation that is stronger than TMs. No such are known, and the evidence against them is that a bunch of other models turn out to be equivalent than TMs. There's a considerable amount of mathematical fame if you can find one, though.
2. TMs, etc., are the most powerful "fair" model of computation and there is no physically realizable "unfair" model[1,2]. This has some interesting implications for the "superintelligent AI" stuff. This is also the general consensus as far as I know.
3. Something out there is capable of performing "unfair" computations. (The Oracle of Delphi? Something woo-woo?) I think there are information theoretic/thermodynamic reasons for discarding this option, but what do I know?
[1] Note that a finite state machine is actually the most powerful known physically realizable form of computation. Your laptop is actually a FSM---it has a finite amount of memory and if you converted all of the mass in the universe into RAM it would still only have a finite amount of memory. All of the stronger models are mathematical abstractions.
There are computation models that are not obviously similar to computers; using DNA segments in a broth to solve instances of the Traveling Salesman problem, or soap bubbles (https://www.americanscientist.org/article/the-soap-film-an-a...). These are also equivalent to FSMs---there is only a finite amount of DNA or soap involved.
[2] The fact that functions exist (like the Halting Problem) that cannot be computed by TMs is not in itself terribly important beyond the implications of that tri-lemma above. Nor does the fact that essentially all functions are uncomputable by TMs (proof sketch: continuous numbers are transcendental with probability 1.0). I mean, the halting problem is a thing, so what? It only becomes important if something in the universe actually computes one of those functions that cannot be computed by TMs---that would rule out #2 and leave everyone scrambling for #1 or #3.
I’m under the impression that our computing machines are LBAs, not FSMs. Linear bounded automata are practically as powerful as Turing machines. Sipser gives a nice introductory treatment of the subject.
> It only becomes important if something in the universe actually computes one of those functions that cannot be computed by TMs
Or the universe isn’t a computer and doesn’t compute anything. In that case computation is just a human concept that reduces to moving pebbles (calculi) between various jars.
I'm a little uncomfortable with LBAs there because, while there is a limit to the storage based on the input size, there is no limit on the input size. It's a fine distinction, but that's the fun of theory.
1) there’s something non-material about human intelligence (basically, there’s a soul), or 2) something about the processes that created a completely material human intelligence is impossible in principle to reproduce, either implicitly or explicitly.
(1) has the obvious religious angle, but (2) tends to be what’s trotted out when (1) is too overly religious.
With (2), the usual supporting reason is that the conditions are too complex. The problem is that the fundamental rules are just those of physics, which are “simple”. And we have to remember that the initial conditions of the universe were also “simple” and not intelligently set up in a way that could be predicted to create intelligence. It was just a bunch of initially formless matter evolving over time.
By closing the door on even implicit use of physics (which created our own intelligence), which we don’t know enough to rule out completely, there’s the feeling that there’s some kind of magic dust that has to be part of the process or the initial conditions. That would disagree with our current understanding of the laws of physics and early development of the universe.
Ultimately, the real motivation is the desire to maintain the feeling that humans are somehow “special” in the universe.
There are two flimsy arguments for machine intelligence from Hunter and Brooks. The paper is poking holes in that.
E.g., they address Brook’s definition of basic arthropod intelligence, and are positive about it.
> We then summarise the highly useful definition of basic (arthropod) intelligence proposed by Rodney Brooks
All of our models of reality are restricted to computable functions. However, we know that uncomputable functions not only exist, but that nearly all functions are in fact uncomputable. Therefore, it's well within the realm of plausibility that the actual behavior of the universe is governed by uncomputable functions, and we are forever stuck modeling those behaviors with computable approximations.
To claim that reality is entirely computable, one has to show how uncomputable functions can exist therein. I wouldn't call this proof, but it strongly suggests to me that the behavior of the system we call reality is uncomputable, and that subsystems thereof also may not be. If what we call human intelligence is one of those uncomputable subsystems, then it's true that computational AI will never achieve it. Nonetheless, we've gotten pretty far with computable approximations, so machine "intelligence" that's close enough for practical purposes doesn't strike me as impossible even if we inhabit an uncomputable reality.
The article works from the absurd premise that an AI would have to perfectly model its environment, but no living creatures do this. It also decides that we can’t create general AI because we don’t know how to do it. Therefore it’s impossible. Seriously, it’s right there in the conclusion.
Like if we created an AI 100 years ago, could the AI 100 years later learn how to use an ipad or understand what twitter is or what a meme is? What if brain changes from cultural (environmental) change are both complex and thus creating mathematical models that would change the intelligence of the AI in the way the brain changes is impossible. Like physical changes in the circuits in the brain that are so distributed, interconnected and complicated and subtle yet very specifically "tailored" to the complex system so as to make them virtually impossible to abstract or model in any way, and thus changes the "mathematical model" of the brain that is sort of "virtualized" at a fixed point in time.
Edit: Well to put it a little more explicitly: What if the real reason brains are intelligent is not because of the brain alone, but also because of the underlying physical systems like molecules, maybe even going all the way down to quantum mechanics and that those lower levels cause changes over time that fundamentally alters the function of the brain but still has the evolutionary potential of the lower level physical stuff.
If you have 2 levels: 1) the brain 2) the underlying physical stuff below neurons
1 is a virtualized fixed point in time that we can model and 2 is part of a complex system that alters 1 but importantly in a way that cannot be computed without simulating that stuff at the lower level. I feel like this is sort of implied in the article because either intelligence can be abstracted completely accurately or there will (as the paper claims) always be lower level physical changes that alters the intelligence in a way that cannot be computed at the brain/intelligence level. I don't know if this is true though tbh
It's also well within the realm of plausibility that the behavior of the universe is governed by invisible, intangible unicorns. :-)
If you can provide an example of something in the universe actually computing an theoretically uncomputable function, then there is a gigantic problem somewhere and everyone is going to have to do some re-thinking. If.
In fact, so far we’re not able to completely computationally predict any behavior of reality. Even the marvelous theory of quantum electrodynamics is only shown to be accurate to, last I knew, about a dozen places.
Due to an unfortunately widespread misunderstanding of the Church-Turing thesis, far too many otherwise intelligent persons with some CS knowledge are completely blinkered to the possibility that the universe could have behaviors that are real, but not describable with computable functions beyond approximately. The practicing laboratory scientists I’ve spoken with don’t generally share that defect since they’re used to everything being approximate.
The paper claims to have refuted the possibility of what Searle calls Strong AI, but speculating that the universe has behaviors not describable with computable functions does not justify that conclusion. As Simonh pointed out above [1], Strong AI does not imply or depend on everything about the universe being computable, and even if it did, mere speculation that some unknown thing about the universe might not be computable would not justify inferring that therefore Strong AI must be impossible.
This paper is combination of an unargued-for tacit premise with a rather blatant case of burden shifting.
[1] https://news.ycombinator.com/item?id=29291286
As a postulate, I'm happy to accept it for the sake of argument. And if we assume the universe is a Turing machine then sure the existence of human intelligence proves that a Turing Machine can simulate human intelligence and therefore it's computable.
Unfortunately, as far as I can tell nothing further follows. Perhaps it's correct, but it's rather dull too.
No, I am not making that claim. Not sure how you inferred that.
... by any chance, are you the author of the paper linked in this post? I am asking because I wonder if you are a philosopher by training, rather than being an actual AI researcher. If you are, I wonder if you'd heard of the essay "Newton's Flaming Laser Sword" (https://philosophynow.org/issues/46/Newtons_Flaming_Laser_Sw...)
Evidently then whatever claim it was you were making was not clear to me. I’d be obliged if you made it so. In case it’s not obvious I supposed you implied the universe is a TM on account of that was the first way it occurred to me to make sense of your claim.
Incidentally, that Newton’s laser sword monograph was interesting but philosophically rather immature. I think the author would greatly benefit from reading some pragmaticist philosophy, in particular that of C.S. Peirce. Who, by the way, was a mathematician and logician in the Frege tier. Frankly though I do share the author’s disdain for the more navel-gazing pursuits that occupy many contemporary philosophy departments.
https://news.ycombinator.com/item?id=29293687
Even if the set of uncomputable functions outnumbers the set of of computable functions, I still don't see how your conclusion follows. The rules that govern a coherent universe are not randomly sampled from the set of all functions.
You need to at least try to propose something that explains the premise and the contradiction.
Does human intelligence arise out of processes within the human brain? If not, then how else? If yes, then why are those processes somehow out of reach of human science to investigate and manipulate?
How can intentional human actions be limited to a certain definition of computability while human intelligence is not?
AI hasn't mastered common-sense reasoning yet. That's likely going to come last, but the amount of things AI can understand is set to only expand IMO.
However there is still some hope as the vast majority of papers bring an innovation but almost never attempt to merge/synergize with other papers innovations. If human resources where allocated at merging the top 10 papers on a given task, I'm sure it would lead to a major accuracy improvement.
The traditional examples of common sense knowledge given when introducing the topic of A.I. are sufficiently imprecise to only be true given further common sense interpretation. For example: “things fall when you let go of them” unless they’re buoyant, or they fly, or they’re already on the ground, or you were in free-fall when you let go — these exceptions won’t really surprise anyone, and yet it’s both more compact and more accurate to say Σf=ma, f_g=G(m_1)(m_2)/r^2 etc.
I think you may be confusing automated processing with communicable abstracted insight.
If this isn't obvious consider the difference between producing an AI that can play chess, producing an AI that learns to play chess, and producing a research program that produces an AI that can play chess and summarises all the resulting developments and insights.
For example, John Searle (of the infamous Chinese Room "argument"/paradox) posits that a simulation of a mind - however realistic and convincing it may be - is not the same as an actual mind any more than virtual reality is the same as actual reality. For example, you could have a chatbot that passes the Turing test but inside it is just smoke and mirrors (e.g. ML models.)
Which is to say if we could somehow replace Searle with a robot whose appearance and behavior were indistinguishable from that of the actual Searle then it would only confirm what we already know.
(Though it is fun to imagine the actual Searle secretly watching his robotic replacement in anger as it does things that he would never do but which are still completely convincing to his duped students, while faculty colleagues take a liking to robo-Searle in a way that they never did to the original.)
[1] https://arxiv.org/abs/1703.10987
this is the thing i've always sort of loved about philosophy. they just kinda make shit up, provide their own definitions that are rooted in a bamboozling by use of flowery language, and then once they've stated all their definitions with their conclusions baked in, they hop, skip and jump down the path which now obviously leads to the conclusion they started with.
it's kind of like a form of mathematics where they define their own first principles in each argument with the express purpose of trying to build the most beautiful path to their conclusions. it really is a beautiful form of art, like architecture for ideas.
I thought of it initially as a useful way to model the abstract, the hypothetical, and the integrity our own ideas and perceptions.
But so many philosophers tried to use their arguments to prove things about the world. Like a less powerful form of economics, which itself is based on the "if we model X this way, Y" mindset.
I like your conceptualization of philosophy as art. I'll probably to refer to it that way hereon.
Although computer scientists can use genetic algorithms inspired by evolution to "breed" better backgammon algorithms, this is quite a few orders of magnitude simpler than emulating a true evolution of intelligent biological life.
The point is intelligent biological life forms are less complicated than the "factory" that produced them.
because we don't know enough about evolutionary processes nor intelligent life to make statements like that, and "more complicated" is completely ill-defined.
how do we know that evolution isn't simply a few basic rules, a lot of randomness and a lot of time?
if one were to parameterize an entire line of evolution over time, and one were to parameterize a single intelligent being over time, then it is likely that the number of bits required to describe that evolutionary line (and the space of all evolutionary lines) is greater than the number of bits required to completely describe a single intelligent life form over time.
this still tells us nothing about the rules behind evolution, how an intelligence actually works, how evolution actually works and what would be necessary to manifest an intelligence.
The whole field of emergent complexity exists to answer questions about this. Questions which only exist because there are many situations where "this" is evidently not true.
The factual claim is about the history of evolution: if we take that history to include everything produced during that history, then it is trivially true that the whole is greater than any subset of the things it produced - but so what? It is true for the creation of a microprocessor as well. There is no argument here that rules out the creation of artificial intelligence that does not also apply to the creation of microprocessors.
https://www.wolframalpha.com/examples/science-and-technology...
It maybe looks as an advertisement for Mathematica.
Rather, I interpret Wolfram's idea as:
Surprisingly complex patterns can be produced by simple/concise rules.
In my interpretation, the ultimate example of this would be the unfolding of everything that has ever happened as the consequence of the laws of physics, and some initial condition of the universe.
The human experience of computer programming shows a problem with Wolfram's thought. Wolframism in a nutshell says,
Consider now that the complexity of computer software "reduces" to simple rules for manipulating 0's and 1's executed on processors, rules like However, these simple rules don't make complicated software---otherwise, many of us here on HN would have to find other work.Rather, complicated humans make complicated software using these simple rules. (Digression: It's of course too hard to make modern software using these simple rules, so we use complicated toolchains instead to deal with the simple rules.)
Even with Conway's Game of Life, it is often overlooked that in order to actually run a game simulation, you either need to build a very complicated electronic computer to host the "simple" simulation, or you need a complex intelligent life form with a pencil and a great deal of both paper and patience to carry out the rules step-by-step.
Moreover, the simple rules for Conway's Game of Life were created (or discovered?) in the complicated brain of mathematician John Conway. It seems only the most intelligent and complex minds come up with these "simple" mathematical rules like Einstein's "E=mc^2" or Newton's "f=ma".
Also, I know from experience that randomly seeding Conway's Life almost never produces interesting results, such as long lived stable colonies with "gliders" frolicking to and fro. An intelligent human needs to fine tune the initial conditions for the game to get interesting universes that don't quickly become lifeless or stagnant. Interesting universes are sufficiently rare that, when found, are worthy of publishing and nerding out with your friends over.
So Wolframism in its supreme form says the Universe began with simple rules and these simple rules created stars and planets, humans and their brains. But simple rules need a machine to run the rules, a machine more complicated than the rules themselves. And even given a machine to run the rules, most rule sets and most initial configurations produce dead universes.
The task of understanding the entire evolutionary process step by step that led to intelligent life is indeed more complex than any single thing that arose from it, as a necessity. One can't know every stage by which the brain developed and the purpose of that without also being able to understand the method by which it accomplishes those tasks.
As an AI engineer though, the main question is not if I can understand what I created and how I created it, but rather can I engineer an environment that itself will produce the desired outcome. This is often the experience of modern AI engineers, try to find any who could explain exactly how any large model like GPT-3 works, or how it even came to work as it trained. It would be impossible to sufficiently describe and grasp the complexity of the entire process in a human lifetime with a human brain. Yet, one can with not too much trouble understand how to create an environment that will give rise to the same model again.
Humans seem to be able to do many things, like run businesses or govern nations, without truly comprehending exactly all the moving pieces that accomplished it. I suppose this could be thought of as finding "leverage", where certain, more easily understood and controlled processes, can allow one to accomplish much harder and more complex things.
While the points about the enormous complexity of evolution and intelligence are taken and agreed with, I'm not convinced that they mean we couldn't recreate them by leveraging simple processes like genetics and natural selection to recreate them artificially.
Let's define the problem as Evolution(inputs) = Intelligence. The claim is that complexity(function) + complexity(inputs) > complexity(outputs). Now to show that the parent claim is not necessarily true (which is not the same as proving that it is false), we just need to show that there exists a combination of complexity function and a system that does not satisfy the above constraints.
1. Let's examine information compressibility as a complexity function. There are a few examples of where a simple set of rules and inputs could produce basically an infinite stream of incompressible information. Examples include Conway game of life, double pendulum, fractals, all irrational numbers, etc...
2. Now to tie that back to Evolution, the authors avoid defining evolution or its inputs, which means they could be quite simple yet produce mind boggling complexity. Therefore the argument that the evolution must be more complex than intelligent life is backwards (if you buy my complexity definition anyhow :P).
3. Of course this kind breaks down if we discretize evolution because at that point all of the existing life is an input into the evolution. So complexity(evolution) + complexity(life[t] + environment[t]) > complexity(life[t+1]) is obviously true for some t and t + 1. For example if t is right before a mass extinction event and t+1 is right after.
This is somewhat unrelated, but I am quite partial of theory that life in general and intelligence in particular is driven by entropy. Or maybe less confusingly (because who the hell knows what entropy is) is driven by macro tendency of everything towards lowest energy states. Life in this case is smart matter that bridges activation energy gap to extract available energy gradients as fast as possible. Here is the concept explained by people who put a lot more thought into it: https://www.quantamagazine.org/a-new-thermodynamics-theory-o...
Assuming the universe itself is described by something like the standard model, or perhaps something even conceptually simpler like vibrating modes of ether, string theory, etc, then clearly conceptual complexity does indeed arise from simplicity. In fact entropy suggests that is the only direction possible.
Philosophy includes questions like “how do we decide whether something is true or trustworthy,” or “what constitutes a good or a bad way to make a case for something.” If you’re going to throw philosophy out, you can’t question anything any more
If we were to consider mathematics and computer science as part of philosophy, then we might say that as a mode of inquiry, philosophy has had great success in achieving multidisciplinary consensus and international impact. But if we were to consider philosophy as a specific branch of academic organization, then we might be disappointed at the fruits emerging from that field.
more to the point here, computer science and mathematics are very similar to philosophy in that authors invent a set of abstractions and then construct rules for how they interact in a self-consistent manner.
i've always sort of seen pure philosophy as an art for exercising rhetorical and general reasoning skills, which then seem to have the most natural applications in fields like law and politics.
i think where it can go sideways is cases where philosophers will attempt to apply reasoning to premises or fields that they don't fully understand.
that said, as we march into the post truth era, perhaps philosophy will have another moment to shine as a framework for training reasoning skills in an uncertain world.
and who said anything about ridicule? art is important! perceptive exercises and exploration of ideas strengthen our skills for reasoning.
There is absolutely no reason why this shouldn‘t be possible. Actually, we could already do it if we understood the brain enough and could model it good enough, even if the emulation might not be real time.
Ok, I don't like the mathematical definitions of intelligence either (although I might be convincable and they do have some advantages over other definitions I've seen), but this refutation seems to be a prime example of proof-by-assertion.
"Brooks defines an AI agent, again, as an artefact that is able ‘to move around in dynamic environments, sensing the surroundings to a degree sufficient to achieve the necessary maintenance of life and reproduction'."
And this definition implies many things we know to be intelligent (i.e. people) are not. So there's that.
"There are three additional properties of logic systems of importance for our argument here: 1. Their phase space is fixed. 2. Their behaviour is ergodic with regard to their main functional properties. 3. Their behavior is to a large extent context-independent."
Aaaaand here we go...
"As we learn from we standard mathematical theory of complex systems [23], all such systems, including the systems of complex systems resulting from their interaction, 1. have a variable phase space, 2. are non-ergodic, and 3. are context-dependent."
Ok, to the extent that the first statement is true about "logic systems", it is also true about any physically realizable, material system. On the other hand, the "complex system", to that same extent, is not physically realizable. (Consider "a variable phase space means that the variables which define the elements of a complex system can change over time" or "a non-ergodic system produces erratic distributions of its elements. No matter how long the system is observed, no laws can be deduced from observing its elements." and question how much information is required for this in the authors' sense.)
And there we have the intrusion of the immortal soul into the argument that artificial intelligence is impossible.
I'll put my argument out there and let the flames come as they will.
Strong AI is about as likely to emerge from our current state of the art AI machinery as it is to emerge suddenly out of moon rocks. That's to say the fear of machines becoming self-conscious and posing an existential threat to us, especially replacing us in the evolutionarily sense, is completely unfounded.
This isn't to say that building machines capable of doing exactly that isn't possible - we and all living things are proof that it's possible - it's to say that achieving this level of engineering is on par with intergalactic mass transit or Dyson spheres - way out of our league for the foreseeable. And, even if we had the technology, it would be so entirely foolish to undertake that no sentient species would do it.
That said, there's a substantial argument to make that we will augment ourselves with our own machinery so throughoughly that we will become unrecognizable and in effect, accomplish the same task through merging with the machine. This is likely, but not at all to be like the experience of the singularity in that all of humanity is suddenly arrested and deposed by autonomous AI.
An interesting scenario in this vein is if a few powerful individuals can wield autonomous systems, modify themselves and simply wipe out all the competition, then in effect the rest of us wouldn't know the difference. This outcome is actually I think on the more likely side, albeit a good ways away in the future.
Less likely but still totally legitimate as a concern is the idea that AI could be very easily weaponized. This is a real problem and is I think behind the more substantive warnings by good thinkers on the topic. Like bioweapons, we might be wiped out by an machine that's been intentionally programmed and mechanically empowered to cause real harm. This kind of danger could also be emergent, in that a machine might be capable of deciding that it ought to take certain actions as well as have the capacity to take them, and then, voila, mass murder.
However it seems unlikely that such a mistake would be made, or that a bad actor would be capable to commit such an intentional crime. I think this is on par with nuclear MAD: even total madmen dictators hit the pause on the push-the-button instinct. And an AI MAD or similar would surely take as much resource to produce as a nuke arsenal. In other words, the resources required to build such a machinery are on the order of a nation-state, and perhaps more complicated to achieve than a nuclear arsenal, so probably more likely to be stopped or fail in-process rather than succeed.
So there are dangers from AI but I would say they are lesser than the accumulated danger of industrial society rendering they planet uninhabitable, which should of course occupy our primary concern these days.
The idea that the biological evolutionary 'machine' whose motive for existence is accumulated over billions of years of entropic adaptation can be out engineered, or accidently replicated by modern computational AI is silly - the two aren't in the same league and it's hubris to suppose otherwise. There's more intelligence in the toe of a lady bug than in an the computing power ever made.
In sum the danger from emergent AI is overstated, however the concern is most welcome to the extent that it informs wisdom and care in consideration for our techno-industrial impact on the biosphere.
“Department of Philosophy”
hmm
You may call it Borrible's first tautology. Or perhaps bias. Yes, Borrible' bias sounds clever. At least to me. And that is what counts, doesn't it?
I don't really know what that fucking world really is, but nonetheless it exists.
With temporarily stable local dynamics, some parts of the world began to copy themselves. Albeit with errors and quirks. The recurring processes of the surrounding builded the mold for the debris that collects in the swirls.
Some of those copies developed representations of their surroundings. First in form of simple notes sticking on themselves, being themselves. Which was an advantage, when they bumped into another. They could navigate that thing I called world. Which made their copy process stable.
With a lot of time and try and error, some parts of those parts of parts of that thing I called world even developed some really fancy little dollhouse worlds in this part of the world that will later call itself the brain.
And the most advanced ham actors in that dollhouse put more tiny little dolls in that house, the most precious one, ego. It represented that part of the world that started the whole shebang, the body. And it equipped that tiny little dollhouse with a lot of woundrous and a lot of silly things, some animated , some not. And it took great delight in it, it even fancied itself a god and pushed the tiny little ego around doing his biddings.
But for the most part it just tried to please itself and learn about the world and itself, based on all that input it got somehow from the world. And the drama that ham actor and his Muppet Friends acted.
Exactly like all those good little boys and girls do on their playgrounds since time immemorable.
When I was young, something happend. My dolls started to become 'Little Computer People'.
And people my generation and that before developed fancy models about this Matrioshka Doll World, about Worlds in Worlds in World in Worlds. Infinite regress, sometimes recursive, sometimes not. A calaidoscopic mirror, sometimes dark, sometimes shiny.
Simulacron 1, 2, 3 and so on until there is no energetic process in that thing I call world, that can be harvested.
And every time those models became more complex, they gave more agency to that part of the world that is now mumbling about building a new Ghost in the Machine.
Apart from the possibly insurmountable practical problems, I see no reason in principle why it should become more complex in the form of artificial intelligence.
As an aside, it's great to be that part of the world, but beware. It may all end the moment that ham actor in that dollhouse cuts the strings to that world he is living of.
A risk deeply embedded in this structure. Of an agent acting in a model of the world.
The agent is subject to the risk of his striving to make himself independent of the world.