Part of this discussion illustrates something about philosophy that once struck me as odd. Lucas carefully constructs a somewhat formal argument for his position, that would, if correct, rule out the computational theory of mind. One counterargument takes off from the point that Lucas' argument assumes that any computational model of a mind must be consistent, and depends on a mind being able to establish its own consistency. In response, we are told "Lucas thinks it is unlikely that an inconsistent machine could be an adequate representation of a mind." So now the careful, structured argument is reduced to a personal opinion? (The article includes some more specific variations on the theme by Lucas, but they all seem to me to be appeals to plausibility.)
To me, this seems to leave Lucas with the burden of proof, and until he or someone else can counter this objection with something more than an opinion, the argument is dead in the water. I used to wonder why philosophy would hang an elaborate argument on personal opinions, but I have come to appreciate that whenever you speculate about matters where there is insufficient evidence to settle the issue, it always comes down to opinions, though this is often overlooked in the discussion.
Don't think of it as "a mind", but as "the knowledge held by a mind": one of the defining properties of knowledge is truth.
(That's not intended as an endorsement of Lucas-Penrose. There are deeper flaws in the argument, like the assumption that the truth of the knowledge of the mind implies that the mind KNOWS its own truthfulness.)
You are doing good philosophy here. Though: I'd be careful to move from the fact that Lucas' argument comes down to a personal opinion to the fact that all philosophy comes down to personal opinions.
I assume you already know about the concept of "commitment" in Philosophy, but I'll explain it briefly here in case anyone else comes across this comment.
Sometimes while you can't show that a fact is categorically false (given some available evidence), you can show that people who believe that fact are committed to believing other facts. In this case Lucas' own argument has committed him to the "personal opinion" that an inconsistent machine cannot be an adequate representation of mind. But his belief isn't entirely unfalsifiable. If new evidence makes him give up this belief (e.g. a rise of paraconsistent logics) he'll have to give up his original belief along with it.
It's not about personal opinions. Or rather it is, but not in an obvious way.
Human minds are clearly not consistent. Human minds are a loose collection of cobbled-together heuristics.
Logic is one of those heuristics. It has limits because ultimately "logic" ends up being not a personal opinion, but a personal experience of the truth - or otherwise - of a proposition.
Developed and trained minds have generated a heuristic of independent consensus about truths and experience, and strategies for increasing the correlation between the truth experience and the reality experience.
Less developed untrained minds have a much more naive and subjective experience of "truth."
But both work on the same foundation - a nebulous and indefinable subjective experience that is associated with truth, and is typically mistaken as the definition of it.
It's impossible to "prove" truth, because the experience of deciding whether or not something is true is absolutely subjective.
Any attempt to make an objective definition will always be filtered through that subjectivity and reduced to either having that experience or not having it.
We're back to qualia, as usual. No one knows what "having an experience" really means. Until that question is answered, there are no tools for dealing with the rest.
You can argue "But what about the scientific method, and mathematical proofs?" Same answer - they're practices that generate the truth experience in trained minds. They have a better predictive record and are better at finding consistent patterns than the disjointed heuristics of untrained minds. But they still don't - and can't - prove "truth" in any kind of ultimate definitive guaranteed-to-be-objective way.
They certainly can't guarantee that the truth experience is never biased, contingent, limited, naive, circumstantial, or possibly even just plain wrong.
The implications for AI are obvious. If a machine is better at finding consistent patterns, it will appear smarter than a human. It will still be smarter even if it makes errors that are obvious to a human, because at the same time it won't be making errors that humans do make, and which are similarly "obviously wrong" to the AI.
It's the hit rate that matters, not idealised philosophical perfection.
So take the argument as 'if you agree with me that an inconsistent machine is not a good representation of the mind', then the rest of my argument must follow.
Think of it as analogous to all of the mathematical quasi-proofs that assume certain conjectures are true. They can often be valuable steps to an actual proof, as we saw with Fermat's theorem.
Thanks - that seems to be the idea I was groping for.
In this particular case, though, it seems close to circularity - if the primary purpose of the argument is to refute the computational theory of mind, then ruling out a whole class of machines a priori looks to me like begging a chunk of the question - though I appreciate that it leads to a better-defined question.
Philosophy covers a lot of ground (e.g. the subject matter that is generally called "science" today was once called "natural philosophy" and much of the contemporary science of psychology has its roots in the late 19th and early 20th century American philosophical movement called "pragmatism" (or later "pargmaticism" by CS Pierce in order that the term would be so ugly as to not be misappropriated)). And there are people who argue that psychology is not really a science and usually more because people will argue about anything than out of deep objectivity that is applied to their own position.
Anyway, there is a manifold in philosophic space in which proving things to win arguments matters and a manifold where torturing philosophy with the scientific method matters and a manifold in which there is doubt about the external world and one in which confusion arises from lack of rigor regarding language and one in which the goal is to prove the existence of gods and a manifold of other manifolds.
We all get to take our pick because philosophy is a big tent that is big enough that a philosopher is free to wander among the lion tamers and clowns and acrobats and look at what is interesting. Radical skepticism is a useful intellectual tool, but so are Platonic idealism and deconstruction and Kantian metaphysics. Anyone who thinks they've got it all correct is probably ignoring human fallibility (but maybe not).
> One counterargument takes off from the point that Lucas' argument assumes that any computational model of a mind must be consistent, and depends on a mind being able to establish its own consistency.
Which would obviously be false. There are clearly true theorems we will never be able to prove. Simply because we can prove many theorems that simpler formal systems cannot, entails nothing about our logical consistency or completeness.
There's an argument to be made that these logical constructs you mention might just nkt be meaningful in the real world. Everything has to come down to zeroth order logic, eventually, I guess.
At the risk of having my junior philosopher's secret decoder ring taken away, the true definition of philosophy is an elaborate logical structure built to decorate your previously existing, unreasoned prejudices.
Only someone that knows very little of philosophy would say such a thing. Oh, and I doubt you'd appreciate someone similarly pissing on your profession.
Philosophers are concerned with two points: soundness and validity. A valid argument is one where the conclusion follows from the premises. The way to think about this is 'If I assume the premises are true, is my conclusion likely to be true'. A sound argument is a valid argument with true premises.
Now valid arguments can be interesting in their own right even if their premises aren't true.
Given this, the proper way to read Lucas's opinion is: I am taking it as given that the computational theory of mind requires a consistent system of inference rules + math; if that isn't true, my argument might be unsound but not invalid (and its validity is interesting); but I also think it is true for x,y,z reasons (though these additional reasons do not constitute a proof).
In other words, the Lucas-Penrose argument is interesting from both a truth-of-premises and validity perspective which is far from dead in the water.
Here is the quote from Gödel supposedly supporting this:
"So the following disjunctive conclusion is inevitable: Either mathematics is incompletable in this sense, that its evident axioms can never be comprised in a finite rule, that is to say, the human mind (even within the realm of pure mathematics) infinitely surpasses the powers of any finite machine, or else there exist absolutely unsolvable diophantine problems of the type specified" . . . (Gödel 1995: 310).
Wikipedia describes Godel as a nonreligious theist who believed in the existence of an afterlife. In that context, it is not surprising that he would draw far-reaching conclusions that a human being's behaviour cannot be described by a formal system.
It won't be contentious among people who know much about Gödel. He spent his later years working on things like proofs of the existence of God. Like many geniuses he went far beyond what his profession was comfortable with.
What if the human mind is computational, but isn't one unified system. I think most neurologists today believe the mind is composed of many loosely coupled systems communicating with each other?
Both the Lucas-Penrose argument and the counterarguments seem to assume that consciousness is a phenomenon within the laws of physics which arises from more fundamental laws (e.g. chemistry or quantum mechanics), which is (to me) a completely non-obvious assumption.
For example, I perceive my own thoughts rather than other people's. This implies that my own thoughts would have an additional physical property, that of being perceived by me, which the thoughts of other people lack. However, according to all up-to-date theories of physics and neurology, there is no difference between my own thoughts and other people's that would explain the presence of this additional property.
Aren't your own thoughts similar to a CPU's registers or cache? One's thoughts seem to occur within the context of the brain and the billions of neighboring (to the ones inducing the current "thought") neurons. Its as obvious to me that you can't perceive another's thoughts just as surely as one CPU can't "perceive" the registers of another.
The perception of ones own thoughts seems very physical, as is the equally obvious (to me) explanation that you can't directly perceive the thoughts of another.
For you to perceive your thoughts doesn't require different properties to other people's thoughts. Just different values to common properties.
It's not clear why you would expect this to be inherently different from any other neurological response. For example, if I stub my toe, are you suggesting that science doesn't have an answer to why you don't feel it?
I am suggesting that the fact that I (the perceiving subject) am myself (the biological entity) rather than you (a different biological entity, but ostensibly subject to the same laws) is (a) an objective fact (b) not within the domain of science. I do not expect this fact to map to any observable physical properties.
It's because the thing generating the thoughts (you) is coupled to the thing perceiving the thoughts (you) via pathways that can carry the information. I, however, am not connected to that thing (you) via equivalent pathways.
I am, as luck would have it, connected in a more or less identical fashion to the thing creating my thoughts (me).
> Now suppose that we construct the Gödel sentence for this formal system. Since the Gödel sentence cannot be proven in the system, the machine will be unable to produce this sentence as a truth of arithmetic. However, a human can look and see that the Gödel sentence is true. In other words, there is at least one thing that a human mind can do that no machine can. Therefore, “a machine cannot be a complete and adequate model of the mind” (Lucas 1961: 113). In short, the human mind is not a machine.
That is not a difference between human and machine. It's a rhetorical trick at best, and an unreasonable appeal to mysticism at worst.
A human mind looking at the Gödel sentence while strictly adhering to the same formal structure could also not see that the Gödel sentence is true. Conversely, a machine that is allowed to operate outside of this postulated formalism could perfectly well make an inference that the Gödel sentence is true.
The Lucas-Penrose argument boils down to a questionable attempt at a logic bomb. So by first tying the machine's hands, they argue that the machine's hand's are indeed tied, and therefore humans - who in this example have been given free rein - can do something that you explicitly disallowed machines from doing.
What formalism, if any, is the human mind subject to? Isn't our current conception of a machine that it will bound to some formalism in terms of reasoning? Are we so bound?
Human biology, and by extension human minds, follow a myriad of rules that are a result of biological processes being played out. Likewise "machines" also follow the rules laid out by their physical design. There is no fundamental difference in this regard, there is no new set of information-theoretical rules that comes into play when you switch to a carbon-based machinery.
> What formalism, if any, is the human mind subject to?
In the context of this argument, the mind of the human mathematician would be subject to the rules of the formal system. Since Gödel we know that such a formal system cannot be proven true inside the system itself. However, an entity that views the system from the outside can potentially see it's true. There is no fundamental difference between a human mind and a hypothetical machine mind as far as the capability to come to the same conclusion is concerned. That's why Lucas took special care to specify upfront that the machine mind is prohibited from coming to that conclusion, by confining it to live within the formal system only. Yes, their argument is really that circular.
> Isn't our current conception of a machine that it will bound to some formalism in terms of reasoning? Are we so bound?
We are not, and neither would a generally intelligent machine. The burden of proof is on Lucas/Penrose to show that a generally intelligent machine would still be bound to that formalism, at which point it would cease to be considered generally intelligent. They seem to argue that their postulate is true because the individual components of such a machine are bound by those rules, but then again so are the individual components of our brains.
So your claim is that the Human mind is a physical machine and that computers are also physical machines and thus both are bound to physical laws... and this negates Penrose how?
Do you think Penrose disagrees that the Human mind is bound by physical laws??
I think Penrose is handwaving to try and have it both ways: he's saying the brain obeys physical laws, but that it does not embed a formal computational model. But that's impossible: the second is a logical consequence of the first.
"But that's impossible: the second is a logical consequence of the first."
No, it is not. It is far from proven that universal physical law can everywhere be simulated by a Turing machine. If you had proof of that you'd be up for a Nobel. So publish your paper or perhaps consider that Roger Penrose is not an idiot and you might not know what you are talking about.
It is not proven and possibly it can never be proven. But it can be disproven, and given the success of the Church-Turing thesis, I would say that the burden of proof rest on those that claim that a system more powerful than a Turing machine exist.
I'm not making a claim, I argue based on what we already know. Penrose is the one claiming something here.
> Do you think Penrose disagrees that the Human mind is bound by physical laws??
Yes. Having read his work in the past, I know that he disagrees the human mind is bound by physical laws. This is the whole reason behind the argument they're putting forth to begin with: they postulate that humans are capable of making an inference that could not possibly be made by a machine intelligence.
However, it has never been shown that we are not a machine intelligence.
He argues that quantum magic enables human minds to do things that formal computer systems cannot do. I'm not sure what his position is on computers that incorporate quantum magic though.
I suspect the argument manyoso is setting up here is based on sophistry around the concept of "physical laws". His stance is most likely that quantum woo is part of the physical laws, in the same way a fundamentalist religious person would assert that the content of a holy book is part of the physical laws.
What I - and I hope most people - mean by the phrase is different though, it's the body of actual physical laws that have been shown scientifically.
> I'm not sure what his position is on computers that incorporate quantum magic though.
Interesting thought experiment. I suspect he would be fine with those in theory, however since he already declared this special connection to the supernatural for humans it'd be interesting what computer components he would consider equivalent. But the beauty of his position is that, in the event of a generally intelligent computer emerging, he could always claim that any given nanoscale computing component is the silicon analogue of "quantum microtubules" (which was his woo-related organelle of choice in neurons). It's not like he ever has to prove his claims in order to be taken seriously.
It is not quantum woo and it is not quantum magic. Objective collapse theories are the complete opposite of what you are disparaging. Objective collapse theories are more physical than what we currently have.
Few people have made more contributions in the twentieth century to actual physics than Penrose. He is one of the world's leading experts on general relativity and his objective collapse theory is nothing more that when quantum gravity is understood that gravity will play a role in the collapse of the wavefunction.
To describe Penrose as engaging in 'quantum woo' or 'quantum magic' is grossly inappropriate. It just gives evidence that you don't understand what he is proposing. See Scott Aaronson for what an rebuttal to Penrose looks like not the ham fisted straw man takedowns from the likes of people in this comment section.
If gravity plays a role in wave function collapse that will pose serious questions for the Church-Turing hypothesis. It might very well make it impossible for non-Quantum computers to accurately model quantum gravity.
Penrose does not argue for "something computationally more powerful than objective collapse" rather, he argues that the physical Church-Turing thesis is false due to quantum gravity. Something he is an expert on, btw.
> I suspect the argument manyoso is setting up here is based on sophistry around the concept of "physical laws". His stance is most likely that quantum woo is part of the physical laws, in the same way a fundamentalist religious person would assert that the content of a holy book is part of the physical laws.
That falls far below the standard of discussion we ask users to abide by here. Please make your points directly and without such cheap insinuations.
It's not an incorrect summary of his position. I agree it may not look like a totally objective characterization on my part, but you're saying this in a discussion where I was literally told to "go home", by the same user you're protecting here.
If you think my comments in their entirety detracted from the discussion, I'd like you to tell me. Otherwise I'm asking you to consider this statement in the context of both manyoso's and my comments as a whole.
This is a discussion about quantum quackery, and you're stepping in on behalf of the pro-quackery side that has no problem telling me personally that I "have no idea what" I'm "talking about", and other dismissive discourse.
It is incorrect. Penrose emphatically says the human brain is governed by physical law. To assert otherwise is just factually incorrect. Read his books or read his papers. If you disagree, please provide one quote that shows otherwise.
I am sorry to have said "go home" and will refrain in the future. Still, you have lodged what I consider factually incorrect assertions about Penrose's argument. When asked for evidence you have provided none.
You misunderstand what I'm talking about. What dang criticized me for is a correct summary of your position. Let me elaborate:
> "We proposed [...] that consciousness depends on biologically ‘orchestrated’ coherent quantum processes in collections of microtubules within brain neurons, that these quantum processes correlate with, and regulate, neuronal synaptic and membrane activity [...] This orchestrated OR activity (‘Orch OR’) is taken to result in moments of conscious awareness and/or choice. [...] We conclude that consciousness plays an intrinsic role in the universe."
( Stuart Hameroff, Roger Penrose; Consciousness in the universe; Physics of Life Reviews, Volume 11, Issue 1, March 2014, Pages 39-78 )
and
> "If gravity plays a role in wave function collapse that will pose serious questions for the Church-Turing hypothesis. It might very well make it impossible for non-Quantum computers to accurately model quantum gravity. Penrose does not argue for "something computationally more powerful than objective collapse" rather, he argues that the physical Church-Turing thesis is false due to quantum gravity. Something he is an expert on, btw."
( manyoso, Hacker News, discussion id 14444443 )
None of these statements, while they are using the language of science, bears any discernable relationship to physical reality in a scientific sense. The postulate that any consciousness-related activity of neurons happens in structural elements like microtubules is a collection of unproven hypotheses divorced from both empirical and theoretical experience. The same goes for this preposterous claim about quantum gravity: there is currently no workable theoretical framework for quantum gravity, but this doesn't stop Penrose from confabulating properties onto the term so it can become a new go-to locus for his ideas on quantum consciousness.
An observer will notice a pattern here: mystical truth claims about a hypothesis with no basis in either experimentation nor theory are being "supported" by hiding their supposed mechanisms of action inside yet-unexplored regions of the map. It's a god of the gaps argument.
Now, you know this, as does Penrose. Both of you are trying to preempt this criticism by professing quantum consciousness is a fundamental part of the laws of nature, and hence it should not be categorized as woo. You are essentially rejecting the accusation of mysticism by asserting that magic is not metaphysical. Which was the observation dang chided (and apparently penalized) me for.
For what it's worth: you decisively won the argument, both by popular opinion and the support of the site's admin. Accept a victory when it presents itself.
I chided manyoso elsewhere. The problem with your comment was not that it incorrectly summarized anything, it's that it broke the HN guidelines by calling names ("quackery", "sophistry", "woo", "fundamentalist"). That matters far more to HN than whether Penrose is wrong about Gödel.
Commenting here requires being scrupulously civil, even when others are not. Any lower standard would lead to an unstable system.
In a discussion where one side freely makes stuff up as they go along and attempts to put the burden of proof on everybody else, I stand by these characterizations.
When someone asserts in an authoritative tone that quantum gravity is the key to unlocking consciousness at the heart of the universe, that is utter hogwash and deserves to be called out as such. You may personally not like my pointing this out, but the entire "field" of quantum consciousness is a religious movement that makes unsubstantiated claims about the nature of reality.
It actually is quackery. Redefining the term 'physical laws' to include magic in order to shield an opinion from criticism is actually sophistry. Insisting that spiritual convictions are factual statements about the universe is actual fundamentalism. These aren't bad words wielded to hurt someone, they are supposed to convey meaning.
I never meant to denigrate anyone personally. I strongly feel that opinions must be fair game though when it comes to criticism. In doing so suppose I must come across more harshly than I intended or maybe I'm just exceptionally bad at articulating myself. Whatever my intent, it's clearly not working out all that well. It's probably a good idea for me to take a break from HN for a while.
> I know that he disagrees the human mind is bound by physical laws.
Last I heard, Penrose appeals to quantum woo to claim that the human mind isn't strictly algorithmic. But sources of quantum randomness isn't enough to push you into dualism, quantum shit is part of our physical universe and while there is no proof that the human mind critically exploits quantum phenomena for it's computation, we can certainly build machines that have RNGs exploiting quantum randomness.
> Penrose appeals to quantum woo to claim that the human mind isn't strictly algorithmic.
Penrose doesn't appeal to quantum woo, he has a quite well articulated set of assumptions and arguments , related to loop quantum gravity [1], about how objective collapse of the wave function may occur at bio-physically feasible decoherence times.
Now, whether you think his biophysical theories of consciousness are valid, or even required (whether you buy the microtubules argument/hypothesis), is one thing. But stating that Penrose- arguably the 20th century's foremost mathematical physicist- is a practicioner and/or a spreader of 'quantum woo'-- that belies a level of mis-informantion so catastrophically high that it renders your quoted statement indistinguishable from a pure ad-hominem.
LQG is not believed to allow hypercomputation, i.e. the solving of Turing-uncomputable problems. Penrose's entire argument is based on the idea that the human brain is a hypercomputer, which is why it cannot in principle be simulated by a computer (or any other machine).
It's worth noting that Penrose's field of expertise is General Relativity, not quantum physics, and definitely not Computer Science / philosophy of computation. I see the situation with Penrose as equivalent to Linus Pauling's unfortunate foray into Vitamin C pseudoscience late in his life.
Penrose' claim is much simpler. He thinks that gravity playing a role in wave function collapse will show that non-quantum computers are incapable of accurately modeling quantum gravity. IOW, he thinks gravity will end up showing the physical Church-Turing thesis to be false.
Say what you will about Penrose, but can you seriously deny that he is one of the world's foremost experts on gravity?? You might not like his conclusion, but to say that he is out of his element here is ridiculous. Are you in your element when discussing gravity causing wave function collapse? Who do you think would be more in their element on this subject than Penrose??
Call me back when Penrose convinces a well-regarded quantum physics expert. At that point we can call up Scott Aaronson and ask him his opinion, as Aaronson is an expert on the intersection of quantum physics and computing.
Scott is a friend of mine. I've talked with him face to face about this very subject. He disagrees with Penrose, but his objections are far more nuanced and respectful than anything I have seen here. He would be horrified to see Penrose belittled and his arguments not given a fair reading. Penrose is an intellectual hero to Scott and someone to be admired.
That's good to know. I read his post on his 'debate' with Roger and really enjoyed it. Also of interest may be the discussion between him and Hameroff in the comments section.
Meta-comment:
It a sad sort of situation when intellectual communities, whether they be HN or another, tend to idolize a set of individuals and demonize others.
Thanks @manyosos for your comments in this thread, you really helped elevate the discourse.
@Chronos (can't reply for some reason, so I'm just replying to my own post and tagging you)
Its true that LQG is not known a natural schema to implement hyper-computation. However, since we don't well understand the time-dynamics of twistors ( how these operators may interact non-linearly through time), i don't see any a-priori reason why it is not a possible scheme for super-computation (you could make a good counter- response based on occam's razor, which I'd grant).
Look, lots of thereotical CS folks get disturbed by the idea of hyper-computation/super-turing machines, but in truth Turing machines are a toy model in comparison to true physics; as such, it doesn't take a whole lot more to get something more powerful; Siegelmann and colleagues have shown that real weighted, analog recurrent nets have super-turing abilities [1] [2]. While Aaronson and other raise good questions about physical realizability of such systems, a good thing to keep in mind is that these discussions often take place at computational 'limit' cases, eg solving intractable PSPACE problems, which may not be as relevant to more pedestrian problems solvable by biological systems. Central point: dynamically evolving systems iteratively exploring through (from) in-consistent systems towards more and more consistent ones have many of the same compelling qualities we would call 'super-turing'. Also see the lit on evolving turning machines.
Finally, while I don't agree with the magnitude your Pauling analogy, I certainly agree with you statement that Penrose is out of his depth here. While I do not , presently, buy his argument that quantum effects are necessary to realize consciousness, I remain open to the idea until we know more about BOTH physics and computation.
Also, humans often believe things that aren't provable in any formal system S because they're not true. Even good mathematicians believe (hopefully temporarily) false propositions. So the argument "some true things are not provable by formal logic in S, therefore if a human can believe something not provable in S, they must not be a formal system" is valid, but does not imply "no machine can be constructed that believes something not provable in S" since a little sprinkling of randomization can make anything possible. A machine that can prove any true proposition with p>2/3 and only believes false propositions with p<1/3 would be pretty useful.
Indeed. To make such a claim demonstrates a misunderstanding of what formalism is. When human beings develop formalisms, they do so on the basis of semantics. If I wish to formalize something, I must first have a relevant understanding of the nature of the thing, or things involved and the relations between them. The development of formalism is a translation of semantics into syntactic rules devoid of meaning. Doing so allows us to mechanize reasoning. Algebra is an example of an early move in this direction. Something we learn in mathematics is that two theories concerning different objects can be formally equivalent.
Computers are strictly devoid of semantics. They are, at best, purely syntactic machines. There is no amount of syntax than can ever lead to semantics. Strictly speaking, computers don't even compute.
How does one create a machine with such untied hands? The problem I have here is that while you can run any formal system on a universal computer, you do so by embedding the semantics of your formal system into those of some machine, and you have to make changes from outside that embedding (writing a new interpreter) if you decide you want to shift formalisms. Meanwhile, humans can't step outside the computation their neurons run and reprogram their base method of thought (as far as we're aware, and distinguished from reprogramming one's set of premises), but can change the formal system they're simulating (as one does frequently in math and the sciences).
In short, I agree you entirely with except that I don't see how we can design a machine with hands as untied as ours. Given a mechanistic view of the brain one concludes that it must be possible, but I still don't understand what formal trick would enable this difference in power. It does seem to give up consistency somehow (humans make mistakes) but not entirely (humans can detect mistakes they've made). I'd love to hear more thoughts on this matter.
Lots of folks missing the forest for the trees. Objecting to Penrose' argument on the grounds that it assumes Humans are logically consistent when that is no the case misses the point.
The claim is that you can not adequately model the Human mind with a Turing machine. Pointing out that Human's are not logical only strengthens this claim and does not refute it. Further, the claim at this non-Turing machine-like ability of Humans is valuable and allows us insight that Turing machine is fundamentally incapable of.
See Penrose argument. He argues that the fact that our minds are not Turing machines allows us knowledge above and beyond what Turing machines are capable of. I don't think this necessarily entails inconsistency, but replying to his argument that humans are inconsistent does not refute his argument. That is what I was trying to point out.
> The claim is that you can not adequately model the Human mind with a Turing machine. Pointing out that Human's are not logical only strengthens this claim and does not refute it.
False. "2+2=5" is not a true statement, but I can construct a computer program that purports to compute the sum of 2 and 2 yet produces 5 as an answer. You can object "you got the addition wrong!" but that's irrelevant. If there is a person who believes (falsely) that 2 plus 2 equals 5, then the computer program could be an accurate model of their thought processes. There's nothing about "computers consistently produce the same answer" that implies "computers always produce the correct answer". "Computers always produce the correct answer" is a much more powerful claim, so the onus is on those who make the claim to prove it.
Yes, you can make a computer spit out nonsense, but this is a far cry from the computer actually behaving in an illogical way. A Turing machine's entire operation is logical. It is built with logic. It is a logical machine.
Penrose claims the human mind is not a Turing machine. That it is not logical. That it is not built with logic. That it is not a logical machine.
Pointing out that the human mind is not logical is thus restating Penrose' point and not refuting it.
Penrose is not arguing that the human mind is illogical. He is arguing that it is uncomputable, which is a big difference.
In particular, he's saying that the human mind is capable of outputting true statements that cannot be proven to be true via any bounded number of proof steps. I don't disagree with that.
However, given that the human mind is capable of outputting false statements -- witness this conversation, wherein at least one of us is outputting false statements -- Penrose has failed to prove that the human mind is non-Gödelian, i.e. can output all the true statements that exist, while outputting no false ones.
It's easy to write a computer program that outputs every statement provable from the Peano axioms: start with the most primitive possible statements, then progressively output more complex ones. This is because the Peano axiom system is "recursively enumerable", to use Computer Science terminology. The program never terminates, but any provable statement you name will be output at some finite time.
Additionally, it's easy to write a computer program that outputs every possible combination of symbols. Most of the combinations aren't statements at all. Of the ones that are statements, most of them are not consistent with the Peano axioms. Of the ones that are consistent with the Peano axioms, most of them are unprovable. But if the set of truths is countable, this scheme guarantees that the program will output every true statement that exists.
It is possible to write a computer program that checks if a statement is provable from the Peano axioms. The naïve way is to run the program that outputs every statement as a subroutine, then halt if the subroutine prints out the statement which we wish to verify. This program will halt iff the statement is provable.
It is NOT possible to write a computer program that checks if a statement is unprovable from the Peano axioms. Such a program may be able to detect a subset of unprovable statements. The subset it can detect may even be countably infinite. But there are some statements which will cause the program to run forever.
Let statement X represent a formulation of "the Goldbach conjecture is true" written in the language of Peano arithmetic. Does the program run forever if asked whether X is provable? If you are a non-Gödelian being, you will be able to answer that question with no error, because Gödel's Incompleteness Theorem only applies to formal systems which can prove all true statements.
> A human mind looking at the Gödel sentence while strictly adhering to the same formal structure could also not see that the Gödel sentence is true. Conversely, a machine that is allowed to operate outside of this postulated formalism could perfectly well make an inference that the Gödel sentence is true.
I think the key move you're making here is dictated by this. There isn't a largest model that will be able to capture all such truths (hence Gödel's incompleteness theorem) and what you would need is a system that is able to constantly build further systems else handle the ambiguity and do model-switching.
Not that it's impossible to do, but have we proven that such a system is theoretically possible yet within conventional frameworks?
Thanks for this spark of a comment in an otherwise perturbed discussion. I cannot reply to all the child comments but hopefully I can help elucidate the misconception you are indicating.
> It's a rhetorical trick at best
This is precisely it. It works because 'true' is defined for the formal system but not for the human. It so happens that the humans in question say well 'of course we know what Godel's proposition means and what true is' but that's just because they are familiar with those words in an entirely unrelated sense to the system being described (akin to say the common speech use of 'energy').
Since you need a system at least as powerful to evaluate the propositions of an underlying system and in that more powerful system a similar proposition can be formed.
Anologous to this is Wittgenstein's problem on the irreducibility of rules: that one can always ask for the explanation of a rule, explanation of an explanation an so on ad infinitum. So it is essentially impossible to determine whether a rule is being applied correctly or not. It so happens that we (humans) can stop asking and 'apply'; but I'm not convinced this is something machines cannot do.
Wittgenstein's "problem on the irreducibility of rules" is brainfart at best. Any rule that can be actually _followed_ cannot be logically precise, and vice-versa. If you try to reduce anything from the real-world to pure logic you'll surely end up crazy.
there are abstract rules and practical laws, principles, whatchamacallit. Laws bind abstract rules to real thingies. The ground truth values are right and wrong en lieu of good and bad - basic emotions that are at the basis of experience. Trying to verbalize (the word logic is related to logos, old greek for tongue, language logic) every basic emotion you will surely go irrationaly crazy. Therefore, context is assumed and language is underspecific.
No, it means I think a human mathematician is capable of working within a formal structure by following the rules of that formalism. It's a way of saying that humans can do formal math.
Of course you are correct in suspecting I do assume the human mind is a machine, because we have seen zero evidence that it's not the case, but that's not in any way part of the sentence you quoted.
> When we notice an inconsistency within ourselves, we generally “eschew” it, whereas “if we really were inconsistent machines, we should remain content with our inconsistencies, and would happily affirm both halves of a contradiction”
I feel that only philosophers & logicians are this black-and-white. There are many people content to occupy a midpoint on the spectrum of "This fact is true"..."This fact is false"; some even will accept a range!
One fact I think we can all agree on is that philosophers are: old; white; dead; men; and wrong.
Love too get downvotes with no comments/feedback. Is this about the dead white men? I take it back. I didn't mean it. I changed my mind. I never meant for it to be taken this way. of all the possible interpretations of my words, I intended you to adopt the one where they were both witty and erudite. It's your fault you picked the wrong one. I stand by my words. I am right. Only I am right. I have seen into the heart of Man and I know its true meaning.
1. Let's suppose for sake of argument that humans really can see the inherent truth of "Peano Arithmetic is consistent". That doesn't mean humans violate Gödel's Incompleteness Theorem: it could just mean that humans use axioms stronger than PA.
2. Gödel's Incompleteness Theorem only applies to systems that are perfectly logically consistent. Not sure how Penrose didn't notice, but humans... aren't.
3. When scientists proposed Quantum Mechanics as a replacement for Classical Mechanics, it was on them to explain how Quantum Mechanics simplified to Classical Mechanics in the common case. "Penrose Mechanics" is an even more radical departure — especially from a physics of computation standpoint, as Penrose Mechanics by definition would allow solving at least some of the problems in (ALL - R) in ~polynomial time. Penrose needs to explain how Penrose Mechanics reduces to Quantum Mechanics in the common case.
4. Penrose proposes that (a) there exist new physics, (b) that evolution has learned to computationally exploit the new physics via microtubules, and yet (c) that humans are the only lineage to make use of this feature of microtubules, even though microtubules are found in all eukaryotic cells (from mushrooms to amoebae). From a predator-prey standpoint alone, it would seemingly be a huge evolutionary advantage to be able to compute NP or R functions in polynomial time. (That ability is not _strictly_ implied by Penrose Mechanics, but it's a very likely consequence.) Penrose needs to explain why only humans are taking advantage of the computational power of microtubules, when microtubules have existed for billions of years and across millions of species. (TL;DR: It's the pineal gland all over again.)
#1 Please show how to program a Turing machine to use "axioms stronger than PA"
#2 Please show how to program a Turing machine so that it is not logically consistent at a fundamental level?
#3 Are you questioning the viability of objective collapse theories in general? To date we have no experimental evidence of said theories, but I don't think anyone has suggested they are inconsistent with QM. Are you? If so, please show how...
#4 Show where Penrose has said humans are only species to make use of microtubules and your claims regarding Penrose' theory of objective collapse and having staggering new computational speedups are ... without any evidence. Please provide some.
The tenor of your comment suggests you find Penrose to be an idiot and missing obvious problems. Consider that perhaps you are the one misunderstanding what he is proposing and that he is not, in fact, an idiot.
1. Easy. Program a computer with the axioms of ZFC. Not powerful enough? Program it with ZFC+Con(ZFC). Repeat as necessary.
2. The Turing machine itself is logically consistent. The semantic interpretation of its data need not be logically consistent: it's possible to write a computer program that prints "2+2=5" to the screen. Likewise, a human brain may be made of physics and physics may be logically consistent, but the semantics of the data in the brain (the "beliefs") need not be consistent.
3. No, no need to reject collapse itself (though I'm more of an Everett guy). Penrose postulates the existence of a new level of physics that is not Quantum Mechanics. People who aren't careful call it an extension of QM, but QM's computing model is limited to solving a small subset of NP problems (BQP) in polynomial time, and BQP is theorized to be a proper superset of P. However, Penrose's proposed physics would allow solving problems outside of NP in polynomial time. Hence, Penrose's physics IS NOT quantum physics.
4. It is physically impossible for computers (as we understand them today) to solve problems outside of the set we call "Turing complete". Penrose claims that there's at least one problem outside that set that the human brain can nonetheless solve: checking the consistency of formal systems. If Penrose's assumption were true, computers (as we understand them today) would consequently be strictly less powerful than the human brain. The crux of Penrose's position is accepting this consequence as true, then asking how that's possible. I reject that hypercomputation of problems beyond Turing-completeness is what's happening, i.e. that humans simply hold unjustified beliefs about the answers to such problems.
I do not doubt that Penrose is an intelligent and educated person. However, his expertise is General Relativity. It's bad enough when he tries dipping his toes into Quantum Physics, but when he wades into Physics of Computation, he is no longer acting as an expert, but as an interested layman. It's only because of the authority he holds as a GR expert that his argument is treated seriously, but that authority is not actually relevant to his argument.
#1 Not easy. Prove it. Show a Turing machine programmed with ZFC that can not be modeled by a Turing machine programmed with PE. You can not do so because Turing machines are universal. The fact that you claim this is easy tells me you do not understand what a Turing machine is.
#2 See fundamental level. Penrose claims that what sets us apart from Turing machines is fundamental. But whatever, you've refuted whatever point you wanted to make by saying human's are not logically consistent. Your snark about Penrose not understanding is as empty as whatever point you were trying to make.
#3 Cite a paper showing that objective collapse gives these supposed algorithmic speedups or go home. NOTE: objective collapse theories are an active area of research and are not limited to Penrose by any means. Your claim that they are inconsistent with QM begs for evidence. Cite some or stop spreading nonsense.
#4 Your answer here is completely void of any context to the question: where Penrose says that microtubules are limited to humans. I take it you concede that he did not say any such thing?
You treat Penrose as an idiot missing obvious problems. I think it far more likely that you've misunderstood. Pity your lack of humility might make it impossible for you to understand what he actually argues rather than your strawmen.
> You can not do so because Turing machines are universal.
Just because Turing machines are universal making it possible to write a program that asks a question about another type of Turning machine doesn't mean that question is decidable.
Show a Turing machine programmed with ZFC axioms that can do something that a Turing machine programmed with PE can not do. If a Turing machine programmed with PE can simulate the Turing machine with ZFC and thus give the same answer, then I state that the program with ZFC is nothing of the sort.
The paper[1] I linked to in my other post shows that you cannot perform such a simulation because it is undecidable. (Specifically, BB(N >= 1919) is provably undecidable in ZFC).
I read that when it came out, but I don't see how that is relevant to this question. The link provides a program for a Turing machine that probes ZFC. So can you create a Turing machine that probes ZFC, but can't be proven not to halt? The link emphatically says yes! And it also puts some bounds on the number of states necessary to program such a machine.
Please show how that paper describes a Turing machine 'programmed with PE' can't simulate a Turing machine 'programmed with ZFC'?
Perhaps we are arguing over what 'programmed with PE' or 'programmed with ZFC' means? The parent post seemed to claim that it was possible to construct a physical computer with the axioms of ZFC built-in so to speak. As opposed to one with PE built-in. Which obviously calls into question the parent post understanding of what exactly idealized Turing machines are.
There is just tons of reading on this if you care to scour the literature. Penrose' objective collapse theory is usually given as the prime example of the whole genre.
I already read through https://en.wikipedia.org/wiki/Penrose_interpretation when I was trying to figure out what nonsense you were spewing. The article does not mention the Church-Turing thesis, hypercomputation, Gödelian incompleteness, or the Lucas-Penrose argument.
People can be known for more than one thing. Those two things can be unrelated, even if they're in the same field.
In Penrose book that introduced his thoughts on this argument he devotes an entire chapter to his objective collapse theory and quantum gravity. You asked for a paper and I gave you many. You want to know more pick up his book.
Also: before I bother building a Turing machine that implements a proof-generator for statements in ZFC, you should do me the courtesy of showing your investment by building me a Turing machine that multiplies two integers.
Turing machines suck. Building a Turing machine that implements ZFC proof-generation is a project appropriate to a graduate-level paper, not something to toss off in an Internet pissing contest.
"... a one-tape, two-symbol Turing machine with 7,918 states, whose behavior (when run on a blank tape) can never be proven from the usual axioms of set theory, under reasonable consistency hypotheses."
1. If Lucas and Penrose are right, there are statements which are unequivocally true, but which cannot be proved in any known formal system. It should be possible to exhibit a few. (Extra credit if they avoid theology.)
If the only such statements are artificial Gödelian sentences, I'm afraid I'm...uninterested, in the same way Lucas is uninterested in inconsistent humans.
2. Lucas and Penrose assert that some entities are significantly more powerful than others. To my knowledge, Lucas describes no mechanism for this. (To his credit, Penrose does, but, to my knowledge, the Bell inequality among other things means quantum mechanics is fundamentally random. That's not helpful.)
What could be the nature of such a mechanism? If it's material, it should be possible to include it in future machines. If it's not, we are in the realm of theology again.
3. So humans are intelligent and intelligent-seeming machines only simulate intelligence. If the difference is not common, obvious, and readily meaningful, then where is the distinction? ("So what? I can't write an opera either.")
Sigh, they don't claim that. You are also arguing with straw men. I wish people would read what Penrose actually argues instead of just assuming he is an idiot (one of the most brilliant minds of this or any century) because he argues against Strong AI.
There are good refutations of Penrose' argument, but I've seen none of them here. Just a bunch of bandwagon hackers assuming Penrose doesn't know what he is talking about and then putting up lazy straw man takedowns.
Don't claim what? That there are statements that are known to be true but which cannot be proved, or that the human mind is strictly stronger than a machine?
Edit: Or perhaps that Penrose is not claiming that QM is some kind of super secret sauce that makes multicellular life intrinsically stronger than computing machines?
To be clear, I'm speaking of Penrose specifically. Penrose goes no further than Godel. He doesn't add anything to Godel's argument about formal systems and statements.
All Penrose is saying is that Turing machines can not be the basis for modeling the human mind. He is not saying that the human mind is not governed by physics. He is not saying that the universe is not governed by physics. And he is not resorting to 'quantum woo' like some here are saying.
Penrose' argument is that there is something non-algorithmic going on in the human brain and that this non-algorithmic process is governed by physical laws. He then goes on to discuss objective collapse theories for quantum mechanics and how that may be the source of non-algorithmic behavior in the human brain.
To be clear, objective collapse theories have as of yet no experimental evidence. But, crucially, they are not inconsistent with QM as some above have said. Also, the precise biological mechanism that Penrose postulates for how the brain operates with objective collapse is highly controversial ie, microtubules.
Models of computation have some interesting properties. One is that all of the "reasonable" ones are less powerful than Turing machines (FSMs, stack machines, etc.) or equivalent to Turing machines (recursive functions, combinators, TMs with multiple tapes, etc.). It's effectively a closed category.
There are models that are more powerful than TMs, but the ones I know about have "unreasonable" powers. The one that comes to mind came from A.K. Dewdney (I think); it does an unbounded amount of work in a single time step by having an unbounded number of computational units. And that is just not kosher.
So there is a certain amount of circumstantial evidence for the definition of "computable" being something like a Turing machine. And this definition of computable has limits, as in Godel (never thought I'd miss my phone's keyboard) and Turing.
As you describe, Lucas, Penrose, and others such as Searle want to walk a very fine line. They want to claim that the human mind is not limited in the way that computable functions are limited, which I understand; it certainly doesn't feel like I am so limited. On the other hand, they want to avoid going off the edge into full-blown mysticism: dualism and ye immortal soul.
Unfortunately for those like me (and I know I am) who take materialism somewhat seriously, their arguments feel more like the post-Darwin arguments that, yes, animals and stuff do evolve like that, but people are different. The closed set of computational models of TM-equivalence do seem to capture "computable functions", and those computable functions do seem to cover what is possible from any system built on biology, chemistry, and ultimately physics. Their arguments seem to be either making a distinction that doesn't make a difference, or sneaking in supernatural powers. (You can see the truth or falsehood of a statement without a proof? Reliably?)
So arguments for a non-algorithmic middle ground between TM-equivalence and "quantum woo" (great phrase, btw) are giving me problems. If I am a mixed bag of incomplete and inconsistent, with limited rationality that is not well described as a formal system but is weaker than a Turing machine, as I suspect I am, then the appearance of a non-algorithmic process is simply a misunderstanding.
On the other hand, if the non-algorithmic goings-on are correct, "quantum woo" feels like the most reasonable description of their powers. I'd really like some strong, unarguable evidence before I accept that step.
Let's take the question of the human mind and whether it can be modeled by a non-quantum Turing machine out of the equation.
Penrose's contention can be summed up without it. He basically thinks that gravity will be discovered to play a role in quantum wave function collapse. And that this role will render non-quantum Turing machines incapable of accurately modeling quantum gravity in some very serious way. Another way of saying it is that he believes when a real theory of quantum gravity is developed we'll find that non-quantum Turing machines will not be able to accurately model the physics of a black hole.
Note that this says nothing whatsoever about the human mind or microtubes or formal systems, etc. This is about his subject of expertise: gravity, mathematical physics, and quantum gravity. He's one of the world's leading experts in these fields.
And if he is right, he conjectures it will have strong implications for non-quantum Turing machines as a basis for Strong AI.
That's a good point. I'm tempted to point out that quantum computers can currently be emulated by classical computers, but that may be beside the point.
Yes, quantum supremecy has not been demonstrated, but the question of whether non-quantum Turing machines can accurately model all physical laws has not. Of course, we don't even have a quantum theory of gravity for them to model at this point...
Would you please not argue like this here? We all understand how annoying ignorant internet dismissals can be, but if you respond merely by excreting bile into the system, you're making it even worse than they are.
Among your other comments there are some that are quite substantive (yay) and others like this one (boo). Kindly stick with the substance even if other people aren't. That enhances credibility and helps tend the garden for everyone.
> there are statements which are unequivocally true, but which cannot be proved in any known formal system
No. You are misunderstanding Godel's theorem. For any given formal system there exists a Godel sentence which the system cannot prove. But the Godel sentence for every system is different. Moreover, there is no algorithm for constructing a Godel sentence. If there were, then a TM could follow that algorithm to construct its own Godel sentence.
So every formal system has a (different) Godel sentence, but there does not exist a universal Godel sentence that is unprovable by all formal systems.
Constructing the sentence is algorithmic. The key part is asserting that the sentence is true. For instance ZFC will prove that A:(ZFC cant prove GodelStatement(ZFC) OR ZFC is inconsistent).GodelStatement is an algorithmic function which takes a list of axioms in some formal framework.
Now, lets make ZFC' which is ZFC + 'ZFC is consistent'.
ZFC' can prove A(as it contains ZFC) and uses the consistency of ZFC to prove A':(ZFC cant prove Godel(ZFC)) .
Similarly, a program decides a set of problems(like does this Turing machine halt or does this diophantine equation have solutions) with access to its source code will be able to algorithmically generate the problem instance on which it will not halt. You can modify the program to create a new program which will use this information, but it will be a new program.
> Similarly, a program decides a set of problems(like does this Turing machine halt or does this diophantine equation have solutions) with access to its source code will be able to algorithmically generate the problem instance on which it will not halt. You can modify the program to create a new program which will use this information, but it will be a new program.
So this all depends on what you consider a "new program". Suppose we write a program that actually does this: analyzes its own code, algorithmically produces the Godel sentence for that code, then modifies its code to include that as an axiom, forever. That program is a program, and that program will also have a Godel sentence, but one which it cannot construct using that algorithm. You need a different algorithm to construct Godel sentences for programs that constructs Godel sentences on programs that don't construct Godel sentences.
The result is a hierarchy of Godel-sentence constructors. At any level in the hierarchy the procedure is algorithmic, but going from one level to the next is not. It's similar to the process of constructing the hierarchy of ordinals, which is also provably non-algorithmic.
There is a program, G, which will take as input the source code of the program, P (in some fixed language) and output Turing machine, T which doesnt halt, such that either P will fail to say T doesnt halt, or P has mistakes(claims halting for some Turing machines when they dont halt). Some programs P can algorithmically run P', P'' etc, but as long as that process is part of the source code of P, G will work.
Yes, but I'm the one who made the original claim that "there is no algorithm for constructing a Godel sentence". So I'm the one who gets to say what I meant by that :-)
True. Both haves of what I asked for are stronger than what Godel demonstrated.
But for this argument to be applicable to the Theory of Mind, it seems to me that it is exactly what Lucas and Penrose are claiming to exist.
If the only un-provable sentences are Godelian constructions, the whole issue isn't philosophically interesting. But that's not true, to my understanding.
So, there are statements that cannot be proven in, say, ZFC. Some of these statements are, I assume, "interesting"; they aren't simply constructs that are designed to be unprovable. Finally, for Lucas and Penrose's arguments to be meaningful, some of those statements should be obviously true or false---that is the only way for the human mind to not be limited in the way ZFC is limited.
If the truth of the statement is not obvious, then I'm not clear on how the human mind is stronger than the formal system in question. If the truth is obvious, but the statements are all variations on "I am lying" designed specifically for a given formal system, then I don't see how this is an actual interesting property of minds.
> If there were, then a TM could follow that algorithm to construct its own Godel sentence.
I don't understand what's the problem here. TMs are allowed to be inconsistent. There's a TM that outputs every single finite string. What would even count as its Godel sentence?
In fact, there is an algorithm that constructs the Godel sentence given a consistent formal system capable of arithmetic. That algorithm is spelled out in Godel's original proof. Now, of course, a TM can only do that to its own formal system if it's inconsistent itself, which means the "Godel sentence" it output is meaningless.
Sure, but that's an uninteresting case. Any inconsistent system that includes basic propositional logic asserts all statements to be true. So in these discussions consistency is generally assumed.
Refuting Lucas is very simple: Godel sentences can only be constructed for circumscribed systems, i.e. a system where you have written down all of the axioms and inference rules. So the only way to circumscribe a human brain is to isolate it from its environment. As soon as you let it communicate with (say) another human brain, you can no longer Godelize it because that brain is not the system any more, it's the two communicating brains working together. That system might have a Godel sentence of its own, but it's not the same as the Godel sentence of individual brains.
So if I can put you in an isolation tank and model your brain, then I can in fact (at least in principle) construct your Godel sentence and know for certain that you will never come to the realization that this outrageously complicated mess I've just constructed (the one that says of itself that you will never come to see its truth) is true, and I can feel smug about the fact that I can see that this is true and you never will. But here's the kicker: I can never gloat about my superiority to you, because as soon as I do you are no longer in isolation. You are communicating with me, and the sentence I've take great pains to construct is no longer your Godel sentence. It is the Godel sentence of you in isolation, but as soon as I tell you your Godel sentence -- indeed as soon as I (or anyone else) say anything to you, my construction falls apart and I have to start over.
Ironically, far from proving that we are not machines, Lucas's argument actually goes a long way towards demonstrating that we are, because coming up with examples of things that we interacting brains know to be true that a particular isolated brain will (with extremely high probability) never come to realize is not difficult. The four-color theorem, for example, is probably beyond any single isolated human brain. Indeed, the very argument you are reading now is probably beyond any single human brain because the brain that's writing it would never have come up with it had Godel and Lucas (and Turing) not paved the way.
If I said "yes" I would be refuting my own argument :-)
I composed the words in the above comment, but I don't actually know if the substance of that argument is original with me. It is heavily based on things I've learned in other places, but I don't have any specific recollection of having seen the argument being framed in exactly that way before. It's possible I invented it, but I doubt it. It's not a particularly deep insight, a minor variation on a deeper theme, that "meta-Godelization" is not algorithmic, a fact that I first learned by reading Hofstadter's "Godel, Escher, Bach." I don't actually know what the original source of that result is (I should probably go find out).
But I'll happily take credit for the pedagogy if it somehow turns out I got there first.
Does your argument hold if you consider that input to the brain is limited to the reception of electromagnetic/mechanical waves through a set of sensors, being the fact that one can transmit speech (and Godel sentences) through those sensors is just a higher level abstraction?
Put in another way: Couldn't I in principle define such a circumscribed system with all axioms and inference rules by including all of those sensors and all of their possible states into the system?
Yes, of course you can. But if you are part of that system, then you can't Godelize it. In order to Godelize a system, not only does the system need to be circumscribed, but the thing doing the Godelizing has to be outside of the boundaries of the circumscription.
Let's recall the original argument:
"a mechanist formulates a particular mechanistic thesis by claiming, for example, that the human mind is a Turing machine with a given formal specification S. Lucas then refutes this thesis by producing S’s Gödel sentence, which we can see is true, but the Turing machine cannot. Then, a mechanist puts forth a different thesis by claiming, for example, that the human mind is a Turing machine with formal specification S’. But then Lucas produces the Gödel sentence for S’, and so on, until, presumably, the mechanist simply gives up."
When you are presented with S there are two possibilities: either S includes you, or it doesn't. If S includes you, then you can't Godelize it. If S doesn't include you, then you might be able to Godelize it, but if you present your result to the thing that S is supposed to be a model of -- indeed as soon as you interact with that thing in any way -- then S is no longer an accurate model of that system because now it is interacting with something that, by assumption, was not part of the model.
No. I'm neither assuming nor showing that the human mind is a machine. All I'm showing is that the Lucas-Penrose argument that the mind is not a machine is invalid, because it glibly assumes that the human mind is capable of Godelizing any possible mechanistic model of the mind, and that is not necessarily true. If the mind is a machine, then it will not be able to Godelize a mechanistic model that includes that mind as a component. The best any given mind can do is construct the Godel sentence for a mechanistic model of some other mind. But that does not show that the first mind is not a machine.
Franky it's irrelevant. Penrose's insistence that the human mind isn't algorithmic in the Turing sense is born from his human insecurities. But if the human mind isn't bound in such a way, than any man-made computer need not be bound either. Penrose's argument falls short of being a proper dualist argument. Furthermore Roger Penrose has not proven his claims, and they are not supported by mainstream neuroscientists. He appeals to quantum woo that he pulled out of his ass.
It's not ok to argue like this on HN, so please don't. It lowers the quality of discussion in its own right and sets off a downward spiral from others.
Edit: Actually, we've banned your account. Your comment history shows that you've broken HN's civility rule so egregiously, and so repeatedly, that we should have banned it long time ago. If you don't want to be banned on HN, you're welcome to email hn@ycombinator.com and promise to follow the rules scrupulously in the future.
Well, I don't think our brains work by applying "logic" when we think about Gödel's theorem (or anything else). We evolved to do pattern matching, so each of us spends decades to learn more and more sophisticated pattern matching techniques, starting from mom and dad's faces to counting things to multiplication tables to, finally, understanding Gödel's theorem (for some people).
In light of this, we can reinterpret "However, a human can look and see that the Gödel sentence is true" as:
When you take the best pattern matching machine nature has produced, train it with distilled essentials of centuries of mathematical geniuses, and present it with the Gödel sentence, the pattern matcher strongly signals "This looks true!"
When viewed this way, it looks less like an inherently human-specific achievement, but more like exactly how ML algorithms work. The only difference is that ML cannot quite pull it off, so far.
Penrose argument, while it can be challenged, is actually much more careful than dismissals here suggest.
For instance people talking about 'humans are not consistent', we are talking about effort directed into a well specified formal domain, and after long term reflection.
So, of course, people make mistakes in mathematics all the time, but we are talking about consistency after intense review over long period of time. We write a program, and cant find the bug even if we look at it for a google years.
To the claim, 'Godel statement is not interesting relative to other statements', the goal is that to demonstrate non-algorithmic nature of some human activity, which itself would be a big deal if true.
A more qualified version of the LP claim can be expressed as follows 'One of the following is true 1) The activity of humans in some formal domain(like mathematics) is inconsistent in a deep way(not routine mistakes). 2)Humans are reflective consistent, algorithmic but will never have access to the source code(a claim about limits about brain research). 3) We are reflective consistent, can access source code, but wont be able to see why the source itself is consistent.'
142 comments
[ 3.4 ms ] story [ 193 ms ] threadTo me, this seems to leave Lucas with the burden of proof, and until he or someone else can counter this objection with something more than an opinion, the argument is dead in the water. I used to wonder why philosophy would hang an elaborate argument on personal opinions, but I have come to appreciate that whenever you speculate about matters where there is insufficient evidence to settle the issue, it always comes down to opinions, though this is often overlooked in the discussion.
(That's not intended as an endorsement of Lucas-Penrose. There are deeper flaws in the argument, like the assumption that the truth of the knowledge of the mind implies that the mind KNOWS its own truthfulness.)
The Gettier problem: https://en.m.wikipedia.org/wiki/Gettier_problem
I assume you already know about the concept of "commitment" in Philosophy, but I'll explain it briefly here in case anyone else comes across this comment.
Sometimes while you can't show that a fact is categorically false (given some available evidence), you can show that people who believe that fact are committed to believing other facts. In this case Lucas' own argument has committed him to the "personal opinion" that an inconsistent machine cannot be an adequate representation of mind. But his belief isn't entirely unfalsifiable. If new evidence makes him give up this belief (e.g. a rise of paraconsistent logics) he'll have to give up his original belief along with it.
Human minds are clearly not consistent. Human minds are a loose collection of cobbled-together heuristics.
Logic is one of those heuristics. It has limits because ultimately "logic" ends up being not a personal opinion, but a personal experience of the truth - or otherwise - of a proposition.
Developed and trained minds have generated a heuristic of independent consensus about truths and experience, and strategies for increasing the correlation between the truth experience and the reality experience.
Less developed untrained minds have a much more naive and subjective experience of "truth."
But both work on the same foundation - a nebulous and indefinable subjective experience that is associated with truth, and is typically mistaken as the definition of it.
It's impossible to "prove" truth, because the experience of deciding whether or not something is true is absolutely subjective.
Any attempt to make an objective definition will always be filtered through that subjectivity and reduced to either having that experience or not having it.
We're back to qualia, as usual. No one knows what "having an experience" really means. Until that question is answered, there are no tools for dealing with the rest.
You can argue "But what about the scientific method, and mathematical proofs?" Same answer - they're practices that generate the truth experience in trained minds. They have a better predictive record and are better at finding consistent patterns than the disjointed heuristics of untrained minds. But they still don't - and can't - prove "truth" in any kind of ultimate definitive guaranteed-to-be-objective way.
They certainly can't guarantee that the truth experience is never biased, contingent, limited, naive, circumstantial, or possibly even just plain wrong.
The implications for AI are obvious. If a machine is better at finding consistent patterns, it will appear smarter than a human. It will still be smarter even if it makes errors that are obvious to a human, because at the same time it won't be making errors that humans do make, and which are similarly "obviously wrong" to the AI.
It's the hit rate that matters, not idealised philosophical perfection.
Think of it as analogous to all of the mathematical quasi-proofs that assume certain conjectures are true. They can often be valuable steps to an actual proof, as we saw with Fermat's theorem.
In this particular case, though, it seems close to circularity - if the primary purpose of the argument is to refute the computational theory of mind, then ruling out a whole class of machines a priori looks to me like begging a chunk of the question - though I appreciate that it leads to a better-defined question.
Anyway, there is a manifold in philosophic space in which proving things to win arguments matters and a manifold where torturing philosophy with the scientific method matters and a manifold in which there is doubt about the external world and one in which confusion arises from lack of rigor regarding language and one in which the goal is to prove the existence of gods and a manifold of other manifolds.
We all get to take our pick because philosophy is a big tent that is big enough that a philosopher is free to wander among the lion tamers and clowns and acrobats and look at what is interesting. Radical skepticism is a useful intellectual tool, but so are Platonic idealism and deconstruction and Kantian metaphysics. Anyone who thinks they've got it all correct is probably ignoring human fallibility (but maybe not).
Which would obviously be false. There are clearly true theorems we will never be able to prove. Simply because we can prove many theorems that simpler formal systems cannot, entails nothing about our logical consistency or completeness.
Now valid arguments can be interesting in their own right even if their premises aren't true.
Given this, the proper way to read Lucas's opinion is: I am taking it as given that the computational theory of mind requires a consistent system of inference rules + math; if that isn't true, my argument might be unsound but not invalid (and its validity is interesting); but I also think it is true for x,y,z reasons (though these additional reasons do not constitute a proof).
In other words, the Lucas-Penrose argument is interesting from both a truth-of-premises and validity perspective which is far from dead in the water.
Gödel didn't prove that. He saw that it was true.
"So the following disjunctive conclusion is inevitable: Either mathematics is incompletable in this sense, that its evident axioms can never be comprised in a finite rule, that is to say, the human mind (even within the realm of pure mathematics) infinitely surpasses the powers of any finite machine, or else there exist absolutely unsolvable diophantine problems of the type specified" . . . (Gödel 1995: 310).
http://math.stanford.edu/~feferman/papers/penrose.pdf (PDF)
For example, I perceive my own thoughts rather than other people's. This implies that my own thoughts would have an additional physical property, that of being perceived by me, which the thoughts of other people lack. However, according to all up-to-date theories of physics and neurology, there is no difference between my own thoughts and other people's that would explain the presence of this additional property.
The perception of ones own thoughts seems very physical, as is the equally obvious (to me) explanation that you can't directly perceive the thoughts of another.
It's not clear why you would expect this to be inherently different from any other neurological response. For example, if I stub my toe, are you suggesting that science doesn't have an answer to why you don't feel it?
I am, as luck would have it, connected in a more or less identical fashion to the thing creating my thoughts (me).
Physics is ok with all of that.
There's no current evidence that it is fundamentally outside that domain.
That is not a difference between human and machine. It's a rhetorical trick at best, and an unreasonable appeal to mysticism at worst.
A human mind looking at the Gödel sentence while strictly adhering to the same formal structure could also not see that the Gödel sentence is true. Conversely, a machine that is allowed to operate outside of this postulated formalism could perfectly well make an inference that the Gödel sentence is true.
The Lucas-Penrose argument boils down to a questionable attempt at a logic bomb. So by first tying the machine's hands, they argue that the machine's hand's are indeed tied, and therefore humans - who in this example have been given free rein - can do something that you explicitly disallowed machines from doing.
There could be many that could answer some such questions, paraconsistent and inconsistent for example.
Human biology, and by extension human minds, follow a myriad of rules that are a result of biological processes being played out. Likewise "machines" also follow the rules laid out by their physical design. There is no fundamental difference in this regard, there is no new set of information-theoretical rules that comes into play when you switch to a carbon-based machinery.
> What formalism, if any, is the human mind subject to?
In the context of this argument, the mind of the human mathematician would be subject to the rules of the formal system. Since Gödel we know that such a formal system cannot be proven true inside the system itself. However, an entity that views the system from the outside can potentially see it's true. There is no fundamental difference between a human mind and a hypothetical machine mind as far as the capability to come to the same conclusion is concerned. That's why Lucas took special care to specify upfront that the machine mind is prohibited from coming to that conclusion, by confining it to live within the formal system only. Yes, their argument is really that circular.
> Isn't our current conception of a machine that it will bound to some formalism in terms of reasoning? Are we so bound?
We are not, and neither would a generally intelligent machine. The burden of proof is on Lucas/Penrose to show that a generally intelligent machine would still be bound to that formalism, at which point it would cease to be considered generally intelligent. They seem to argue that their postulate is true because the individual components of such a machine are bound by those rules, but then again so are the individual components of our brains.
Do you think Penrose disagrees that the Human mind is bound by physical laws??
No, it is not. It is far from proven that universal physical law can everywhere be simulated by a Turing machine. If you had proof of that you'd be up for a Nobel. So publish your paper or perhaps consider that Roger Penrose is not an idiot and you might not know what you are talking about.
> Do you think Penrose disagrees that the Human mind is bound by physical laws??
Yes. Having read his work in the past, I know that he disagrees the human mind is bound by physical laws. This is the whole reason behind the argument they're putting forth to begin with: they postulate that humans are capable of making an inference that could not possibly be made by a machine intelligence.
However, it has never been shown that we are not a machine intelligence.
Cite one place where he says anything like this or go home.
What I - and I hope most people - mean by the phrase is different though, it's the body of actual physical laws that have been shown scientifically.
> I'm not sure what his position is on computers that incorporate quantum magic though.
Interesting thought experiment. I suspect he would be fine with those in theory, however since he already declared this special connection to the supernatural for humans it'd be interesting what computer components he would consider equivalent. But the beauty of his position is that, in the event of a generally intelligent computer emerging, he could always claim that any given nanoscale computing component is the silicon analogue of "quantum microtubules" (which was his woo-related organelle of choice in neurons). It's not like he ever has to prove his claims in order to be taken seriously.
Few people have made more contributions in the twentieth century to actual physics than Penrose. He is one of the world's leading experts on general relativity and his objective collapse theory is nothing more that when quantum gravity is understood that gravity will play a role in the collapse of the wavefunction.
To describe Penrose as engaging in 'quantum woo' or 'quantum magic' is grossly inappropriate. It just gives evidence that you don't understand what he is proposing. See Scott Aaronson for what an rebuttal to Penrose looks like not the ham fisted straw man takedowns from the likes of people in this comment section.
Penrose does not argue for "something computationally more powerful than objective collapse" rather, he argues that the physical Church-Turing thesis is false due to quantum gravity. Something he is an expert on, btw.
Says who? Penrose is blacklisted, cite someone else.
That falls far below the standard of discussion we ask users to abide by here. Please make your points directly and without such cheap insinuations.
If you think my comments in their entirety detracted from the discussion, I'd like you to tell me. Otherwise I'm asking you to consider this statement in the context of both manyoso's and my comments as a whole.
This is a discussion about quantum quackery, and you're stepping in on behalf of the pro-quackery side that has no problem telling me personally that I "have no idea what" I'm "talking about", and other dismissive discourse.
I am sorry to have said "go home" and will refrain in the future. Still, you have lodged what I consider factually incorrect assertions about Penrose's argument. When asked for evidence you have provided none.
> "We proposed [...] that consciousness depends on biologically ‘orchestrated’ coherent quantum processes in collections of microtubules within brain neurons, that these quantum processes correlate with, and regulate, neuronal synaptic and membrane activity [...] This orchestrated OR activity (‘Orch OR’) is taken to result in moments of conscious awareness and/or choice. [...] We conclude that consciousness plays an intrinsic role in the universe." ( Stuart Hameroff, Roger Penrose; Consciousness in the universe; Physics of Life Reviews, Volume 11, Issue 1, March 2014, Pages 39-78 )
and
> "If gravity plays a role in wave function collapse that will pose serious questions for the Church-Turing hypothesis. It might very well make it impossible for non-Quantum computers to accurately model quantum gravity. Penrose does not argue for "something computationally more powerful than objective collapse" rather, he argues that the physical Church-Turing thesis is false due to quantum gravity. Something he is an expert on, btw." ( manyoso, Hacker News, discussion id 14444443 )
None of these statements, while they are using the language of science, bears any discernable relationship to physical reality in a scientific sense. The postulate that any consciousness-related activity of neurons happens in structural elements like microtubules is a collection of unproven hypotheses divorced from both empirical and theoretical experience. The same goes for this preposterous claim about quantum gravity: there is currently no workable theoretical framework for quantum gravity, but this doesn't stop Penrose from confabulating properties onto the term so it can become a new go-to locus for his ideas on quantum consciousness.
An observer will notice a pattern here: mystical truth claims about a hypothesis with no basis in either experimentation nor theory are being "supported" by hiding their supposed mechanisms of action inside yet-unexplored regions of the map. It's a god of the gaps argument.
Now, you know this, as does Penrose. Both of you are trying to preempt this criticism by professing quantum consciousness is a fundamental part of the laws of nature, and hence it should not be categorized as woo. You are essentially rejecting the accusation of mysticism by asserting that magic is not metaphysical. Which was the observation dang chided (and apparently penalized) me for.
For what it's worth: you decisively won the argument, both by popular opinion and the support of the site's admin. Accept a victory when it presents itself.
Commenting here requires being scrupulously civil, even when others are not. Any lower standard would lead to an unstable system.
When someone asserts in an authoritative tone that quantum gravity is the key to unlocking consciousness at the heart of the universe, that is utter hogwash and deserves to be called out as such. You may personally not like my pointing this out, but the entire "field" of quantum consciousness is a religious movement that makes unsubstantiated claims about the nature of reality.
It actually is quackery. Redefining the term 'physical laws' to include magic in order to shield an opinion from criticism is actually sophistry. Insisting that spiritual convictions are factual statements about the universe is actual fundamentalism. These aren't bad words wielded to hurt someone, they are supposed to convey meaning.
I never meant to denigrate anyone personally. I strongly feel that opinions must be fair game though when it comes to criticism. In doing so suppose I must come across more harshly than I intended or maybe I'm just exceptionally bad at articulating myself. Whatever my intent, it's clearly not working out all that well. It's probably a good idea for me to take a break from HN for a while.
Last I heard, Penrose appeals to quantum woo to claim that the human mind isn't strictly algorithmic. But sources of quantum randomness isn't enough to push you into dualism, quantum shit is part of our physical universe and while there is no proof that the human mind critically exploits quantum phenomena for it's computation, we can certainly build machines that have RNGs exploiting quantum randomness.
Penrose doesn't appeal to quantum woo, he has a quite well articulated set of assumptions and arguments , related to loop quantum gravity [1], about how objective collapse of the wave function may occur at bio-physically feasible decoherence times.
Now, whether you think his biophysical theories of consciousness are valid, or even required (whether you buy the microtubules argument/hypothesis), is one thing. But stating that Penrose- arguably the 20th century's foremost mathematical physicist- is a practicioner and/or a spreader of 'quantum woo'-- that belies a level of mis-informantion so catastrophically high that it renders your quoted statement indistinguishable from a pure ad-hominem.
[1] https://www.wikiwand.com/en/Loop_quantum_gravity
It's worth noting that Penrose's field of expertise is General Relativity, not quantum physics, and definitely not Computer Science / philosophy of computation. I see the situation with Penrose as equivalent to Linus Pauling's unfortunate foray into Vitamin C pseudoscience late in his life.
Say what you will about Penrose, but can you seriously deny that he is one of the world's foremost experts on gravity?? You might not like his conclusion, but to say that he is out of his element here is ridiculous. Are you in your element when discussing gravity causing wave function collapse? Who do you think would be more in their element on this subject than Penrose??
Its true that LQG is not known a natural schema to implement hyper-computation. However, since we don't well understand the time-dynamics of twistors ( how these operators may interact non-linearly through time), i don't see any a-priori reason why it is not a possible scheme for super-computation (you could make a good counter- response based on occam's razor, which I'd grant).
Look, lots of thereotical CS folks get disturbed by the idea of hyper-computation/super-turing machines, but in truth Turing machines are a toy model in comparison to true physics; as such, it doesn't take a whole lot more to get something more powerful; Siegelmann and colleagues have shown that real weighted, analog recurrent nets have super-turing abilities [1] [2]. While Aaronson and other raise good questions about physical realizability of such systems, a good thing to keep in mind is that these discussions often take place at computational 'limit' cases, eg solving intractable PSPACE problems, which may not be as relevant to more pedestrian problems solvable by biological systems. Central point: dynamically evolving systems iteratively exploring through (from) in-consistent systems towards more and more consistent ones have many of the same compelling qualities we would call 'super-turing'. Also see the lit on evolving turning machines.
Finally, while I don't agree with the magnitude your Pauling analogy, I certainly agree with you statement that Penrose is out of his depth here. While I do not , presently, buy his argument that quantum effects are necessary to realize consciousness, I remain open to the idea until we know more about BOTH physics and computation.
[1] http://www.mitpressjournals.org/doi/abs/10.1162/NECO_a_00263 [2] http://www.kurzweilai.net/super-turing-machine-learns-and-ev...
No, that's just a current limitation of the kind of machines we know how to make.
Neurobiology. Chemistry. Physics. Of course, any machine will also be subject to the same formalism, so....
To start with, no physical entity has an unbounded memory available.
Computers are strictly devoid of semantics. They are, at best, purely syntactic machines. There is no amount of syntax than can ever lead to semantics. Strictly speaking, computers don't even compute.
Further reading: http://edwardfeser.blogspot.com/2013/10/do-machines-compute-...
Can be, but do not have to be. A distinction through which a host of mistakes have fallen, including Donald Hoffman's 'disproof' of functionalism.
In short, I agree you entirely with except that I don't see how we can design a machine with hands as untied as ours. Given a mechanistic view of the brain one concludes that it must be possible, but I still don't understand what formal trick would enable this difference in power. It does seem to give up consistency somehow (humans make mistakes) but not entirely (humans can detect mistakes they've made). I'd love to hear more thoughts on this matter.
The claim is that you can not adequately model the Human mind with a Turing machine. Pointing out that Human's are not logical only strengthens this claim and does not refute it. Further, the claim at this non-Turing machine-like ability of Humans is valuable and allows us insight that Turing machine is fundamentally incapable of.
I'm unclear on how that is valuable.
False. "2+2=5" is not a true statement, but I can construct a computer program that purports to compute the sum of 2 and 2 yet produces 5 as an answer. You can object "you got the addition wrong!" but that's irrelevant. If there is a person who believes (falsely) that 2 plus 2 equals 5, then the computer program could be an accurate model of their thought processes. There's nothing about "computers consistently produce the same answer" that implies "computers always produce the correct answer". "Computers always produce the correct answer" is a much more powerful claim, so the onus is on those who make the claim to prove it.
Yes, you can make a computer spit out nonsense, but this is a far cry from the computer actually behaving in an illogical way. A Turing machine's entire operation is logical. It is built with logic. It is a logical machine.
Penrose claims the human mind is not a Turing machine. That it is not logical. That it is not built with logic. That it is not a logical machine.
Pointing out that the human mind is not logical is thus restating Penrose' point and not refuting it.
In particular, he's saying that the human mind is capable of outputting true statements that cannot be proven to be true via any bounded number of proof steps. I don't disagree with that.
However, given that the human mind is capable of outputting false statements -- witness this conversation, wherein at least one of us is outputting false statements -- Penrose has failed to prove that the human mind is non-Gödelian, i.e. can output all the true statements that exist, while outputting no false ones.
It's easy to write a computer program that outputs every statement provable from the Peano axioms: start with the most primitive possible statements, then progressively output more complex ones. This is because the Peano axiom system is "recursively enumerable", to use Computer Science terminology. The program never terminates, but any provable statement you name will be output at some finite time.
Additionally, it's easy to write a computer program that outputs every possible combination of symbols. Most of the combinations aren't statements at all. Of the ones that are statements, most of them are not consistent with the Peano axioms. Of the ones that are consistent with the Peano axioms, most of them are unprovable. But if the set of truths is countable, this scheme guarantees that the program will output every true statement that exists.
It is possible to write a computer program that checks if a statement is provable from the Peano axioms. The naïve way is to run the program that outputs every statement as a subroutine, then halt if the subroutine prints out the statement which we wish to verify. This program will halt iff the statement is provable.
It is NOT possible to write a computer program that checks if a statement is unprovable from the Peano axioms. Such a program may be able to detect a subset of unprovable statements. The subset it can detect may even be countably infinite. But there are some statements which will cause the program to run forever.
Let statement X represent a formulation of "the Goldbach conjecture is true" written in the language of Peano arithmetic. Does the program run forever if asked whether X is provable? If you are a non-Gödelian being, you will be able to answer that question with no error, because Gödel's Incompleteness Theorem only applies to formal systems which can prove all true statements.
I think the key move you're making here is dictated by this. There isn't a largest model that will be able to capture all such truths (hence Gödel's incompleteness theorem) and what you would need is a system that is able to constantly build further systems else handle the ambiguity and do model-switching.
Not that it's impossible to do, but have we proven that such a system is theoretically possible yet within conventional frameworks?
> It's a rhetorical trick at best
This is precisely it. It works because 'true' is defined for the formal system but not for the human. It so happens that the humans in question say well 'of course we know what Godel's proposition means and what true is' but that's just because they are familiar with those words in an entirely unrelated sense to the system being described (akin to say the common speech use of 'energy').
Since you need a system at least as powerful to evaluate the propositions of an underlying system and in that more powerful system a similar proposition can be formed.
Anologous to this is Wittgenstein's problem on the irreducibility of rules: that one can always ask for the explanation of a rule, explanation of an explanation an so on ad infinitum. So it is essentially impossible to determine whether a rule is being applied correctly or not. It so happens that we (humans) can stop asking and 'apply'; but I'm not convinced this is something machines cannot do.
To say that you assume a human mind is a machine. Why isn't that begging the question?
Of course you are correct in suspecting I do assume the human mind is a machine, because we have seen zero evidence that it's not the case, but that's not in any way part of the sentence you quoted.
I feel that only philosophers & logicians are this black-and-white. There are many people content to occupy a midpoint on the spectrum of "This fact is true"..."This fact is false"; some even will accept a range!
One fact I think we can all agree on is that philosophers are: old; white; dead; men; and wrong.
1. Let's suppose for sake of argument that humans really can see the inherent truth of "Peano Arithmetic is consistent". That doesn't mean humans violate Gödel's Incompleteness Theorem: it could just mean that humans use axioms stronger than PA.
2. Gödel's Incompleteness Theorem only applies to systems that are perfectly logically consistent. Not sure how Penrose didn't notice, but humans... aren't.
3. When scientists proposed Quantum Mechanics as a replacement for Classical Mechanics, it was on them to explain how Quantum Mechanics simplified to Classical Mechanics in the common case. "Penrose Mechanics" is an even more radical departure — especially from a physics of computation standpoint, as Penrose Mechanics by definition would allow solving at least some of the problems in (ALL - R) in ~polynomial time. Penrose needs to explain how Penrose Mechanics reduces to Quantum Mechanics in the common case.
4. Penrose proposes that (a) there exist new physics, (b) that evolution has learned to computationally exploit the new physics via microtubules, and yet (c) that humans are the only lineage to make use of this feature of microtubules, even though microtubules are found in all eukaryotic cells (from mushrooms to amoebae). From a predator-prey standpoint alone, it would seemingly be a huge evolutionary advantage to be able to compute NP or R functions in polynomial time. (That ability is not _strictly_ implied by Penrose Mechanics, but it's a very likely consequence.) Penrose needs to explain why only humans are taking advantage of the computational power of microtubules, when microtubules have existed for billions of years and across millions of species. (TL;DR: It's the pineal gland all over again.)
#2 Please show how to program a Turing machine so that it is not logically consistent at a fundamental level?
#3 Are you questioning the viability of objective collapse theories in general? To date we have no experimental evidence of said theories, but I don't think anyone has suggested they are inconsistent with QM. Are you? If so, please show how...
#4 Show where Penrose has said humans are only species to make use of microtubules and your claims regarding Penrose' theory of objective collapse and having staggering new computational speedups are ... without any evidence. Please provide some.
The tenor of your comment suggests you find Penrose to be an idiot and missing obvious problems. Consider that perhaps you are the one misunderstanding what he is proposing and that he is not, in fact, an idiot.
2. The Turing machine itself is logically consistent. The semantic interpretation of its data need not be logically consistent: it's possible to write a computer program that prints "2+2=5" to the screen. Likewise, a human brain may be made of physics and physics may be logically consistent, but the semantics of the data in the brain (the "beliefs") need not be consistent.
3. No, no need to reject collapse itself (though I'm more of an Everett guy). Penrose postulates the existence of a new level of physics that is not Quantum Mechanics. People who aren't careful call it an extension of QM, but QM's computing model is limited to solving a small subset of NP problems (BQP) in polynomial time, and BQP is theorized to be a proper superset of P. However, Penrose's proposed physics would allow solving problems outside of NP in polynomial time. Hence, Penrose's physics IS NOT quantum physics.
4. It is physically impossible for computers (as we understand them today) to solve problems outside of the set we call "Turing complete". Penrose claims that there's at least one problem outside that set that the human brain can nonetheless solve: checking the consistency of formal systems. If Penrose's assumption were true, computers (as we understand them today) would consequently be strictly less powerful than the human brain. The crux of Penrose's position is accepting this consequence as true, then asking how that's possible. I reject that hypercomputation of problems beyond Turing-completeness is what's happening, i.e. that humans simply hold unjustified beliefs about the answers to such problems.
I do not doubt that Penrose is an intelligent and educated person. However, his expertise is General Relativity. It's bad enough when he tries dipping his toes into Quantum Physics, but when he wades into Physics of Computation, he is no longer acting as an expert, but as an interested layman. It's only because of the authority he holds as a GR expert that his argument is treated seriously, but that authority is not actually relevant to his argument.
#2 See fundamental level. Penrose claims that what sets us apart from Turing machines is fundamental. But whatever, you've refuted whatever point you wanted to make by saying human's are not logically consistent. Your snark about Penrose not understanding is as empty as whatever point you were trying to make.
#3 Cite a paper showing that objective collapse gives these supposed algorithmic speedups or go home. NOTE: objective collapse theories are an active area of research and are not limited to Penrose by any means. Your claim that they are inconsistent with QM begs for evidence. Cite some or stop spreading nonsense.
#4 Your answer here is completely void of any context to the question: where Penrose says that microtubules are limited to humans. I take it you concede that he did not say any such thing?
You treat Penrose as an idiot missing obvious problems. I think it far more likely that you've misunderstood. Pity your lack of humility might make it impossible for you to understand what he actually argues rather than your strawmen.
Just because Turing machines are universal making it possible to write a program that asks a question about another type of Turning machine doesn't mean that question is decidable.
[1] http://www.scottaaronson.com/blog/?p=2725
Please show how that paper describes a Turing machine 'programmed with PE' can't simulate a Turing machine 'programmed with ZFC'?
Perhaps we are arguing over what 'programmed with PE' or 'programmed with ZFC' means? The parent post seemed to claim that it was possible to construct a physical computer with the axioms of ZFC built-in so to speak. As opposed to one with PE built-in. Which obviously calls into question the parent post understanding of what exactly idealized Turing machines are.
There is just tons of reading on this if you care to scour the literature. Penrose' objective collapse theory is usually given as the prime example of the whole genre.
People can be known for more than one thing. Those two things can be unrelated, even if they're in the same field.
Turing machines suck. Building a Turing machine that implements ZFC proof-generation is a project appropriate to a graduate-level paper, not something to toss off in an Internet pissing contest.
http://www.scottaaronson.com/blog/?p=2725
"... a one-tape, two-symbol Turing machine with 7,918 states, whose behavior (when run on a blank tape) can never be proven from the usual axioms of set theory, under reasonable consistency hypotheses."
If the only such statements are artificial Gödelian sentences, I'm afraid I'm...uninterested, in the same way Lucas is uninterested in inconsistent humans.
2. Lucas and Penrose assert that some entities are significantly more powerful than others. To my knowledge, Lucas describes no mechanism for this. (To his credit, Penrose does, but, to my knowledge, the Bell inequality among other things means quantum mechanics is fundamentally random. That's not helpful.)
What could be the nature of such a mechanism? If it's material, it should be possible to include it in future machines. If it's not, we are in the realm of theology again.
3. So humans are intelligent and intelligent-seeming machines only simulate intelligence. If the difference is not common, obvious, and readily meaningful, then where is the distinction? ("So what? I can't write an opera either.")
There are good refutations of Penrose' argument, but I've seen none of them here. Just a bunch of bandwagon hackers assuming Penrose doesn't know what he is talking about and then putting up lazy straw man takedowns.
Edit: Or perhaps that Penrose is not claiming that QM is some kind of super secret sauce that makes multicellular life intrinsically stronger than computing machines?
That.
All Penrose is saying is that Turing machines can not be the basis for modeling the human mind. He is not saying that the human mind is not governed by physics. He is not saying that the universe is not governed by physics. And he is not resorting to 'quantum woo' like some here are saying.
Penrose' argument is that there is something non-algorithmic going on in the human brain and that this non-algorithmic process is governed by physical laws. He then goes on to discuss objective collapse theories for quantum mechanics and how that may be the source of non-algorithmic behavior in the human brain.
To be clear, objective collapse theories have as of yet no experimental evidence. But, crucially, they are not inconsistent with QM as some above have said. Also, the precise biological mechanism that Penrose postulates for how the brain operates with objective collapse is highly controversial ie, microtubules.
There are models that are more powerful than TMs, but the ones I know about have "unreasonable" powers. The one that comes to mind came from A.K. Dewdney (I think); it does an unbounded amount of work in a single time step by having an unbounded number of computational units. And that is just not kosher.
So there is a certain amount of circumstantial evidence for the definition of "computable" being something like a Turing machine. And this definition of computable has limits, as in Godel (never thought I'd miss my phone's keyboard) and Turing.
As you describe, Lucas, Penrose, and others such as Searle want to walk a very fine line. They want to claim that the human mind is not limited in the way that computable functions are limited, which I understand; it certainly doesn't feel like I am so limited. On the other hand, they want to avoid going off the edge into full-blown mysticism: dualism and ye immortal soul.
Unfortunately for those like me (and I know I am) who take materialism somewhat seriously, their arguments feel more like the post-Darwin arguments that, yes, animals and stuff do evolve like that, but people are different. The closed set of computational models of TM-equivalence do seem to capture "computable functions", and those computable functions do seem to cover what is possible from any system built on biology, chemistry, and ultimately physics. Their arguments seem to be either making a distinction that doesn't make a difference, or sneaking in supernatural powers. (You can see the truth or falsehood of a statement without a proof? Reliably?)
So arguments for a non-algorithmic middle ground between TM-equivalence and "quantum woo" (great phrase, btw) are giving me problems. If I am a mixed bag of incomplete and inconsistent, with limited rationality that is not well described as a formal system but is weaker than a Turing machine, as I suspect I am, then the appearance of a non-algorithmic process is simply a misunderstanding.
On the other hand, if the non-algorithmic goings-on are correct, "quantum woo" feels like the most reasonable description of their powers. I'd really like some strong, unarguable evidence before I accept that step.
Penrose's contention can be summed up without it. He basically thinks that gravity will be discovered to play a role in quantum wave function collapse. And that this role will render non-quantum Turing machines incapable of accurately modeling quantum gravity in some very serious way. Another way of saying it is that he believes when a real theory of quantum gravity is developed we'll find that non-quantum Turing machines will not be able to accurately model the physics of a black hole.
Note that this says nothing whatsoever about the human mind or microtubes or formal systems, etc. This is about his subject of expertise: gravity, mathematical physics, and quantum gravity. He's one of the world's leading experts in these fields.
And if he is right, he conjectures it will have strong implications for non-quantum Turing machines as a basis for Strong AI.
Among your other comments there are some that are quite substantive (yay) and others like this one (boo). Kindly stick with the substance even if other people aren't. That enhances credibility and helps tend the garden for everyone.
No. You are misunderstanding Godel's theorem. For any given formal system there exists a Godel sentence which the system cannot prove. But the Godel sentence for every system is different. Moreover, there is no algorithm for constructing a Godel sentence. If there were, then a TM could follow that algorithm to construct its own Godel sentence.
So every formal system has a (different) Godel sentence, but there does not exist a universal Godel sentence that is unprovable by all formal systems.
Now, lets make ZFC' which is ZFC + 'ZFC is consistent'. ZFC' can prove A(as it contains ZFC) and uses the consistency of ZFC to prove A':(ZFC cant prove Godel(ZFC)) .
Similarly, a program decides a set of problems(like does this Turing machine halt or does this diophantine equation have solutions) with access to its source code will be able to algorithmically generate the problem instance on which it will not halt. You can modify the program to create a new program which will use this information, but it will be a new program.
No, it isn't. But it's a non-trivial point.
> Similarly, a program decides a set of problems(like does this Turing machine halt or does this diophantine equation have solutions) with access to its source code will be able to algorithmically generate the problem instance on which it will not halt. You can modify the program to create a new program which will use this information, but it will be a new program.
So this all depends on what you consider a "new program". Suppose we write a program that actually does this: analyzes its own code, algorithmically produces the Godel sentence for that code, then modifies its code to include that as an axiom, forever. That program is a program, and that program will also have a Godel sentence, but one which it cannot construct using that algorithm. You need a different algorithm to construct Godel sentences for programs that constructs Godel sentences on programs that don't construct Godel sentences.
The result is a hierarchy of Godel-sentence constructors. At any level in the hierarchy the procedure is algorithmic, but going from one level to the next is not. It's similar to the process of constructing the hierarchy of ordinals, which is also provably non-algorithmic.
But for this argument to be applicable to the Theory of Mind, it seems to me that it is exactly what Lucas and Penrose are claiming to exist.
If the only un-provable sentences are Godelian constructions, the whole issue isn't philosophically interesting. But that's not true, to my understanding.
So, there are statements that cannot be proven in, say, ZFC. Some of these statements are, I assume, "interesting"; they aren't simply constructs that are designed to be unprovable. Finally, for Lucas and Penrose's arguments to be meaningful, some of those statements should be obviously true or false---that is the only way for the human mind to not be limited in the way ZFC is limited.
If the truth of the statement is not obvious, then I'm not clear on how the human mind is stronger than the formal system in question. If the truth is obvious, but the statements are all variations on "I am lying" designed specifically for a given formal system, then I don't see how this is an actual interesting property of minds.
I don't understand what's the problem here. TMs are allowed to be inconsistent. There's a TM that outputs every single finite string. What would even count as its Godel sentence?
In fact, there is an algorithm that constructs the Godel sentence given a consistent formal system capable of arithmetic. That algorithm is spelled out in Godel's original proof. Now, of course, a TM can only do that to its own formal system if it's inconsistent itself, which means the "Godel sentence" it output is meaningless.
Sure, but that's an uninteresting case. Any inconsistent system that includes basic propositional logic asserts all statements to be true. So in these discussions consistency is generally assumed.
So if I can put you in an isolation tank and model your brain, then I can in fact (at least in principle) construct your Godel sentence and know for certain that you will never come to the realization that this outrageously complicated mess I've just constructed (the one that says of itself that you will never come to see its truth) is true, and I can feel smug about the fact that I can see that this is true and you never will. But here's the kicker: I can never gloat about my superiority to you, because as soon as I do you are no longer in isolation. You are communicating with me, and the sentence I've take great pains to construct is no longer your Godel sentence. It is the Godel sentence of you in isolation, but as soon as I tell you your Godel sentence -- indeed as soon as I (or anyone else) say anything to you, my construction falls apart and I have to start over.
Ironically, far from proving that we are not machines, Lucas's argument actually goes a long way towards demonstrating that we are, because coming up with examples of things that we interacting brains know to be true that a particular isolated brain will (with extremely high probability) never come to realize is not difficult. The four-color theorem, for example, is probably beyond any single isolated human brain. Indeed, the very argument you are reading now is probably beyond any single human brain because the brain that's writing it would never have come up with it had Godel and Lucas (and Turing) not paved the way.
Have you formulated it by yourself?
I composed the words in the above comment, but I don't actually know if the substance of that argument is original with me. It is heavily based on things I've learned in other places, but I don't have any specific recollection of having seen the argument being framed in exactly that way before. It's possible I invented it, but I doubt it. It's not a particularly deep insight, a minor variation on a deeper theme, that "meta-Godelization" is not algorithmic, a fact that I first learned by reading Hofstadter's "Godel, Escher, Bach." I don't actually know what the original source of that result is (I should probably go find out).
But I'll happily take credit for the pedagogy if it somehow turns out I got there first.
Put in another way: Couldn't I in principle define such a circumscribed system with all axioms and inference rules by including all of those sensors and all of their possible states into the system?
Let's recall the original argument:
"a mechanist formulates a particular mechanistic thesis by claiming, for example, that the human mind is a Turing machine with a given formal specification S. Lucas then refutes this thesis by producing S’s Gödel sentence, which we can see is true, but the Turing machine cannot. Then, a mechanist puts forth a different thesis by claiming, for example, that the human mind is a Turing machine with formal specification S’. But then Lucas produces the Gödel sentence for S’, and so on, until, presumably, the mechanist simply gives up."
When you are presented with S there are two possibilities: either S includes you, or it doesn't. If S includes you, then you can't Godelize it. If S doesn't include you, then you might be able to Godelize it, but if you present your result to the thing that S is supposed to be a model of -- indeed as soon as you interact with that thing in any way -- then S is no longer an accurate model of that system because now it is interacting with something that, by assumption, was not part of the model.
Is that right?
> woo that he pulled out of his ass
It's not ok to argue like this on HN, so please don't. It lowers the quality of discussion in its own right and sets off a downward spiral from others.
Edit: Actually, we've banned your account. Your comment history shows that you've broken HN's civility rule so egregiously, and so repeatedly, that we should have banned it long time ago. If you don't want to be banned on HN, you're welcome to email hn@ycombinator.com and promise to follow the rules scrupulously in the future.
We detached this subthread from https://news.ycombinator.com/item?id=14446064 and marked it off-topic.
In light of this, we can reinterpret "However, a human can look and see that the Gödel sentence is true" as:
When you take the best pattern matching machine nature has produced, train it with distilled essentials of centuries of mathematical geniuses, and present it with the Gödel sentence, the pattern matcher strongly signals "This looks true!"
When viewed this way, it looks less like an inherently human-specific achievement, but more like exactly how ML algorithms work. The only difference is that ML cannot quite pull it off, so far.
For instance people talking about 'humans are not consistent', we are talking about effort directed into a well specified formal domain, and after long term reflection. So, of course, people make mistakes in mathematics all the time, but we are talking about consistency after intense review over long period of time. We write a program, and cant find the bug even if we look at it for a google years.
To the claim, 'Godel statement is not interesting relative to other statements', the goal is that to demonstrate non-algorithmic nature of some human activity, which itself would be a big deal if true.
A more qualified version of the LP claim can be expressed as follows 'One of the following is true 1) The activity of humans in some formal domain(like mathematics) is inconsistent in a deep way(not routine mistakes). 2)Humans are reflective consistent, algorithmic but will never have access to the source code(a claim about limits about brain research). 3) We are reflective consistent, can access source code, but wont be able to see why the source itself is consistent.'