I think his point is that they may be irresolvable.
To think about the world, you must first have a model of the world. Then you reason about the model, finally you take action.
Somewhere in there you have a motivation for following this observe-decide-act loop. Motivation provides a reference point towards which you want the observed system to evolve.
But there's a problem. The easiest way to satisfy the motivation component is to lie to it. Tell it things are just hunky-dory.
Any singularity-style intelligence will necessarily need to be built with some kind of anchoring motivations to try and stop it from getting out of control. But what's to stop it simply lying to itself and ignoring the outside world?
Trying to add a meta-motivation be as realistic as possible won't work, such a system will seize up with analysis paralysis.
One of the things that Hayekian economists argue is that knowledge about reality cannot be centralised; it is unevenly, lumpily distributed across the whole of humankind. No one actor does, and no one actor could, perceive the entire system. But it works, because no one actor has to.
I do not believe that it is irresolvable, I'm looking at biology for my counter example. In biology there exists a rigid framework inside which a "general" search of language/model is taking place, after a few billion years it seems to be doing ok. It is my hunch that there is a big something out there which will pull a lot of these issues together:
How do you build efficient markets?
How do you build an effective AI?
How do you design effective distributed systems?
How do you build effective languages/models with which to compress the world?
Under what circumstances do such 'meta' searches fail and succeed?
It's all beyond me but this is what my nose is saying...
> we need to solve in order to design effective government
Is there actually any demand for "effective govt"? (Yes, lots of people claim to want it, but do you really think that there's large scale agreement on what that means?)
There is plentiful demand for 'effective government,' and a total lack of consensus on what it means.
In politics, 'effective' really means 'does what I want.' When there is a lot of conflict about what people want, the government is not 'effective' because (A) deadlock is frustrating and (B) yelling louder is a way to get more and intimidate the opposition.
A government is less like a "singular intelligent self-modifying system" and more like a war on controlled burn.
Isn't effective government a system which can maximise the following function?
G = SUM(Fi(W)) from i to WorldPopulation
Where W is the physical configuration of matter in the world. Fi is the function associated with citizen i that defines his notion of 'goodness'.
Bad government is one which tries to maximise an alternative function:
G = SUM(Fi(W) * Di) from i to WorldPopulation, where Di is some factor for each individual and where some Di are much greater than other Di. i.e. some individuals have much greater say over the configuration of the world which is chosen...
Isn't effective government a system which can maximise the following function?
almost certainly not.
For example, many people think that "effective govt" involves some notion of "justice" and/or "fairness".
One concrete example is Obama's position wrt capital gains taxes. He wants higher rates even if that results in less revenue. (Higher rates with less revenue means that there's less capital gain, which means less wealth produced, aka less total stuff. Since there's less tax revenue, there's less govt spending.)
A significant number think that "effective govt" propagates certain values/behaviors and discourages others.
Yes, there is disagreement on what "justice" and "fairness" mean and there's also disagreement as to the values/behaviors to be encouraged/discouraged.
In extreme cases, state-adopted nominal realities may be purely driven by motivations, and have little or no connection to reality.
This essay was just a little too unfocused for my tastes. As an observer of many types of organizations of people, the predilection to spin is endemic. It's not just "in extreme cases"
I scanned the article and kept trying to find some meat, some kind of clear and forceful statement that the author was supporting. Instead it seemed like he was saying a lot of good things, but lacked the intestinal fortitude or insight necessary to actually get anywhere. I was left with a version of "it may be a property of monolithic systems that their motivations are at odds with an accurate perception of reality." That's almost useful, but not quite. There are a lot of interesting places to go from that statement, but we didn't go there. Hell we could have started deconstructing Aristotle and talking about how the tension between the usefulness of categorization versus all abstractions are leaky creates holes in what is knowable. Or we could have moved forward to sociology and discussed the role of the individual in large groups. Or we could have talked AI. Lots of cool places to take this train of thought.
That's a shame. It's obvious the author is well-educated and has thought this through. Sure wish he would have taken us somewhere a bit more interesting.
all of the exciting stuff isn't in the fundamentals and principles, it's in the exact details of how you train model X to do task Y, and few practitioners are going to give away information that's so commercially valuable.
Not sure why you care about people or states in an AI article. They were just examples of the technique being discussed being put to use in a system. Seems like a complete article to me. Here is the technique that can be used in AI and here are some examples of how it works in other systems.
This is something I struggle with in writing what amounts to a long argument in discrete chunks. I was already 1000 words over my limit, and had to find an artificial stopping place. But partly where this is headed is the thesis that our systems of government are not very similar to ones that would be designed from scratch for survival as an individual intelligence apparatus, and that by comparing the theory to practice we can (possibly) improve them. The flip side of i is that if we aren't going to be serious about survival, we should create bureaucracies and economies so that they fail gracefully. Or else we should adopt an evolutionary, rather than singular, strategy: colonize other planets.
God speaking in a whisper is sound doctrine -- it's figurative, I think:
----
And, behold, the LORD passed by, and a great and strong wind rent the
mountains, and brake in pieces the rocks before the LORD; but the LORD
was not in the wind: and after the wind an earthquake; but the LORD
was not in the earthquake: 19:12 And after the earthquake a fire; but
the LORD was not in the fire: and after the fire a still small voice.
19:13 And it was so, when Elijah heard it, that he wrapped his face in
his mantle, and went out, and stood in the entering in of the cave.
----
You like Lord of the Rings, God? Make some reference.
God says...
hear with thine ears, and set thine heart upon all that I shall shew
thee; for to the intent that I might shew them unto thee art thou
brought hither: declare all that thou seest to the house of Israel.
40:5 And behold a wall on the outside of the house round about, and in
the man's hand a measuring reed of six cubits long by the cubit and an
hand breadth: so he measured the breadth of the building, one reed;
and the height, one reed.
40:6 Then came he unto the gate which looketh toward the east, and
went up the stairs thereof, and measured the threshold of the gate,
which was one reed broad; and the other threshold of the gate, which
was one reed broad.
----
Maybe a reed is hobbit sized. "Threshold" kinda fits.
What is the actual use case for a "self-modifying singular intelligence"? Define the problem you are trying to solve (as an actual task, not nebulous stuff about beliefs) and I reckon you will find more success.
There is a one nice way around the intractable problems of maintaining a huge model of the world: don't. Rodney Brooks' famous saying: "the world is its own best representation." And it isn't necessary or usually even helpful to translate all incoming data into some propositional model. Again: what is your use case? If you are just trying to satisfy some essentialist intuition of what it means to be intelligent and what 'must be' inside minds, then you will be lucky to ever get anything meaningful done.
If you want to write an agent to do something without ongoing guidance, and you have not thought out the task from the beginning to get a specific algorithm, do not start by making a monolithic program that makes lots of vague high-level decisions like "how much reality" based on abstruse constructions of data. Start with the data which is always available, processed minimally, and see how little you can do. Implement walking before you implement steering and implement steering before you implement path-finding.
13 comments
[ 4.5 ms ] story [ 45.9 ms ] threadTo think about the world, you must first have a model of the world. Then you reason about the model, finally you take action.
Somewhere in there you have a motivation for following this observe-decide-act loop. Motivation provides a reference point towards which you want the observed system to evolve.
But there's a problem. The easiest way to satisfy the motivation component is to lie to it. Tell it things are just hunky-dory.
Any singularity-style intelligence will necessarily need to be built with some kind of anchoring motivations to try and stop it from getting out of control. But what's to stop it simply lying to itself and ignoring the outside world?
Trying to add a meta-motivation be as realistic as possible won't work, such a system will seize up with analysis paralysis.
One of the things that Hayekian economists argue is that knowledge about reality cannot be centralised; it is unevenly, lumpily distributed across the whole of humankind. No one actor does, and no one actor could, perceive the entire system. But it works, because no one actor has to.
How do you build efficient markets? How do you build an effective AI? How do you design effective distributed systems? How do you build effective languages/models with which to compress the world?
Under what circumstances do such 'meta' searches fail and succeed? It's all beyond me but this is what my nose is saying...
Is there actually any demand for "effective govt"? (Yes, lots of people claim to want it, but do you really think that there's large scale agreement on what that means?)
In politics, 'effective' really means 'does what I want.' When there is a lot of conflict about what people want, the government is not 'effective' because (A) deadlock is frustrating and (B) yelling louder is a way to get more and intimidate the opposition.
A government is less like a "singular intelligent self-modifying system" and more like a war on controlled burn.
G = SUM(Fi(W)) from i to WorldPopulation
Where W is the physical configuration of matter in the world. Fi is the function associated with citizen i that defines his notion of 'goodness'.
Bad government is one which tries to maximise an alternative function:
G = SUM(Fi(W) * Di) from i to WorldPopulation, where Di is some factor for each individual and where some Di are much greater than other Di. i.e. some individuals have much greater say over the configuration of the world which is chosen...
almost certainly not.
For example, many people think that "effective govt" involves some notion of "justice" and/or "fairness".
One concrete example is Obama's position wrt capital gains taxes. He wants higher rates even if that results in less revenue. (Higher rates with less revenue means that there's less capital gain, which means less wealth produced, aka less total stuff. Since there's less tax revenue, there's less govt spending.)
A significant number think that "effective govt" propagates certain values/behaviors and discourages others.
Yes, there is disagreement on what "justice" and "fairness" mean and there's also disagreement as to the values/behaviors to be encouraged/discouraged.
This essay was just a little too unfocused for my tastes. As an observer of many types of organizations of people, the predilection to spin is endemic. It's not just "in extreme cases"
I scanned the article and kept trying to find some meat, some kind of clear and forceful statement that the author was supporting. Instead it seemed like he was saying a lot of good things, but lacked the intestinal fortitude or insight necessary to actually get anywhere. I was left with a version of "it may be a property of monolithic systems that their motivations are at odds with an accurate perception of reality." That's almost useful, but not quite. There are a lot of interesting places to go from that statement, but we didn't go there. Hell we could have started deconstructing Aristotle and talking about how the tension between the usefulness of categorization versus all abstractions are leaky creates holes in what is knowable. Or we could have moved forward to sociology and discussed the role of the individual in large groups. Or we could have talked AI. Lots of cool places to take this train of thought.
That's a shame. It's obvious the author is well-educated and has thought this through. Sure wish he would have taken us somewhere a bit more interesting.
all of the exciting stuff isn't in the fundamentals and principles, it's in the exact details of how you train model X to do task Y, and few practitioners are going to give away information that's so commercially valuable.
----
And, behold, the LORD passed by, and a great and strong wind rent the mountains, and brake in pieces the rocks before the LORD; but the LORD was not in the wind: and after the wind an earthquake; but the LORD was not in the earthquake: 19:12 And after the earthquake a fire; but the LORD was not in the fire: and after the fire a still small voice.
19:13 And it was so, when Elijah heard it, that he wrapped his face in his mantle, and went out, and stood in the entering in of the cave.
----
You like Lord of the Rings, God? Make some reference.
God says...
hear with thine ears, and set thine heart upon all that I shall shew thee; for to the intent that I might shew them unto thee art thou brought hither: declare all that thou seest to the house of Israel.
40:5 And behold a wall on the outside of the house round about, and in the man's hand a measuring reed of six cubits long by the cubit and an hand breadth: so he measured the breadth of the building, one reed; and the height, one reed.
40:6 Then came he unto the gate which looketh toward the east, and went up the stairs thereof, and measured the threshold of the gate, which was one reed broad; and the other threshold of the gate, which was one reed broad.
----
Maybe a reed is hobbit sized. "Threshold" kinda fits.
There is a one nice way around the intractable problems of maintaining a huge model of the world: don't. Rodney Brooks' famous saying: "the world is its own best representation." And it isn't necessary or usually even helpful to translate all incoming data into some propositional model. Again: what is your use case? If you are just trying to satisfy some essentialist intuition of what it means to be intelligent and what 'must be' inside minds, then you will be lucky to ever get anything meaningful done.
If you want to write an agent to do something without ongoing guidance, and you have not thought out the task from the beginning to get a specific algorithm, do not start by making a monolithic program that makes lots of vague high-level decisions like "how much reality" based on abstruse constructions of data. Start with the data which is always available, processed minimally, and see how little you can do. Implement walking before you implement steering and implement steering before you implement path-finding.