My wording is wrong, because it sounds like I'm saying that Bentham is adopting the policy ad hoc. A better way to state this is that Bentham starts out as an agent that does not give into brinksmanship type games,…
Yes, the point of the GP comment is exactly this, if Bentham becomes an agent that goes for C, he also explicitly discourages the mugger from being an agent that would cut off their fingers for a couple of bucks. Notice…
Do we want to talk about a hypothetical world where deontology was the underlying moral principle? Where, for example, a large agency in charge of approving vaccines decided to delay approval of a life saving because,…
I've been studying the language for a while, and recently made the switch to Japanese-Japanese dictionaries, after using EDICT for a long time. This has highlighted some reservations I have about it. The most available…
Maybe being shocked means that the person talking about the subject is misrepresenting it, because they themselves don't understand the arguments and are inadvertently projecting. For example, Ray Kurzweil would…
I agree with a lesser form (timescale on which the AI self improves is probably going to be long enough for humans to deal with it) of your first argument, but I'm confused at how, given 'merely' lots of compute…
Have you ever read The Checklist Manifesto[0]? I may be reading too much into this post, but the lessons you learned from this interview process have frighteningly close parallels to the lessons in the books. I doubt…
Even antivaxxers have people more patient and more understanding of their opponents actually debunking their object level thinking. They say that vaccines cause autism? We say that the original study was wrong! Why do…
You seem to be implying that intentional slowing is MIRI's official stance, without any showing support. Given that the original response was in response to the specific accusation of "no AI experts say that this is a…
This seems as if it is conflating the worry that nuclear bombs would ignite the atmosphere with supersonic travel. Who predicted this?
I'm afraid of making a middlebrow dismissal but I'm going to post it anyway, in hopes that someone just skimming would not be mislead. The question is what Michael Jordan thinks of the "concept of the singularity", and…
> Honestly asking, why would they? I dont see the obvious answer So, your intuition is right in a sense and wrong in a sense. You are right in that AI systems probably won't really have the "emotion of wanting", why…
By 'upgrade everything from human secure' I meant that some targets aren't necessarily appealing to human targets but would be for AI targets. For example, for the vast majority of people, it's not worthwhile to hack…
The "more to it" is "if the AI is much faster at thinking than humans, then even humans in the observe/decide/act are not secure". AI systems having bugs also imply that protections placed on AI systems would also have…
A typical example, which I don't really like, is that once it gains some insight into biology that we don't have (a much faster way of figuring out how protein folding works). It can mail a letter to some lab,…
Er, sorry for giving the impression that it'd be a supervillain. My intention was to indicate that it'd be a weird intelligence, and that by default weird intelligences don't do what humans want. There are some other…
Smart is just a shorthand for a complicated series of lower level actions consisting of domain knowledge, raw computational speed and other things yes. I don't think we're really disagreeing about this. However, I do…
Note: The not ELI5 version is Nick Bostrum's Superintelligence, a lot of what follows derives from my idiosyncratic understanding of Tim Urban's (waitbutwhy) summary of the situation [0]. I think his explanation is much…
There are some skills in physics which are sort of foreign to the math method of problem solving. (Speaking as a physics major) For example, picking a nice reference frame in simple mechanics problems is something that…
One approach that I learned was to use your fingers to underline the current line that you're reading, going at a "normal" speed when you're tired and then trying to "drag" your eyesight along when you want to push…
I'm currently having a failure of imagination here, but how would you social engineer a password manager? The tricks I'm thinking of involve fooling the user into thinking a site is something it's not or guessing some…
For those interested, guild wars is the game, and the process is covered below in the 'Your Computer is broken' section: http://www.codeofhonor.com/blog/whose-bug-is-this-anyway HN Discussion:…
I was also concerned about this issue, and have instead taken a nice medium path: After changing a password, I would try to log into the service at approximately the same intervals the Anki algorithm dictates until I…
My wording is wrong, because it sounds like I'm saying that Bentham is adopting the policy ad hoc. A better way to state this is that Bentham starts out as an agent that does not give into brinksmanship type games,…
Yes, the point of the GP comment is exactly this, if Bentham becomes an agent that goes for C, he also explicitly discourages the mugger from being an agent that would cut off their fingers for a couple of bucks. Notice…
Do we want to talk about a hypothetical world where deontology was the underlying moral principle? Where, for example, a large agency in charge of approving vaccines decided to delay approval of a life saving because,…
I've been studying the language for a while, and recently made the switch to Japanese-Japanese dictionaries, after using EDICT for a long time. This has highlighted some reservations I have about it. The most available…
Maybe being shocked means that the person talking about the subject is misrepresenting it, because they themselves don't understand the arguments and are inadvertently projecting. For example, Ray Kurzweil would…
I agree with a lesser form (timescale on which the AI self improves is probably going to be long enough for humans to deal with it) of your first argument, but I'm confused at how, given 'merely' lots of compute…
Have you ever read The Checklist Manifesto[0]? I may be reading too much into this post, but the lessons you learned from this interview process have frighteningly close parallels to the lessons in the books. I doubt…
Even antivaxxers have people more patient and more understanding of their opponents actually debunking their object level thinking. They say that vaccines cause autism? We say that the original study was wrong! Why do…
You seem to be implying that intentional slowing is MIRI's official stance, without any showing support. Given that the original response was in response to the specific accusation of "no AI experts say that this is a…
This seems as if it is conflating the worry that nuclear bombs would ignite the atmosphere with supersonic travel. Who predicted this?
I'm afraid of making a middlebrow dismissal but I'm going to post it anyway, in hopes that someone just skimming would not be mislead. The question is what Michael Jordan thinks of the "concept of the singularity", and…
> Honestly asking, why would they? I dont see the obvious answer So, your intuition is right in a sense and wrong in a sense. You are right in that AI systems probably won't really have the "emotion of wanting", why…
By 'upgrade everything from human secure' I meant that some targets aren't necessarily appealing to human targets but would be for AI targets. For example, for the vast majority of people, it's not worthwhile to hack…
The "more to it" is "if the AI is much faster at thinking than humans, then even humans in the observe/decide/act are not secure". AI systems having bugs also imply that protections placed on AI systems would also have…
A typical example, which I don't really like, is that once it gains some insight into biology that we don't have (a much faster way of figuring out how protein folding works). It can mail a letter to some lab,…
Er, sorry for giving the impression that it'd be a supervillain. My intention was to indicate that it'd be a weird intelligence, and that by default weird intelligences don't do what humans want. There are some other…
Smart is just a shorthand for a complicated series of lower level actions consisting of domain knowledge, raw computational speed and other things yes. I don't think we're really disagreeing about this. However, I do…
Note: The not ELI5 version is Nick Bostrum's Superintelligence, a lot of what follows derives from my idiosyncratic understanding of Tim Urban's (waitbutwhy) summary of the situation [0]. I think his explanation is much…
There are some skills in physics which are sort of foreign to the math method of problem solving. (Speaking as a physics major) For example, picking a nice reference frame in simple mechanics problems is something that…
One approach that I learned was to use your fingers to underline the current line that you're reading, going at a "normal" speed when you're tired and then trying to "drag" your eyesight along when you want to push…
I'm currently having a failure of imagination here, but how would you social engineer a password manager? The tricks I'm thinking of involve fooling the user into thinking a site is something it's not or guessing some…
For those interested, guild wars is the game, and the process is covered below in the 'Your Computer is broken' section: http://www.codeofhonor.com/blog/whose-bug-is-this-anyway HN Discussion:…
I was also concerned about this issue, and have instead taken a nice medium path: After changing a password, I would try to log into the service at approximately the same intervals the Anki algorithm dictates until I…