Well, it's less about your 'free will' and more about one's inability to generate a random number.
I'm not going to take the time to test it, but if you flipped a coin before each key press, this algorithm would eventually get to 50% accuracy. But in the short term it would probably be <50%, because it assumes it won't actually get random input.
Actually, if a player is flipping a coin, it is literally impossible to design an algorithm that has expectation other than 50%. Even in the short term, it cannot have an expectation that's <50%.
Sklansky tells in the Theory of Poker that you need to be unpredictable when deciding whether to call a possible bluff, and "tune" it to the cards that are known.
So you may need to make a yes-no decision where you decide on yes about 65% of the time, when you consider lots of those decisions.
His solution? Glance at your watch. Divide the current minute into proportionate parts. See in which part the seconds hand is standing.
I counted up from 0 in binary (excluding leading zeroes) and even with a completely predictable series, it only did slightly better than 50%. I was expecting more (that on-line game of 20 questions was freaky good).
I have been having way too much fun with this, but it really goes off the rails sometimes. I am currently in a stream where the correct answer is "Ambrose Bunside". It asked me if my character has been dead for more than 100 years, then two questions later asked me if my character uses musical.ly. :/
I feel like it sometimes just wastes questions near the end when it's already narrowed the possibilities down. Maybe that's even on purpose, for showy effect.
After all, it's pretty impressive when it appears to correctly get Frank Sinatra out of "does your character play video games for a living."
The explanation of how it works says it uses 5-grams to predict the next bit, so if you use any sequence in which all 6-grams occur equally often, this won't be able to predict you at all.
Actually, an interesting challenge might be to construct a sequence that gives the Oracle as low a score as possible (is there a nicer simpler way to describe this than simulating the Oracle's code and always choosing what it doesn't expect?).
Sure, is there an easy way to generate that algorithmically without maintaining an explicit history? Is there a well-known sequence with that property? Do Gray codes have it, for example?
You'd want a sequence that contains every 6-gram exactly once and ends with the same 5 digits as it starts with: such a sequence would be 2^6=64+5=69 bits long, and then you'd repeat the first 64 over and over. (Or just contains exactly once up to cycles.)
The Gray code turns out to work if you require the n-grams to be aligned to n-character boundaries. So if you interpreted "00001111" as containing only the two 4-grams "0000" and "1111", you're OK; but if you allow arbitrary alignments, so that "00001111" contains the 5 4-grams "0000", "0001", "0011", "0111", and "1111", the Gray code doesn't work.
For example, the 4-bit Gray code (under this interpretation) when concatenated into a single bitstream begins "00000001", which then contains the 4-grams "0000", "0000", "0000", "0000", "0001", so the predictor gains an advantage right at the outset. (I didn't check whether it then loses its advantage over time... that's an interesting question.)
Your premise "open the source code for this, run it in node, get the probability of guessing f or d, and then guess the one that's lower" goes straight against the explicit request to try to type randomly, everything that follows althoug may be correct doesn't apply here.
You are onto something here. This reminds me of various hierarchies of logic(Bertrand Russell's arguments). To beat a person such as you, all it has to do is lie - and tell you at the end it has been lying all along.
Sure, there are ways to "break" it. The point here is to demonstrate that if you try to intuitively go for manual "non-randomness", you won't be as random as you think, sometimes surprisingly off from ~50%.
Actually the quickest way I found to "break" it and hover at ~50% is to simply aim for the dead space between the "F" and "D" keys. If it was using non-adjacent keys like "F" and "J" instead it would be better for its purpose. "Q" and "P" might work even better since it would mostly force you to use 2 hands, making it much harder to score close to 50%.
It isn't telling you the odds of what you'll do next. It's telling you how often its hidden predictions been right so far. So if you've hit 100 keys, and it secretly guessed 75 of them right, it will display 0.75. The more you play, the more slowly this number moves, because each additional guess has a smaller effect on the overall average.
I was able to get it to slowly go down, more or less, by switching things up whenever I saw the percentage start increasing. Usually kept doing the same key or pattern as long as it went down and as soon as it started ticking up I switched it.
A couple things I did to get it down to ~.44
Press based on random environmental input (woman's voice on tv = f, man's voice = d, while flipping channels).
Do it very quickly, while paying attention to something else that utterly absorbs your attention, and don't look at the meter.
This is pressing two letters, and you're explicitly told to press them in a way which feels random (which is going to be culturally determined). Aristotle would say that this is about habit, not free will -- unless you're opting to ignore the instructions.
Can you elaborate on how one's perception of randomness is subject to cultural bias? You do have to realize that frequent runs of repeated characters are expected to appear in a truly random sequence, but that's more a matter of basic reasoning (or perhaps education) than culture.
"FFFFFFFF" or "DDDDDDD" are random sequences if they're generated by a random process, but we [note: I wonder who 'we' means] expect "random" to mean "frequently changing" or "chaotic-looking".
This might easily be more about formal education in statistics than about culture, though. I'd expect people from any culture we know about to have the intuitive expectations I mentioned -- identifying randomness with chaos.
At first I was stuck at ~0.95 but I was just striking the keys really fast but then I concentrated and looked at the keys and really tried to feel which key I wanted to press and it worked. I was able to score lower ~0.4. I'm computationally irreducible for now.
Weird. When I tried to just "type random" without looking at the keys, it was routinely getting me around 75%. I tried looking at the keys as you described, and my score immediately dropped to around %50 and was even below %50 for a bit. Then it crept back up to around %60 when I started talking to a co-worker while typing.
There's been a number studies that showed that humans are bad at random number generation.
e.g. "Humans cannot consciously generate random numbers sequences" http://www.ncbi.nlm.nih.gov/pubmed/17888582
I guess that goes back to the original premise of the experiment. That "free will" is unpredictable. And the way I thought about it is that for me to have a true inclination -- not to just do it "randomly" -- I'd have to look at the keys and feel something.
but how does it know to use the opposite for him and not for other guesses? I make a similar mistake when I try to remember something by noting "it's not the choice I'm naturally inclined to choose, it's the other one". After I get used to using that heuristic, the correct choice becomes the one I'm naturally inclined to do, but then I have trouble remembering "is it the choice I'm inclined to do or the opposite". I've since ditched that heuristic when I realize I'm doing it because it is quite detrimental.
That's the trouble with game theory-like heuristics. To a layman like me it's unclear how many times do you want to recursively apply it.
The 2/3 of the average guess game is an interesting illustration where if you take the game theory approach and recursively apply it you'll end up with 0 all the time:
https://en.m.wikipedia.org/wiki/Guess_2/3_of_the_average
It's written in a browser running Javascript, so by definition is only works on serial contexts. Plus, the key switches on your keyboard would give you at most 2 signals in parallel (f and d).
Got it to 0.5991561181434598 after a while (totally random would be "0.5").
Not sure how "free will" comes into play, if there was indeed free will, I could have freely decided to only press one, and the Oracle would have had close to 100% certainty.
Rather, what it measures is randomness or predictability, which is not the same thing as free will (especially "after the fact", e.g. after the choice is made).
Right. It tries to measure how random your brain's PRNG is, which has nothing to do with free will. A simple computer has no free will at all, but after being instructed to type randomly, it will score close to 0.5 without fail.
The "free will" quote comes from a Berkley quantum computing student, who chose keys as independently of pattern as he could [1], and this seems to take it out of context. It might make sense to retitle this submission to something marginally less clickbaity, like "test how predictable you are."
If I had to guess? I'd say most people believe they are capable of choosing their actions - but they would also admit to a reasonably predictable set of habits.
Yes, I think most people don't bring predictability into it at all. Free will is about the feeling of "I could do, or could have done, X instead of Y", but your search space only ever included X, Y, and Z. Just knowing your search space makes you highly predictable even if you maintain the freedom to choose inside that space. (And however we choose, it is not randomly, that would be terrible, but more chaotic in the same sense that weather has too many unknown inputs that we can't predict all the details we want to perfectly.)
Did they predict that I'd wire the {d,f} input to a PRNG? Did they predict that I have a lot of digits of pi memorized, or that I could use the hex 'spigot' formula to generate digits of pi, then convert d=0, f=1 (or the inverse)?
Did they predict which method I'd wire it up with and the exact source of the program I'd use to do so?
I'm going to hazard a 'no', so it looks like I have free will.
Being quite unpredictable just means that you do a lot of random things -- but says nothing about how you came to will them.
If someone, for example, was threatened and ordered to follow for years what a program with millions of suggestions told him to do, he would be very unpredictable, but totally without free will.
And the inverse: one could choose to do the same routine all his life, and thus be very predictable, but it would still be his free choice.
Fair enough, but was only speaking to the "clickbait" charge, by arguing that it's not misleading to characterize this as test of "free will" in the typical usage of the term.
But yeah, it's not the same as being unpredictable: that would look like seizures, not "gosh which ice cream flavor will he pick?:
The only way to assume that those are synonymous would be to ask if one can freely will themselves to be random. However, even that has more to do with aptitude and ability than free will. E.g. I can freely will myself to try and be an NBA player, but I'm bound by physical limitations. However, my inability to be an NBA player would not normally be considered a lacking of free will.
Whether or not determinism and free will are compatible is a very intricate debate that your throwaway remark doesn't do justice to. Look up Compatibilism.
The former is the metaphysical idea that all events are causally linked, a belief which we can never confirm or refute. The latter describes something's observable degree of predictability.
They're not related concepts at all. This demo has nothing to do with free will, Determinism, or Compatibilism - but it's an interesting study in predictability.
Is the distinction roughly "causally determined" vs "logically determined" ?
If we are talking about an abstract computing device, and whether it is deterministic, we mean whether its next state is always uniquely logically determined by the current state, correct?
Hmm, if an abstract machine behaves in a way that is uniquely logically determined, but which is not computable, do we consider such machines to be deterministic? I would think so, but I am not sure. We would be unable to predict it, does that make it not deterministic(computing) ?
Those people are wrong. Other commenters dealt with that below.
RE free will itself, I'm thinking it has more to do with metacognition and general complexity of thoughts. People may have free will, but they're running the same "free will algorithm" - give them the same inputs, and most of the time you'll get the same outputs. A lot of people don't want to consciously accept that, but it's a fact. It's e.g. why economics works at all - because people are predictable at scale.
But when do we most often reference the concept of "free will" in practice? In evaluating degree of responsibility for decisions. "You did X, which was the most obvious and beneficial in the short-term option, but you could've done Y or Z, which would be better - therefore you will suffer consequences". Or, "I won't do X even though the situation tries to manipulate me into doing it; I notice I'm being manipulated so I'll do Y instead". Scenarios like these refer to the human ability of a) non-greedy optimization (long-term planning), and b) being able to go meta many levels up, to notice their own patterns of thoughts and use them as an input to the thinking process. A feedback loop over metacognition if you don't mind.
This is still all pretty deterministic. The apparent randomness - I think - comes from the fact that a) two people never have perfectly the same set of inputs, because the internal state is dependent on one's life history, and b) in some cases slight variations in those inputs cause huge variations in outputs (butterfly/hurricane and all that). In a way, the main connection between "free will" and randomness may be just that a human doesn't have capability to perfectly predict the thought process of another human. We can do that to inanimate objects, we can do that to algorithms we write - at least in principle. But we know that other humans will always suprise us (at least because they can notice we're trying to predict them and start acting random to mess with us).
> "give them the same inputs, and most of the time you'll get the same outputs.... people are predictable at scale."
That's not quite how economics work. It's more like, outputs from a given set of inputs can be characterized in a statistical way at scale (which is subtly different from saying that individuals are predictable at scale.) It's not that every person will choose to buy pizza if it's $11.99 but not if it's $12.00, but instead that if you offer pizza for $11.99 you'll generally get between X and Y sales per day, and if you increase that to $12.00 you can expect to lose about Z sales, and you can mess with that curve to try to maximize revenues or whatever.
I would say that what makes "free will" an interesting concept is that it's about how one chooses to prioritize different values -- how they express meaning by their choices. In the sense of a "free will algorithm", it's how you choose to weight different factors in your algorithm. How do you weight comfort, pleasure, stimulation, challenge, avoidance of pain, etc.? How do you weight those things for others? In practical usage, we talk about "free will" when someone makes a choice that surprises us, showing that their value system or their evaluation matrix is different from ours.
if .5 is completely random, what does it mean that I got it down to 0.4781666666666667? I may have gotten it lower than that, but got tired of taking screenshots. Eventually I ended up back around .53
Any random result series has such "unlikely" occurrences -- you can get 100 6 in a row by throwing dice for example, it's just rare. It's the long term convergence that matters.
It means you were able to fake the algorithm out. Consider a very simple algorithm that guesses based on your previous two characters:
df d
dd d
fd f
ff f
This is presumably much simpler than what the page is doing, but will nonetheless work pretty well on humans by looking for alternations and long runs.
This algorithm is trivially defeated (chance of guessing 0) by the string
ffddffddffdd
Since that holds a pattern just long enough to get the algorithm to guess based on it and then switches.
For a little while, either intentionally or not, you must have been doing a more sophisticated version of this.
With perfect knowledge, it should be possible to get it down to 0.
The "free will" bit is more of a joke than anything, based on a quote from Scot Aaronson's book (explained in the GitHub Readme):
> I couldn’t even beat my own program, knowing exactly how it worked. I challenged people to try this and the program was getting between 70% and 80% prediction rates. Then, we found one student that the program predicted exactly 50% of the time. We asked him what his secret was and he responded that he “just used his free will.”
There's another Aaronson call-back: he's complained about how trivial results (like the core part of the Free Will Theorem) will get a lot more attention if you mention free will:
> I wrote a review of Steven Wolfram's book a while ago where I mentioned this, as a basic consequence of Bell's Theorem that ruled out the sort of deterministic model of physics that Wolfram was trying to construct. I didn't call my little result the Free Will Theorem, but now I've learned my lesson: if I want people to pay attention, I should be talking about free will!
Yeah, I feel like randomness and "free will"-ness are pretty orthogonal. That said, I was able to keep the thing below a maximum of .56-ish without using any of the tricks others have suggested here, like picking an insignificant digit in the accuracy number and pressing 'f' or 'd' based on its value, or using the even/oddness of successive positions in π, or whatever.
So, apparently, my brain's PRNG is pretty random. But I still have absolutely fuck-all idea whether or not I have "free will".
This is silly. Enough people (especially self-reporting), a large percentage will be well below 50% (unless it really is an oracle). About as soon as I started, it quickly dropped into the 30's. Even after several minutes and hundreds or thousands of keystrokes, it hardly moved over 50%.
Can this be used for game AI? I think it could. I wonder if people could apply this to games like Tennis? Or maybe to approaches to the basket in Basketball?
I am using KeePass 1.31's Random Password generator with minus and underline. The oracle is getting 59%. Hmm. Also, when I generate a 64 character string with two possible characters, KeePass claims I have 72 bits of entropy. Hmmm!
Good one. Right handed when really just typing as fast and supposedly as randomly as I can, it was at 0.99 and continuing to converge to 1. Left-handed (especially if I switch which fingers I use occasionally), I was < 0.4.
239 comments
[ 4.2 ms ] story [ 77.1 ms ] threadI'm not going to take the time to test it, but if you flipped a coin before each key press, this algorithm would eventually get to 50% accuracy. But in the short term it would probably be <50%, because it assumes it won't actually get random input.
So you may need to make a yes-no decision where you decide on yes about 65% of the time, when you consider lots of those decisions.
His solution? Glance at your watch. Divide the current minute into proportionate parts. See in which part the seconds hand is standing.
After all, it's pretty impressive when it appears to correctly get Frank Sinatra out of "does your character play video games for a living."
(They were using the grating slogan "the neural net on the Internet" when neural nets were the pariah of AI. It's kind of amazing.)
Actually, an interesting challenge might be to construct a sequence that gives the Oracle as low a score as possible (is there a nicer simpler way to describe this than simulating the Oracle's code and always choosing what it doesn't expect?).
Some searching tells me this already has a name, see https://en.wikipedia.org/wiki/De_Bruijn_sequence
So yes, just take a De Bruijb sequence B(2,6) (algorithms in the link can generate them) and repeat it.
For example, the 4-bit Gray code (under this interpretation) when concatenated into a single bitstream begins "00000001", which then contains the 4-grams "0000", "0000", "0000", "0000", "0001", so the predictor gains an advantage right at the outset. (I didn't check whether it then loses its advantage over time... that's an interesting question.)
This is like showing someone an elaborate mirror, and saying, "See? My copy of you does exactly what you're doing to do! You don't have free will!"
All you have to do is put another mirror in the middle and show them that it doesn't work as they claim.
Actually the quickest way I found to "break" it and hover at ~50% is to simply aim for the dead space between the "F" and "D" keys. If it was using non-adjacent keys like "F" and "J" instead it would be better for its purpose. "Q" and "P" might work even better since it would mostly force you to use 2 hands, making it much harder to score close to 50%.
I think its flawed - Whatever I press it never changes by more than 1%
How can the prediction for 'f' be 0.764 and for 'd' be 0.763 - they don't add up to 1.
Not exactly, seems to be a running average with a specific window.
It's making a (hidden) prediction about what key you'll press, and it's displaying the number of times it was correct over all your key presses.
So, the more you've pressed the keys, the less each successive press will change it's overall success rate.
This might easily be more about formal education in statistics than about culture, though. I'd expect people from any culture we know about to have the intuitive expectations I mentioned -- identifying randomness with chaos.
https://en.wikipedia.org/wiki/Chaitin's_constant
The 2/3 of the average guess game is an interesting illustration where if you take the game theory approach and recursively apply it you'll end up with 0 all the time: https://en.m.wikipedia.org/wiki/Guess_2/3_of_the_average
Not sure how "free will" comes into play, if there was indeed free will, I could have freely decided to only press one, and the Oracle would have had close to 100% certainty.
Rather, what it measures is randomness or predictability, which is not the same thing as free will (especially "after the fact", e.g. after the choice is made).
The "free will" quote comes from a Berkley quantum computing student, who chose keys as independently of pattern as he could [1], and this seems to take it out of context. It might make sense to retitle this submission to something marginally less clickbaity, like "test how predictable you are."
[1] https://github.com/elsehow/aaronson-oracle
Did they predict which method I'd wire it up with and the exact source of the program I'd use to do so?
I'm going to hazard a 'no', so it looks like I have free will.
Being quite unpredictable just means that you do a lot of random things -- but says nothing about how you came to will them.
If someone, for example, was threatened and ordered to follow for years what a program with millions of suggestions told him to do, he would be very unpredictable, but totally without free will.
And the inverse: one could choose to do the same routine all his life, and thus be very predictable, but it would still be his free choice.
But yeah, it's not the same as being unpredictable: that would look like seizures, not "gosh which ice cream flavor will he pick?:
The former is the metaphysical idea that all events are causally linked, a belief which we can never confirm or refute. The latter describes something's observable degree of predictability.
They're not related concepts at all. This demo has nothing to do with free will, Determinism, or Compatibilism - but it's an interesting study in predictability.
If we are talking about an abstract computing device, and whether it is deterministic, we mean whether its next state is always uniquely logically determined by the current state, correct?
Hmm, if an abstract machine behaves in a way that is uniquely logically determined, but which is not computable, do we consider such machines to be deterministic? I would think so, but I am not sure. We would be unable to predict it, does that make it not deterministic(computing) ?
Ghost in the Quantum Turing Machine https://arxiv.org/abs/1306.0159
RE free will itself, I'm thinking it has more to do with metacognition and general complexity of thoughts. People may have free will, but they're running the same "free will algorithm" - give them the same inputs, and most of the time you'll get the same outputs. A lot of people don't want to consciously accept that, but it's a fact. It's e.g. why economics works at all - because people are predictable at scale.
But when do we most often reference the concept of "free will" in practice? In evaluating degree of responsibility for decisions. "You did X, which was the most obvious and beneficial in the short-term option, but you could've done Y or Z, which would be better - therefore you will suffer consequences". Or, "I won't do X even though the situation tries to manipulate me into doing it; I notice I'm being manipulated so I'll do Y instead". Scenarios like these refer to the human ability of a) non-greedy optimization (long-term planning), and b) being able to go meta many levels up, to notice their own patterns of thoughts and use them as an input to the thinking process. A feedback loop over metacognition if you don't mind.
This is still all pretty deterministic. The apparent randomness - I think - comes from the fact that a) two people never have perfectly the same set of inputs, because the internal state is dependent on one's life history, and b) in some cases slight variations in those inputs cause huge variations in outputs (butterfly/hurricane and all that). In a way, the main connection between "free will" and randomness may be just that a human doesn't have capability to perfectly predict the thought process of another human. We can do that to inanimate objects, we can do that to algorithms we write - at least in principle. But we know that other humans will always suprise us (at least because they can notice we're trying to predict them and start acting random to mess with us).
That's not quite how economics work. It's more like, outputs from a given set of inputs can be characterized in a statistical way at scale (which is subtly different from saying that individuals are predictable at scale.) It's not that every person will choose to buy pizza if it's $11.99 but not if it's $12.00, but instead that if you offer pizza for $11.99 you'll generally get between X and Y sales per day, and if you increase that to $12.00 you can expect to lose about Z sales, and you can mess with that curve to try to maximize revenues or whatever.
I would say that what makes "free will" an interesting concept is that it's about how one chooses to prioritize different values -- how they express meaning by their choices. In the sense of a "free will algorithm", it's how you choose to weight different factors in your algorithm. How do you weight comfort, pleasure, stimulation, challenge, avoidance of pain, etc.? How do you weight those things for others? In practical usage, we talk about "free will" when someone makes a choice that surprises us, showing that their value system or their evaluation matrix is different from ours.
It means you were temporarily a statistical outlier.
This algorithm is trivially defeated (chance of guessing 0) by the string
ffddffddffdd
Since that holds a pattern just long enough to get the algorithm to guess based on it and then switches.
For a little while, either intentionally or not, you must have been doing a more sophisticated version of this.
With perfect knowledge, it should be possible to get it down to 0.
> I couldn’t even beat my own program, knowing exactly how it worked. I challenged people to try this and the program was getting between 70% and 80% prediction rates. Then, we found one student that the program predicted exactly 50% of the time. We asked him what his secret was and he responded that he “just used his free will.”
> I wrote a review of Steven Wolfram's book a while ago where I mentioned this, as a basic consequence of Bell's Theorem that ruled out the sort of deterministic model of physics that Wolfram was trying to construct. I didn't call my little result the Free Will Theorem, but now I've learned my lesson: if I want people to pay attention, I should be talking about free will!
http://www.scottaaronson.com/democritus/lec18.html
So, apparently, my brain's PRNG is pretty random. But I still have absolutely fuck-all idea whether or not I have "free will".
Fun two-and-small-change minutes, though.
Don't ask me what 'trying hard' means -- I don't know! :-)
I am using KeePass 1.31's Random Password generator with minus and underline. The oracle is getting 59%. Hmm. Also, when I generate a 64 character string with two possible characters, KeePass claims I have 72 bits of entropy. Hmmm!