Personally I don't think you are given any new information in the Monty Hall problem that's worth anything. You always knew there was a 100% chance that one of the doors in that set has a goat behind it, which door that is is pretty much irrelevant.
Or to put it another way how many people would stick with their chosen door if instead of opening a door and showing a goat they were simply asked "would you like to stick with your current choice of door, or switch and take the best result from the other doors?"
> switch and take the best result from the other doors
That is, effectively, what the choice boils down to. But that being said, since the problem is stated as only getting to pick one "other" door, the "extra information" received is knowing which of the other 2 doors has the best result.
Try thinking of it this way: The information is in which door the host does not open.
2/3rds of the time, your initial choice will be a goat. In those cases the host deliberately avoids opening the remaining door with a car - thereby telling you where it is.
If you chose the car door initially, then indeed you get no new information. But that only happens 1/3rd of the time.
When Monty reveals a goat, your odds increase from 1/3 to 1/2 if Monty is picking randomly and just happened not to reveal a goat (and if he revealed a car we start from the top, or you automatically lose, or you automatically win, or...), but to 2/3 if Monty will reliably reveal a goat. I think it's hard to say that you have no extra information in that latter case.
It's true that we can add other options (pick both doors) of equal likelihood, but it's because of our increased information that the reduced option of picking just one door isn't actually worse off.
The GP knows all this, and agrees it's better to switch (look at the second paragraph). They just dislike the common formulation (that is given in terms of the extra information from the boot being opened), and find another formulation more compelling (your original choice is one door, while the switch is actually to the best prize from the other two doors, not simply to another one door).
The GP is not contesting this, they just think that the whole discarding/revealing of the goats is an unnecessarily complicated explanation for why this works.
The way they put it, the strategy becomes obvious if you instead reformulate the problem like so: there are three doors with three different prizes. You get to pick one door, but are then given a choice: stick with your original choice, or choose to get the best prize from the other two doors.
The solution (switch!) is more obvious in the second formulation. I have no idea if it's easier to convince someone else of the reasoning in the first formulation, or to convince them that the second formulation is equivalent to the original.
I didn’t question the math, but what made me intuitively believe the Monty Hall problem was this:
Me: I am thinking of a number between 1 and a million. If you guess correctly, you get a prize.
you make your guess
Me: Ok, I will reveal that the number is either $YOUR_GUESS or 452,871. You may keep your original guess or switch to 452,871.
Of course, you would switch, because the chances you guessed correctly originally were 1 in a million.
I always think of the Monty Hall problem with 100 doors. And with the game show host revealing 98 'goat' doors thus leaving two doors left (the original guess, and another one). You'd switch then right? :)
Your first paragraph makes it seem like you don't accept the common solution to the Monty Hall problem, which is why people have responded explaining it to you, even though the second paragraph makes it clear that you do, and just dislike the way it is explained.
Personally, I'm not sure that it would be easier to convince someone that the second formulation is equivalent to the original problem, but next time I find a dis-believer I will give it a shot!
The Monty Hall problem is easily resolved by imagining there are a million doors instead of just three. The host opens 9,999,998 doors, revealing goats behind each. You'd obviously switch doors to the one the host left closed, and the reasoning you followed will be just as valid for any number of doors including 3.
This problem is completely different, since the number of degrees of freedom remains fixed throughout the decision process. As someone else pointed out, it is trivially resolvable by realizing that no matter how many times you switch envelopes, you're really only making one choice, and doing so with 50-50 odds of picking the more profitable envelope. The offer to switch amounts to meaningless magician's patter, whether the offer is made once or a million times.
When I was initially exposed to Monty Hall, what confused me the most is that I didn't know that the host would always open a door with a goat, rather than just picking one of the remaining doors at random. In the latter case (i.e. the case where the host can pick a car), even provided that you are in the universe where he picked a goat, switching and staying both yield a 50% chance of success.
I love this problem because it is so simple, and the false line of reasoning is so compelling that it would hardly raise an eyebrow if you saw it in an academic paper and yet the conclusion is so obviously wrong. Decision problems are tricky and in non intuitive ways.
Step 3 (and thus also 1 and 2 on reflection) stands out to me immediately as describing two different situations for the entire game.
The case where switching gives you 2A and the case where switching gives you A/2 describe 2 completely different universes, not two different actions in one.
Agreed. Step 6 is where it breaks down, because it's redefined A as being one of the two envelopes to being the halfway point between those two envelopes. Until step 6, it's describing two parallel views of the universe (depending on which envelope you first get - hence steps 4 and 5), and then mashes them together in a way that doesn't work.
Take the practical example of $100 and $200: A is either $100 or $200 depending on which envelope you receive first, so the equation is either 0.5*A + 0.5*2A, or it is 0.5*A + 0.5*(A/2). It can never be 0.5*2A + 0.5*(A/2)
Even after looking at those numbers, it still felt wrong to me. It eventually occurred to me that this was because my intuition was telling me the total should add up to A. And it doesn't; it adds up to 9/8 A. It took me a while to realize that my intuition was wrong, and that there's no reason the total should be A.
The thing that helped me reconcile this was realizing that the A in the two equations are different values; then replacing them.
(0.5⋅(0.5⋅A + 0.5⋅2.A)) + (0.5⋅(0.5⋅A + 0.5⋅0.5⋅A)) = 9/8 A
^ but the A in the left grouping (where it's 100) is different than the A in the right grouping (where it's 200).
Well, the simple way to put is that "A" isn't fixed. The "expected value" argument in steps 6-7 is acting like "A" is a single value when "A" will be larger or smaller depending on what envelope you picked.
I think that subtly misses the point. The problem is that you're implicitly using a distribution that... isn't a distribution. And with this particular not-a-distribution, whether you should switch or not does not depend on the value of A. But with any actual distribution (... I think?) it does, at which point... no paradox. It's true that it's not clear (at least to me, but perhaps more generally) what distribution we should assume, but as long as we avoid treating something that isn't a distribution as if it were one we avoid the worst of it.
I think the assumption that "for all x, p(x) = p(2x)" implies a uniform distribution on the natural numbers.
Imagine we have a distribution that satisfies that assumption, and then someone tells you they've sampled from that distribution and found that the result is of the form (say) 7*2^k, for some k > 0. That conditional distribution for k would seem to have to be uniform, right?
> We're not dealing with the natural numbers, right?
Well, you can either assume that the envelope's numbers are specified only up to pennies. Or you can just round and say you don't care about sub-pennies.
Or you can go with your suggestion, and look at the lack of a uniform distribution over the reals / rationals.
Well taking the mean of products doesn’t make much sense anyway… but if you take a log and average then you end up with the typical solution. If you take a geometric mean then you end up with 1, which implies there’s no difference either.
No, joe_the_user is exactly correct. The expectation they compute in step 7 requires that A is a constant. The formula seems to just be the regular formula for expectation: Value that B can take times probability that it takes that value, summed over all possible values.
But the "possible values" they put in are A/2 and 2A, which makes no sense. A is a random variable.
Don't forget that random variables are really functions on a probability space with an implicit argument, so (1) when multiple random variables appear in a formula, they're implicitly evaluated at the same argument omega, and the formula must hold for all omega from the probability space, and more importantly, (2) there can't be free-standing omegas in a formula for expectation because expectation is an integral (or sum in this case where the probability space is discrete) over all possible omegas.
In my opinion, the Wikipedia article is making a disservice to readers by mentioning unnecessarily complex mathematical arguments involving bayesian reasoning and infinite distributions. I believe all of these are distractions from the more fundamental typing error that invalidates the switching argument.
> in particular, the puzzle is not solved by the very simple task of finding another way to calculate the probabilities that does not lead to a contradiction
Honestly, the more I've thought about this problem the more that statement bothers me. It is just such a nonsense goal.
The reason the calculations are wrong are fundamentally related to why correcting the calculations leads to the correct probabilities. If you understand why the problem gives you the wrong result, you proceed to correcting it, and you get the right result - that doesn't mean you didn't understand. It means you did. You can tell you did, because you have the correct answer.
But if I show the calculations produce the correct answers, well, that is a step too far. We can now dismiss the understanding on the basis that it got the correct answer, apparently?
It's no wonder the author of the wiki writes such a bullshit claim - that no one can agree on a definitive conclusion. Their terms of engagement are self-defeating. Correctness is error, because correctness means you didn't /really/ understand. Its a no true scottsman fallacy - get the right answer, and you aren't engaging with the 'real' problem.
But its a flawed no true scottsman, because it defines something measurable: it tells us the goal, that it is to avoid this type of mistake in our thinking. And generalized algorithms for solving decisions problems that include this as a case which is successfully solved are many - and they just so happen to be in the imperfect information setting. And that setting has studied the problem of infinite recursion and has solutions for them. Which I can apply. To get the right answer. And, in general, avoid the problems they claim we aim to avoid.
More importantly and to the point of this thread - this isn't an intractable debate, because its been solved and used in production settings for literally decades. So what if some people are going to pretend it isn't? This isn't an unsolved problem. The actual game theory math is /well/ beyond this level of complexity. Its contending with things like environments where you have so much complexity you have to reduce to a blueprint abstraction, not stumbling at a decision problem that is quite literally simpler than rock paper scissors.
You seem to misunderstand the point of that constraint. Correction is not an error or a "step too far," it's just insufficient.
You can arrive at the correct conclusion either by pinpointing exactly where the original argument is incorrect, or you can come up with a completely different argument that does not have an error.
The puzzle challenges you to pinpoint the error because coming up with the correct solution is trivial (and the puzzle is deliberately set up this way).
This is not a "nonsense goal." If this came up in real mathematical research - two papers coming to contradictory conclusions - and no one could find where either paper went wrong, we would have a real paradox on our hands.
I think this, because I disagree with most people here about what the actual paradox is.
I think the paradox is that the algorithm equates the expected value of the contents of an envelope with the expected value of a policy choice for a player. When I correct what I feel is the root of the paradox, my solution drastically differs in fundamental ways such that the way the problem restricts to pointing out the wrong step feels disingenuous.
The entire structure is wrong, because even if you do correct the error that leads to the wrong EV for the envelope, you still haven't resolved the paradox. The right probabilities don't resolve the paradox, because they still imply that always switching has the same EV as not switching. If they were really equal, I could always choose switch, but I can't - so the paradox is still there.
My resolution ends up being so critical of their argument that the entire way they go about solving gets thrown out. I end up seeing, not just a specific wrong EV calculation, but a decision problem that is just fundamentally using an inappropriate algorithm to determine the policy function.
With all due respect ... this is basic probability theory. It's not really controversial what the solution is. The article's failures are mainly pedagogical.
We can agree that "the entire structure is wrong" because the "entire structure" is giving a wrong formula for the EV and saying "this is the formula for the EV."
Yes, switching and not switching have the same EV and you can always switch.
Mike, even when you get the correct result using the right rules the expected value of an envelope is not the expected value of a policy. I'm not agreeing with you that the problem is the expected value calculation. I'm telling you the problem is deeper than that. Fix the expected value calculation and you still have a paradox, because you are making a decision not on your expected value but on the expected value of the envelope. These are two different things and a rational person shouldn't eliminate the dependency between their policy and the reward they get.
To try and stress to you how big a problem this is, pretend you were playing the game and you wanted to find the right move that was going to make your EV the highest. Now, if you go with the logic of the problem, you are allowed to select not on the basis of your EV, but on the basis of some other EV. So instead of choosing the envelope, why not choose something else that has no relationship with our EV? Say, the weather in Alaska. If it is sunny, we like sunny. So switch. If it isn't sunny, we don't like that. So keep. It is crazy to do this, because there is no relationship between the EV you are using as a selection criteria and the EV your policy gets. This is the same situation as using the EV of the envelope. It sounds really crazy when you use the Alaska example, because it is so obviously unrelated. It sound so reasonable when you use the envelope example, because it isn't as obvious that they are unrelated. Yet for the policy of P(switch)=1, the ev of the envelope and the ev of the policy with respect to the game are not the same thing.
Now imagine the wikipedia article for the Alaska problem variant of the two envelope problem. Do you really think everyone would be so focused on the EV of the envelope as the step that was wrong? How could they? We have the same paradox still, but there is no EV calculation for the envelope included in the problem. If we can remove the EV calculation, yet still have the same paradox, it seems to me the paradox is not the expected value calculation.
So what is my solution? Well, to actually find your correct policy function you need to get the argmax of the policy with respect to the game. There are multiple ways to do this:
- Reinforcement learning does it by finding argmax pi with respect to Q_pi(s, a) = R(s') + P(keep)Q_pi(s',keep) + P(switch) Q_pi(s', switch).
- Game theory sets it up a bit differently. You define a similar graph using a different formalism, but simplified to operate over information sets. You can use a thing called regret matching; basically it turns out that if you play in proportion your normalized counterfactual regret, the average of those policies is the optimal best response.
In both cases you need to do something about the fact you're actually on an infinite graph. So the actual solution in the general case looks very very different from their way of solving the problem. It isn't just simple probability; I mean, it is, but taking the limit of an infinite sequence and taking advantage of the properties of markov chains aren't usually what I think of when someone tells me that something is simple probability. That is one formalism. In the other, we do have simple probability, but it isn't necessarily obvious that the central limit theorem gives us the optimal policy when we play in proportion to not just our regret, but our counterfactual regret. So yes, simple probability, but also, most people who know simple probability don't necessarily even know what a counterfactual is. So maybe not that simple after all.
But lets say we stick to the problem. We are here to learn how to avoid this problem, right? Nope. If you don't do things like this, you'll just be wrong in more complicated situations. Because the EV of the envelope is not the EV of the game with respect to your policy. This gets increasingly true as your imperfect information ga...
> We can agree that "the entire structure is wrong" because the "entire structure" is giving a wrong formula for the EV and saying "this is the formula for the EV."
We aren't in agreement about this. I realize we have to fix this, but fixing it doesn't resolve the paradox. It is a red herring.
> Yes, switching and not switching have the same EV and you can always switch.
With all due respect, this isn't true and asserting this doesn't resolve the paradox. See my other reply for why it doesn't resolve the paradox.
This may be progress, but I don't see how it eliminates the "paradox". The claim in the "paradox" answer is that when you switch envelopes, "A" is not one of the possible outcomes. If "A" isn't a possible outcome then it shouldn't show up in the expectation value.
I agree that's true once we know that the two envelopes have 100 and 200 dollars. For me the tricky part is, how do we know it's 100 and 200? Maybe we have two envelopes with 100 and 50 dollars instead.
It doesn't actually say that. It says the other envelope will either contain 2A or A/2. The envelope you're holding is A. The other envelop is either twice that, or half that.
the "writing" doesn't say it but the written formula at bullet point 7 says it. That's the mistake, the formula is wrong and does not describe reality.
This is incorrect. Bullet point 7 in the written formula does not refer to a corresponding pair of envelopes, as you imply. Instead, it refers to "the other" envelope only: 1/2 chance that "the other" envelope contains 2A, and 1/2 chance that "the other" envelope contains A/2. Notice that both references reference the same object.
> The right formula is changing either the 0.5A by A or the 2A by A.
No it's not, that's not how you make EV calculations. As I already said in my previous post, the parts in the EV calculation do NOT refer to the pair of envelopes. Both parts refer to the same object, not to different objects. You are trying to change the EV calculation such that the different parts refer to different envelopes. This does not make any sense.
> Try with A being a real like 100 usd numbers to see it.
...and that's not a way to prove / demonstrate the correctness of math. Yes, if you change these numbers in this way, you get EV 0, which is the correct answer. Just because the answer is correct does not mean the calculation is correct. In this case it isn't.
The person you are responding to is right actually. I drew in an ASCII art a diagram of the various game trees so it will be easier for you to see this:
On Wikipedia they created two trees one with 2A.
-> Keep -> 2A
2A -> Switch -> A
0.5 ->
A -> Keep -> A
-> Switch -> 2A
And another with 1/2A
-> Keep -> A
A -> Switch -> A/2
0.5 ->
A/2 -> Keep -> A/2
-> Switch -> A
Notice now that there is no game in which A is both 2A and A/2.
If you don't think this is the error, you're not thinking in imperfect information terms. You're thinking in perfect information terms. This sort of splitting action is allowed in perfect information games! In imperfect information games, you are in multiple subgames at the same time. A has to be the same A in both subgames, because you can't discriminate between which game you are in. So you're not allowed to do this. It is wrong!
This gets at the core of what the question is trying to make people notice. You can't do this with imperfect information. You can only do it with perfect information. You can't do it over information sets and strategy spaces. You can only do it over states and action spaces.
But they do it. And then they combine them back into the same analysis when they say 1/2(Game tree 1 switch case) + 1/2 (Game tree 2 keep case). And you can't do that, because game tree one depends on game tree ones keep case and game tree 2 depends on its switch case! You can't tell which you are in, so you are in both - not one or the other. You can't combine the two different subgames from two different trees like you could in perfect information! You have to stick with the same game tree.
And that is what the person you are mistakenly correcting is saying. Pick one or the other, you can't have both.
PS: There are other errors in the problem, like that when you solve it with an MDP or the Bellman equations you actually /do/ end up with 100% Switch having an expected value of zero. So if you spotted some other error, that doesn't mean that the person you are correcting is wrong in their correction. There are multiple issues with the way the wikipedia article approaches the conclusion.
PPS: This exact confusion that people are talking about is actually something that happened in practice in the solving of imperfect information games! IIRC the Pluribus paper author corrected a reasoning error of this type when he introduced sub game reach solving? I might be misremembering here, since its been a while since I listened to his talk, but my fuzzy memory suggests that confusion about subgames interacting with each other was the root cause of a failure in some of the poker research for subgame solving in a blueprint abstraction.
> I drew in an ASCII art a diagram of the various game trees so it will be easier for you to see this:
This game tree only makes sense if you change the definition of A. In particular, you need to change the definition to "A = 1/3 of the total amount of money in the envelopes". If you define A like this, then your game tree is correct, and it corresponds to the "simple resolution" described on the Wikipedia page (where they use x instead of A, for clarity). This is an unsatisfactory answer, because it provides an "alternate path" to the correct answer, without demonstrating which step went wrong in the "incorrect path" that leads to the wrong answer. It is not particularly difficult to construct these "alternate paths" that lead to the correct answer. The difficulty of the puzzle is in demonstrating what goes wrong in the incorrect path, as stated on the Wikipedia page:
The puzzle is to find the flaw in the very compelling line of reasoning above. This includes determining exactly why and under what conditions that step is not correct, in order to be sure not to make this mistake in a more complicated situation where the misstep may not be so obvious. In short, the problem is to solve the paradox. Thus, in particular, the puzzle is not solved by the very simple task of finding another way to calculate the probabilities that does not lead to a contradiction.
> Notice now that there is no game in which A is both 2A and A/2.
I have no idea what you are referring to here. I never claimed that A = 2A in any circumstance (there is nonzero amount of money in the envelopes).
> If you don't think this is the error, you're not thinking in imperfect information terms. You're thinking in perfect information terms. This sort of splitting action is allowed in perfect information games! In imperfect information games, you are in multiple subgames at the same time. A has to be the same A in both subgames, because you can't discriminate between which game you are in. So you're not allowed to do this. It is wrong!
I agree with you that the crux of the issue is in the definition of A. In particular, "mixing" "two different A's" within the same computation, even though they sort of mean different things. This is a fuzzy explanation that gives a feeling "something about this explains the issue", but it's not a complete explanation that precisely pinpoints what is wrong.
> This gets at the core of what the question is trying to make people notice. You can't do this with imperfect information. You can only do it with perfect information.
This is false. We can easily construct imperfect information games similar to this one, but tweak a few things in such a way that a "naively constructed EV calculation" yields the correct results.
> And that is what the person you are mistakenly correcting is saying. Pick one or the other, you can't have both.
Nope! I was very clear about which very specific claims I was refuting. They were unrelated to the topics you are bringing up here. I posted a long, clarifying comment to a sibling comment here, you can look that up if you want to refute specific claims.
> So if you spotted some other error, that doesn't mean that the person you are correcting is wrong in their correction. There are multiple issues with the way the wikipedia article approaches the conclusion.
I didn't say "I spotted error X in Wikipedia which explains Two Envelope problem", I said "I spotted error X in this other poster's Hacker News comment". Please read the very specific comments I made about very specific claims made by the person.
> This game tree only makes sense if you change the definition of A.
Switching away from using A to avoid ambiguity.
> This is a fuzzy explanation that gives a feeling "something about this explains the issue", but it's not a complete explanation that precisely pinpoints what is wrong.
Yes it does, actually. Well, to me. I'll go more in depth so it less fuzzy for you.
The issue is that when you handle imperfect information correctly you have counterfactuals implied by your state-oriented reasoning, because you can't actually escape into perfect information world - you're not in it. When you combine the state-derived probabilities the - I don't know the word for this, so forgive me, but the - superposition of the two counterfactuals implied by those state-derived probabilities end up occupying the same space. They exist, because they are implied by the imperfect information game. They just aren't writing them out in the equations on wikipedia.
Edit: My 'spatial' probabilistic reasoning in my head is explained here, but I show you the identity rule that proves this in the next section.
> I never claimed that A = 2A
You aren't, but the counterfactual terms that get realized into the same position, but which weren't written out, claim that Z = 2Z and that Z = Z/2. The two cases were under two different games. So their implied counterfactual components don't agree with each other and we should never have been allowed to combine them.
We agree on this!
You're just more focused on the they come from different games part and not noticing the other part of what I'm telling you: you can equate the counterfactual components that they don't show between the two games, because there is an identity implied by being under imperfect information. Look at the set relationship closely across the two subgames and you'll realize that the set is defined to be equal to itself across all subgames where the set is used.
That is quite literally one of the logical contradictions. This is where they claim that 2=1. The sets aren't equal to each other after their operations. So by the identity implied by the set, they've claimed 2=1.
And yet when I try to claim this is a core aspect you throw out this...
> This is false. We can easily construct imperfect information games similar to this one, but tweak a few things in such a way that a "naively constructed EV calculation" yields the correct results.
This is a really dumb nitpick with very poor support. If you're going to be this fallacious, you might as well go all out. Why not say I'm wrong because first graders learning to add numbers aren't learning how to handle imperfect information subgames? After all, your argument for me being wrong is that the pedagogical value of problems is independent of the subproblems they contain.
> Nope! I was very clear about which very specific claims I was refuting.
You have a more subtle point than I thought you were making. I'll check out your other post.
> I didn't say "I spotted error X in Wikipedia which explains Two Envelope problem", I said "I spotted error X in this other poster's Hacker News comment". Please read the very specific comments I made about very specific claims made by the person.
Well, he does have another error; EV isn't defined over all policy choices. It is a problem on many of the wikipedia solutions too. He does see it - its why he mentioned other people talking about infinite series - but his correction attempt was more like "halt here because of type error" which is something that I as a programmer can respect.
Basically when the wiki says 2A because less than and A/2 because greater than things are okay, but we're in dangerous territory. If we ever introduce uncertainty the imperfect information scenario we are in is going to force us to acknowledge that we don't know which game we are in. Which means if that happens we don't have just 2A, but a information set {2A, A}. And we don't just have A/2, but an information set {A/2, A}. In game theory if you have two subtrees and they share an information set than the information sets are equal to each other. You can't tell which subtree you are in, so the information set contains both subgames. You can think of one as factual and the other as counterfactual and vice versa, but the elements in that set need to be equal. When we get to step seven we do a calculation that reintroduces our uncertainty. We multiply by the probability of being in each case. However, we don't actually know we are in one case and we don't know we are in the other case. So combining these two situations leads to the counterfactual part of subgames emerging. So we have {A, 2A} and {A/2, A}. The sets aren't equal. We can choose some parts of this, like grabbing A and comparing it to 2A. So now we have an identity rule that just let us say 2A=A. 2=1. Our contradiction is found.
> This is an unsatisfactory answer, because it provides an "alternate path" to the correct answer, without demonstrating which step went wrong in the "incorrect path" that leads to the wrong answer.
Eh, I mean, I guess I'm cheating by claiming an interpretation they aren't trying to use. This is why I detest the "no true scottsman" setup to the question. We can create an identity that maps their wrong step into my formalism where what they did is wrong in the way I claim it is. Even though they didn't "try to do it" doesn't mean I can't see that they did do it. But apparently - even though game theory notation neatly avoids these pitfalls - we need to stick to the footgun notation. It just seems stupid to me. If you want to avoid the problem, use the notation that trivializes avoiding the error. I really like the other person's analogy to type errors, because it is such a similar idea to what I'm saying, but they just use a different part of math to assert it. There are ways you can just make this decision fail to typecheck. If you don't want to have these type of errors? Be stricter about your typing so at to prevent them.
> the counterfactual terms that get realized into the same position, but which weren't written out, claim that Z = 2Z and that Z = Z/2.
Nobody makes this claim, directly or indirectly. If you got this result by formalizing the plain English statement into a proof assistant, then the error was introduced during the step where you interpret the plain English into a formal statement. If the claim "Z = 2Z" was inherent to the plain English version of the problem, then it wouldn't be possible to run the wager as a simulation, but it is. The entire Two Envelopes problem - including step 7 - is possible to simulate in code: https://pastebin.com/jPyZVrkx
What the simulation demonstrates is:
- It's perfectly possible to define "expected value of the final envelope in relation to the value of the firstly-chosen envelope", as the Wikipedia page describes. No contradiction exists that would prevent this computation.
- It's also perfectly possible to define "expected value of the final envelope in absolute terms".
- These are different goals to maximize. A rational actor should maximize goal 2 ("expected value of the final envelope in absolute terms") in this variant of the game. It's also possible to construct another variant where a rational actor should NOT maximize goal 2, but should instead maximize goal 1 ("expected value of the final envelope in relation to the value of the firstly-chosen envelope").
- The error in Wikipedia's "compelling line of reasoning" appears to be at step 8 where they conclude that "they stand to gain by swapping". This implies that a rational actor should maximize goal 1, when in fact a rational actor should maximize goal 2 instead. This is the error in Wikipedia's line of reasoning.
> When we get to step seven we do a calculation that reintroduces our uncertainty. We multiply by the probability of being in each case. However, we don't actually know we are in one case and we don't know we are in the other case. So combining these two situations leads to the counterfactual part of subgames emerging. So we have {A, 2A} and {A/2, A}. The sets aren't equal. We can choose some parts of this, like grabbing A and comparing it to 2A. So now we have an identity rule that just let us say 2A=A. 2=1. Our contradiction is found.
I'm not a mathematician, so I don't understand expressions like "counterfactual part of subgames emerging" or "there is an identity implied by being under imperfect information". I appreciate you writing back at length, but the majority of what you wrote went straight over my head. The impression I have is that you formalized the problem in a manner that lead to a contradiction. If this contradiction was inherent to the original English problem statement, it wouldn't have been possible for me to simulate the problem. But it is. So it seems to me that there is no inherent contradiction in the English problem statement, it seems that the contradiction was introduced during the formalization step.
> I detest the "no true scottsman" setup to the question [...] If you want to avoid the problem, use the notation that trivializes avoiding the error.
I wouldn't describe the setup as a "no true scottsman" scenario, because the setup describes what counts as a scottsman: determine which step in the line of reasoning is incorrect, why and under what conditions. I appreciate that you tried to do this when you identified step 7 and explained what was wrong with it in your opinion. This is also what I tried to do in my explanation, with regards to step 8 and the comparison between the two goals.
Besides, it's very trivial to conclude that nothing can possibly be gained by switching (in absolute money terms, which should be the goal a rational actor chooses to maximize). It would be boring and...
> It would be boring and unsatisfying to accept a simple answer that explains how to get the "correct answer" without explaining what made it a paradox in the first place
Well, this is just a place where they do the math wrong. We can quibble about why, but it isn't a paradox. We both agree they do it wrong and we even have an overlap in the reason why - we both think they aren't comparing them relative to all the subgames they are in and we both think that this is required to reason correctly. So lets say we get to the correct relative EV of 0.
If both are worth the same, well, why not always switch? They are the same EV, right?
IMO this is the real paradox. We have a false equivocation. EV(policy, game) is not equal to EV(envelopes), because for the policy `ALWAYS_SWITCH` we get the logical contradiction that undefined = EV(envelope) or if you take the limit with a discount under a different formalism that 0 = EV(envelope)
When you tackle this problem starting there rather than at the other error the entire algorithm changes, because we need to modify the problem so that we can calculate EV with respect to policy. The algorithm changes enough so that it really annoys me, because I have this intuitive feeling that I'm changing the solution so much so that I'm no longer working within the spirit of the puzzle.
That is why I feel like "no true scottsman"; to use a metaphor, they gave me a coloring book and told me to color something beautiful but that I had to stay in the lines, but the lines of are a skunk and I want to draw a sunrise. I don't want to correct just the one particular step. I hate the framework that forces me into the confusion by falsely implying that if I get the right envelope EV, I know my EV.
> If both are worth the same, well, why not always switch?
I feel like this is a detour that we can avoid by adding a small cost to switching.
> I hate the framework that forces me into the confusion by falsely implying that if I get the right envelope EV, I know my EV.
I love it. I love it in the same way I love the trick of a magician who fools me. Also, the answer of "two EVs" is particularly satisfying for me, because it resembles a similar confusion in poker tournaments where you need to account for chip EV and dollar EV separately.
> I feel like this is a detour that we can avoid by adding a small cost to switching.
I don't think its a detour we actually want to avoid; the implications are really fascinating and help us to understand how to solve games more generally.
I agree with adding cost as a way to do it. Actually, that is why I've been saying it is zero or undefined. You know about dynamic programming right? Well, one of the reasons it was invented is kind of related to what we're talking about right now. There are these things called the bellman equations. They can look like this when you think of things in terms of a markov decision process.
Since they're recursively defined, you can compute them more quickly by caching the computation back to front then by computing them front to back. Anyways, putting aside that bit of trivia, do you see the symbol that looks a bit like a y? When we have an infinite sequence like we would if we kept switching you can set that term to be some constant, for example 0.9999999999999. That lets you take the limit, because the infinite sequence is obviously going to converge to zero.
Check out the formula again and notice one of the things that it does which the wikipedia article doesn't. Do you see the symbol for pi? That is talking about the concept of a policy function. A strategy, that way you compute what the agent would get if they played the game, rather than the envelope contents. Under this formalism we've doing an argmax over the policy function for the equation in order to get the policy that has the highest expected value.
> I love it. I love it in the same way I love the trick of a magician who fools me.
If you're actually interested in this sort of thing, I suggest checking out the poker research papers by Noam Brown. He had a talk at NIPS 2017 where he won best paper award. In 2019 he was runner up for best scientific advancement of the year. His work is about applying game simplification to poker to create a bluneprint of the game which is simpler, solving the blueprint using a modified form of counterfactual regret minimization, and then refining the solution during actual play with reach subgame solving. You might have heard of his work, because he was involved in the entire AI now better than humans at poker thing even in no limit breakthrough. I have some belief, but not certainty, that his paper contains a correction of someone who made this style of error. He mentions in the research paper that there was a mistake in another paper where they didn't account for the way subgames influence each other. I see that as the problem here, but I haven't read the other paper, so I don't know if it was the same category of error. Find the paper he referenced, read through it, and see if you're tricked. It is potentially a kind of real world two envelope problem to see if your tools for reasoning about these things is actually helping you to avoid the error in more complicated situations - though, since you disagree with me that this is about handling of imperfect information (or more precisely, the counterfactuals - there is a variant on wikipedia where they removed the probabilites, but counterfactual reasoning still applied to resolve the paradox) maybe you won't see it as related errors.
> You know about dynamic programming right? Well, one of the reasons it was invented is kind of related to what we're talking about right now. There are these things called the bellman equations.
Yes, I've done dynamic programming in competitions. I'm not familiar with bellman equations, and the explanation you provided about convergence, policy functions, etc. went over my head, sorry.
> If you're actually interested in this sort of thing, I suggest checking out the poker research papers by Noam Brown [...] He mentions in the research paper that there was a mistake in another paper where they didn't account for the way subgames influence each other. I see that as the problem here, but I haven't read the other paper, so I don't know if it was the same category of error. Find the paper he referenced, read through it, and see if you're tricked. It is potentially a kind of real world two envelope problem to see if your tools for reasoning about these things is actually helping you to avoid the error in more complicated situations - though, since you disagree with me that this is about handling of imperfect information (or more precisely, the counterfactuals - there is a variant on wikipedia where they removed the probabilites, but counterfactual reasoning still applied to resolve the paradox) maybe you won't see it as related errors.
This sounds really interesting! It was a huge deal in the poker scene when the poker AI developed by Brown and Sandholm defeated pro human players. I read the first few pages of the paper now, but the paper becomes very math-heavy after that. I don't have the necessary background to understand the notation they use. That said, the "mistake" in the way "subgames influence each other" that you referenced, I suspect it was of a far simpler kind - the kind that Brown explains in chapter 2 which he concludes with:
This shows that a player’s optimal strategy in a subgame can depend on the strategies and outcomes in other parts of the game. Thus, one cannot solve a subgame using information about that subgame alone. This is the central challenge of imperfect-information games as opposed to perfect-information games.
Although Brown says "imperfect-information games" here, he actually means a specific type of imperfect-information games: the type where the opponent's strategy is not fixed. We're talking about games where your opponent can change their strategy in order to exploit weaknesses in your strategy. This property was a key requirement of the "coin toss" example game that he provided in chapter 2. If you modified the coin toss game such that the opponent's strategy was fixed, then the situation would change completely. Crucially, Two Envelopes Game is not one of those games where the "opponent" can adapt their strategy according to your strategy. The strategy of the opponent is fixed in Two Envelopes Game. That's why you can solve a single subgame in Two Envelopes game independently of other subgames, even though it's an imperfect-information game. If you are suspectful of this claim, we can verify it by simulations.
> the explanation you provided about convergence, policy functions, etc. went over my head, sorry.
I was agreeing with you when I talked about convergence. You said that adding a cost would solve the problem and I agree that it does. I think you were probably thinking of a literal cost like "one cent". If you choose a cost like that then switching an infinite number of times has an infinite cost. You could instead think of the cost as a fraction of your expectation, the cost is 1% of whatever you end up getting back. Now if you switch an infinite number of times you end up having a cost of zero. That might seem counterintuitive, but recall that 1/3 is .3333 repeating. So when you sum 1/3 + 1/3 + 1/3 you get .9999 repeating. Yet 1/3 + 1/3 + 1/3 is equal to one. Infinitely close to something else is basically being the thing you are infinitely close to. Even though we never get the reward we know that the fraction is becoming infinitely close to zero. People call it "converging" when we have an infinite sequence we can sum to a real value. We know before ever seeing the value that we'll be multiplying it by zero. So we can refactor the equation to be 0*ev(switch) and then take advantage of the identity of 0x=0 to declare the result to be 0. Thus, the calculation converges to zero.
> Although Brown says "imperfect-information games" here, he actually means a specific type of imperfect-information games: the type where the opponent's strategy is not fixed. We're talking about games where your opponent can change their strategy in order to exploit weaknesses in your strategy.
This isn't really the central problem of imperfect information. Consider that in a perfect information game, your opponent will also adjust their strategy to exploit weakness in your strategy. So if it was the central challenge, why does it happen in both? You can rule it out as what he was referring to, because it doesn't discriminate between the two types of games. The central challenge in imperfect information games is you have to play with respect to your information set, not the subgame you are in. So the policies and outcomes of other subgames influences the expected value of the subgame you are in. In perfect information, the only game in your information set is the subgame you are in. So you only have to play with respect to the subgame. That is what makes perfect information different from imperfect information. It discriminates between the two game types.
> This isn't really the central problem of imperfect information. Consider that in a perfect information game, your opponent will also adjust their strategy to exploit weakness in your strategy. So if it was the central challenge, why does it happen in both? You can rule it out as what he was referring to, because it doesn't discriminate between the two types of games.
The type of issue I was referring does not occur in perfect information games.
As a practical example, consider the concept of "balancing your range" in poker. If you play poker without doing that - if you play with a purely exploitative strategy where you are only trying to maximize your EV for each hand - then your strategy will be very easily exploitable by an adaptive opponent. You will frequently end up in river situations where your opponent can deduce whether you have a strong or weak hand, so they can fold to your strong hands and bluff you out of weak hands. In contrast, if you attempt to "balance your range" - that is, consider all the subgames - then you won't end up in these situations as badly. For example, when you make a particular river bet, your opponent might deduce that you have a strong hand 70% of the time and a bluff 30% of the time (as opposed to 100% and 0%).
This issue does not exist in perfect information games. Yes, my definition of "adjusting your strategy to your opponent's strategy" was overly broad to define this issue. But if you think about the poker example, where a poker player will might make a decision like "I need to bluff with this part of my range, because I need to support my strong hands [other subgames] by having some bluffs in my range in this situation" - you won't find a corresponding example from perfect information games like chess. This class of problems is unique to imperfect-information games in which an opponent is allowed to adapt their strategy to yours. If you fix the opponent's strategy, the issue disappears. If you turn the game into a perfect-information game, the issue disappears. Both requirements must be present for this issue to exist.
I thought that Noam Brown was talking about this issue in chapter 2. He discussed a simple coin toss game where one player took a strategy, and then the other player adapted by taking the optimal (exploitative) strategy against them. Then the other player changed their strategy, and the other player again adapted their strategy. And then he described a balanced (GTO) strategy. Then he said this as a conclusion:
> This shows that a player’s optimal strategy in a subgame can depend on the strategies and outcomes in other parts of the game. Thus, one cannot solve a subgame using information about that subgame alone. This is the central challenge of imperfect-information games as opposed to perfect-information games.
I thought that this corresponds perfectly to my poker example. If it doesn't, and it means something completely different, ok, sure. I'm not a mathematician. I can't even read the notation that's used in subsequent part of the paper.
> I thought that this corresponds perfectly to my poker example. If it doesn't, and it means something completely different, ok, sure. I'm not a mathematician. I can't even read the notation that's used in subsequent part of the paper.
I think you understood his point very well. I just think you're making a mistake in trying to recast his claim from "imperfect information games" have this property to "this narrow subset of imperfect information games" has this property. Both imperfect games with an opponent and imperfect games without an opponent have the property of needing to play as if you are in multiple subgames, because the definition of the problem is that you don't know which subgame you are in. His claim was that the policy in one subgame could influence the EV of another subgame - and in this problem, it does. I believe you're thinking of the EV of the envelope when you think you are proving this game doesn't have that property via calculation.
To see his claim applies to this game consider the case where P(Switch)=1. Being able to calculate the EV of the envelope's contents is a bit different from solving the subgame. Here, we have two subgames E1 and E2. We can know a priori what the expected values of envelope's contents in E1 and E2 are. But if you change your policy in E1, it changes your expected value in E2. If you doubt this, remember the core of the paradox again - choose to always switch and your EV is no longer the EV of the envelopes. Ergo, the EV of the subgame E1 is dependent on the strategies and policies of another subgame, E2.
> I think you understood his point very well. I just think you're making a mistake in trying to recast his claim from "imperfect information games" have this property to "this narrow subset of imperfect information games" has this property. Both imperfect games with an opponent and imperfect games without an opponent have the property of needing to play as if you are in multiple subgames, because the definition of the problem is that you don't know which subgame you are in.
If we take the poker example and we modify it by "locking" our opponent's strategy, then this property is removed. Suddenly the optimal strategy for us no longer includes any GTO-like thinking such as "balancing our range", we should simply maximize our EV for each hand "in a vacuum" without any thought to other subgames. We no longer care how our range looks to our opponent, because they are no longer able to change their strategy.
> To see his claim applies to this game consider the case where P(Switch)=1. The policy you chose in one subgame just changed the EV of another subgame.
Sorry, but I don't understand this. This sounds to me like we fix the probability of switching to 1, which means that we end up in infinite loop and the outcome can not be computed. I don't understand this premise, nor its implications for the EV of the other subgame.
> This is a subtle distinction that I mentioned earlier - the EV of the envelope as in the wikipedia problem isn't the EV of the subgame. So you're not solving the subgame if you figure out the EV of the envelope.
The expression you use "EV of the envelope" is ambiguous in this context. I'm not sure if you mean EV relative to the value of the total amount of money in the game, or if you mean EV relative to the value of the firstly-chosen envelope.
When we're talking about "solving a game", we're talking about finding the optimal decisions within a game to maximize the expected value from the game as a whole. In the Two Envelope game we have just one decision: switch or not. So we need to find out if EV(switch) > EV(stay), where both EVs are relative to the total amount of money in the game (not relative to the value of firstly-chosen envelope). We have many ways to conclude that both of these actions have expected value zero. We don't have to be able to compute the "EV of the envelope". We only need to know the relative difference between the EV of these 2 actions. There's many ways of computing them, and all of those ways lead us to the conclusion that the EV of both actions is zero. I'm not aware of any "incorrect" way of computing those EVs such that we would get a nonzero result.
I guarantee you that your solution is going to have to incorporate a set somewhere that includes both subgames. You might forget it does, because you simplify to a scalar, but it is going to be there. In perfect information, it isn't there. In imperfect information it is.
> I guarantee you that your solution is going to have to incorporate a set somewhere that includes both subgames. You might forget it does, because you simplify to a scalar, but it is going to be there. In perfect information, it isn't there. In imperfect information it is.
I'm having a lot of trouble identifying what exactly it is that we disagree about. If you have identified what it is that we disagree on, can you please formulate the disagreement as a wager that can be simulated with code? That way we can easily resolve the disagreement (or conclude that we actually don't have a disagreement).
Your opponents strategy in poker is fixed; they will always play the nash equilibrium strategy, they will never play another strategy. Their strategy is fixed. They will never change it from this setting.
You've claimed that subgame perfect play can be calculated without respect to the subgame you aren't in, because you can make a choice on the basis of the EV of the subgame you are in without respect to the subgames you aren't in.
I disagree. I think you still need to account for every subgame you are in as if you are in all of them.
Let the subgame you are in be you having KK and your opponent having AA. However, obviously - you only know that you have KK.
Therefore, you should be able to compute the strategy which is the best response to 37 suited and according to your logic it should be equal to the best response to AA. After all, you have no means of determining which subgame you are in. So you have to have the same response in both subgames.
So compute the best response for KK to AA and prove that this is also the best response to 37 suited.
However, you've claimed you don't need to calculate this with respect to other subgames. So your computation of 37 suited and your computation for AA must not be equal to each other - if they are, then you share terms. You calculated them with respect to each other.
Let Br = Best response.
Write a program which shows Br(p1, p2, I[KK]) != Br(p1, p2, I[KK]) and Br(p1, p2, I[KK]) = Br(p1, p2, I[KK]) simultaneously. (That is to say, both your policy and your opponents policy are fixed)
My contention is that you can't do this. You claim you can. You are free to use a simpler variant of poker - Kuhn poker - so that the computation becomes more tractable.
Ah, looks like I misunderstood the definition of the term "subgame". I thought that this would be one subgame: "I have KK preflop on the button, and my opponent's range is X". And another subgame might be: "I have 75o preflop on the button, and my opponent's range is X". From my opponent's perspective, these 2 situations would occur on the same level of the game tree and my opponent has no way of distinguishing between these (at this stage of the game tree) due to the hidden information, thus I thought they would be called subgames. But you're telling me that the concept of subgame applies in both directions (not only imperfect information by my opponent's perspective, but also imperfect information in my perspective). So when I say "my opponent's presumed range is X", that sentence doesn't describe 1 subgame, that sentence actually describes multiple subgames.
My earlier point was that if an opponent's strategy is fixed, then I don't have to balance how I play that KK compared to how I play that 75o. That I can play each hand "in a vacuum", in a strategy where the EV of each hand is maximized. Note that this idea is only relevant if my opponent is not playing perfect GTO (contrary to your example). For example, my opponent may have a weakness where their preflop fold frequency is the same regardless of how whether I raise 3BB or 2BB. Against this weakness the optimal play would be to raise 3BB with KK and raise 2BB with 75o. Obviously, this would be a bad strategy if my opponent was allowed to adapt their strategy to me, but it's the optimal play if my opponent's strategy is locked.
To clarify further:
- "exploitative style" poker strategy is made at the level "My hand is 96s, and my opponent's range is presumed to be Y"
- "GTO style" poker strategy is made at the level "My range is X, and my opponent's range is presumed to be Y"
If the opponent's strategy is fixed, then our optimal strategy is to play 100% exploitative. If the opponent's strategy is not fixed, then the optimal strategy will be a mix with elements from both exploitative and GTO strategies. (Again, note that this only makes sense if the opponent is not playing perfect GTO. If the opponent is playing perfect GTO, then obviously the optimal strategy is to also play GTO.)
I thought that Noam Brown was making a point related to this concept. It seems that I was mistaken and his point was related to something else entirely.
> But you're telling me that the concept of subgame applies in both directions (not only imperfect information by my opponent's perspective, but also imperfect information in my perspective). So when I say "my opponent's presumed range is X", that sentence doesn't describe 1 subgame, that sentence actually describes multiple subgames.
Yes; consider a chess move, every move is a subgame - a subtree of that game. He just can't safely say tree, because there are games that are better described as a graph than a tree. Actually, Two Envelope is such a game. The action switch, in Two Envelope, leads to the subgame in which you are at I[null] and need to decide whether to keep or switch - the same situation as you started in. Switch again and you arrive at the subgame I[null]. It contains itself.
> If the opponent's strategy is fixed, then our optimal strategy is to play 100% exploitative. If the opponent's strategy is not fixed, then the optimal strategy will be a mix with elements from both exploitative and GTO strategies. (Again, note that this only makes sense if the opponent is not playing perfect GTO. If the opponent is playing perfect GTO, then obviously the optimal strategy is to also play GTO.)
> That's why you can solve a single subgame in Two Envelopes game independently of other subgames, even though it's an imperfect-information game.
Obviously the EV of envelope 1 is the contents of envelope 1. If you know the probability of reaching it you can calculate the expected value of that envelope by multiplying by the probability of reaching it. But why are you multiplying by 1/2? Probability is defined in terms of sets. What are the set that makes it 1/2? Does that set contain only the subgames that are part of the subgame you are in?
> I'm not a mathematician, so I don't understand expressions like "counterfactual part of subgames emerging" or "there is an identity implied by being under imperfect information". I appreciate you writing back at length, but the majority of what you wrote went straight over my head.
# Defining Counterfactuals
Consider a fair coin flip. You have {Heads, Tails}. Lets assume you are going to get heads, take it as a given - that is what actually happens. It actually happens. It is factual. However, for the purposes of analysis, sometimes it doesn't really matter that we know what happened. We need to consider all the cases that didn't happen. Tails in our analysis would be the counterfactual. These two situations, the factual heads and the counterfactual tails, they're associated with each other. There is a set {HT} that contains both of them. E.g P(head) = |{H}|/|{H, T}|.
When I'm saying counterfactual I'm referring to the events that we didn't assume to be factual, but which we want to keep track of. Probability kind of drops these terms when it says "assume" because it says |{H}|/|{H, T}| becomes |{H}|/|{H}| which is equal to one. This is perfectly fine from a math perspective. The thing is that just like saying 5/x = y lets us move to 5=xy is valid, it has some assumptions built into it. Namely that x can't be zero. Our counterfactuals are a bit like the x, because they offer an easily hidden constraint on what it is valid to do. If you reintroduce uncertainty about whether or not you are in H, you have to do so in a way that takes you back to having the set {H, T}.
Let me show you a practical example of that to make the point very clear:
Let the value of heads be one and the value of tails be zero. P(Head) = 0.5 But assume heads on the same coin flip. P(Head|assumptions) = 1.0 But assume heads again on the same coin flip. P(Head|assumptions) = 1.0. Now since P(Head) is actually true with 1/2 probability: Ev(Head) = 1/2*P(Head|assumption) + 1/2P(Head|assumption)?
Well, it is certainty true that 1/2P(Head|assumption) is a correct term. So you can't say this is wrong on the basis of the assumptions alone. Being very precise, the problem is the neglected counterfactual wasn't handled. Every term leading up to the equation was technically true, but obviously we just did something really weird right? And to be very precise, we neglected the counterfactual associated with heads.
# Defining subgames
Most games can be written out as a game tree. I showed one in my previous post. When you move down the tree, you are in a subgame of that tree.
In a perfect information game like chess if you are in a subgame, a portion lower on the tree, the other parts of the tree don't matter anymore. Your results are independent of the rest of the game tree.
In imperfect information though, just because you are in a subtree doesn't mean you know which subtree you are in. Your actual view into the game is through the information you have. Just like in the {HT} case you have to consider the potential that you have both H or T, in a subgame you have to consider the potential you are in every subgame that is reachable given the information you've seen.
My explanation is correct and I can't say much more to convince you that I've already said.. yours is wrong in precisely why people are confused with the problem and I'm kinda sorry you don't see it.
Since you seem to be making some authority arguments just know that I have a PhD in ML / stats and was top 30 in my country in olympiad level competitive maths so I DO know what I do and in this case with absolute certainty.
Look for the other guys answer if you want more details
> Since you seem to be making some authority arguments just know that I have a PhD in ML / stats and was top 30 in my country in olympiad level competitive maths so I DO know what I do and in this case with absolute certainty.
Let's review what you're claiming with absolute certainty. This is the first claim that I'm contesting:
> they say "one envelope contains 2A and the other A/2".
When pointed out that they actually don't say this, you clarified your point:
> the "writing" doesn't say it but the written formula at bullet point 7 says it. That's the mistake, the formula is wrong and does not describe reality.
The second part of that quote is correct: bullet point 7 has the mistake, the formula is wrong and does not describe reality.
The first part of that quote is incorrect: the written formula at bullet point 7 does not claim that one envelope contains 2A and the other envelope contains A/2. It claims that the first envelope contains A and the second envelope has a 50% probability of containing 2A and a 50% probability of containing A/2.
You correctly identify where the mistake is, but you misidentify what the mistake is.
If you read bullet point 6, it is very clear: "the other envelope contains 2A with probability 1/2 and A/2 with probability 1/2". Notice that bullet point 6 does not claim "One envelope contains 2A and the other envelope contains A/2".
To illustrate my point, consider a simple coin toss game. If the coin comes up heads, you win $2. If it comes up tails, you win $0.50. How might we construct the EV calculation for this coin toss?
0.5 * $2 + 0.5 * $0.50
Notice how both parts of the formula refer to the same coin. They don't refer to different coins. You have one coin that you flip, and depending on how that one coin lands, there is some probability that you get $2, and some probability that you get $0.50. If you looked at that formula, you wouldn't think that the first part refers to different coin that the second part.
Similarly, here we have one envelope (similar to having one coin) and we have uncertainty about the what the envelope contains (similar to having uncertainty about how the coin will land). We have 50% chance that the envelope will contain 2A, and we have 50% chance that the envelope will contain A/2. Thus, we arrive at the (incorrect) formula:
0.5 * 2A + 0.5 * A/2
It's very obvious that both parts of the formula refer to the same envelope, just like in the coin toss formula both parts of the formula refer to the same coin. To be extremely clear, I am not claiming that the formula is correct. I am claiming that both parts of the formula refer to the same envelope, not to different envelopes. To be specific, I am refuting this claim that you made:
> they say "one envelope contains 2*A and the other A/2".
They don't say that.
Furthermore, you make an additional incorrect claim:
> The right formula is changing either the 0.5A by A or the 2A by A.
Here you are just randomly changing the formula in order to make it output 0EV (which is the correct answer). Just because you get a correct answer does not mean your computation was correct. In this case it is incorrect, because A was defined badly. In order to fix the formula you would have to fix the definition of A. If you keep the incorrect definition of A and just shuffle symbols around until you get the correct result, your computation is still incorrect.
> The first part of that quote is incorrect: the written formula at bullet point 7 does not claim that one envelope contains 2A and the other envelope contains A/2. It claims that the first envelope contains A and the second envelope has a 50% probability of containing 2A and a 50% probability of containing A/2.
But it's wrong! Try with literally any A.
> If you read bullet point 6, it is very clear: "the other envelope contains 2A with probability 1/2 and A/2 with probability 1/2". Notice that bullet point 6 does not claim "One envelope contains 2A and the other envelope contains A/2".
Sure but this bullet point is also dead wrong and possibly the start of the scam. If you say A is lowest value, Either it contain A with probability 1/2, or 2A.
Otherwise you are changing the reality of A midway through the sentence.
To meet you halfway, what you may actually want, or see in your head is the possibility that A (in the lowest value sense) may vary in the experiment, and try to compute the expectation of that new problem.
But that's a different problem. And if you want to compute the expectation of this problem with A varying in a range you have to write an integral over the p(A) around the (corrected) equation 7. But the equation 7, even in this integral, need to use the same consistency for A: either you have A being the lowest envelope value and you only write 2A ever, or the highest value and then you only write 0.5A
As I said, I agree that the formula is wrong. I disagree with you regarding where the mistake is and how to fix it.
> Sure but this bullet point is also dead wrong and possibly the start of the scam. If you say A is lowest value, Either it contain A with probability 1/2, or 2A.
I think maybe you have some typos here, because I don't understand what you're trying to say here. In any case our disagreement concerns whether the formula in Wikipedia refers to the same envelope in both parts of the formula, or different envelopes. You kept stating that it refers to different envelopes, and I wrote a long post to refute that. Now you came back saying that the bullet / formula / Wikipedia text is "wrong" and "possibly the start of the scam". Ok, sure, but the EV formula still clearly refers to the same envelope in both parts of the formula, despite the fact that you claimed otherwise with literally "absolute certainty". You were wrong about something that you claimed to know with "absolute certainty", so perhaps you should in the future adjust those estimates downwards with a bit of uncertainty added in?
> Otherwise you are changing the reality of A midway through the sentence.
Nope, A is defined as the "value of the firstly-chosen envelope" in the beginning of the sentence, and A is defined as the "value of the firstly-chosen envelope" in the end of the sentence. The definition for A does not change midway through the sentence.
> But that's a different problem. And if you want to compute the expectation of this problem with A varying in a range you have to write an integral over the p(A) around the (corrected) equation 7. But the equation 7, even in this integral, need to use the same consistency for A: either you have A being the lowest envelope value and you only write 2A ever, or the highest value and then you only write 0.5A
That's not the only way to compute the expectation for this problem. I actually ran some small simulations for this today, representing wagers on this problem. The simulations demonstrate the following claims:
- Steps 1 through 7 in the Wikipedia page are correct, in terms of calculating the "expected value in the other envelope, relative to the value in the firstly-chosen envelope". The expected value for the switch, relative to the firstly-chosen envelope's value, is in fact positive (+25%).
- At the same time, the "expected value for the switch in absolute terms" is zero.
- These claims do not contradict each other.
- The first error in Wikipedia's line of reasoning is step 8 that states "they stand to gain by swapping". This indicates that the player should try to maximize goal "expected value relative to firstly-chosen envelope", which is incorrect. The player should instead maximize goal "expected value in absolute terms".
If you disagree with any of these, let's formulate our disagreement in the form of a wager that can be simulated in code.
I don't think you're wrong exactly, but I just want to try and give you a different way of framing your disagreement.
A man walks to a house with a ladder. He explains that ladders are climbing devices. He tries to set it up against the house, killing plants that he sets it on. The ladder falls over.
One person watches this happen and correctly explains that the ladder is a climbing device and says many very true things about the intentions and proves the genuine thoughtfulness of the ladder users approach. He isn't wrong.
Another person looks at everything from a different perspective. They are a time traveler and they experience the world in reverse chronological order. They say the ladder isn't a climbing device - it is a thing which falls on ground. They say it is a flower killing device. They claim that when the person explains that the ladder was for climbing, they were wrong, because it isn't. They fundamentally disagree about what the ladder is, because they viewed the ladder from a different perspective. He isn't wrong.
They start arguing about what the ladder is. The first person is trying to argue that the ladder isn't a falling device, because it isn't defined to be so. The other person is arguing - this is very critical by the way - not that it is a falling device, but that the way it got used meant it was one and that this disagrees with the statements about what the ladders purpose was. Now, because they aren't noticing this distinction, they argue and they try to prove that the ladder isn't what the other person is claiming it is. Perhaps they think they are winning the argument by pointing out the contradiction, but pointing out the contradiction is agreement that the contradiction exists.
You are using a programmatic understanding of what is happening. You are having to proceed forward from statements in order to get meaning. But there are different paradigms in programming type resolution which might help you understand why you get such sharp disagreement. In programmatic terms you can actually have this backward flow too. b [unknown]; foo(int); foo(b) -> b [int] is implied. You can propagate the type backwards from the function call.
Critically, this is extremely common in math. Mathematics has identities. The relationships between operations flow along those identities. When you let the identities flow backwards, the ladder you two are discussing stops being a ladder. When they make the choice to use a known algorithm, getting the expected value for a subgame under imperfect information, we know the relationships which flow backward from a correct usage. We know what was 'supposed to' be there. They had an incorrect usage. Their ladder wasn't intended to be for falling. But we're experiencing the world in a backwards direction. So we don't know that yet, like you know it. We're treating it as it was used, not as it ought to have been used - flowing what it is backwards from what they mean in the equations, not forward from how they are defined.
So he started trying to explain his thinking to you by moving backward and you noticed that he was saying the ladder was a falling over device, when the ladder maker told you it was something else. And you are right. He isn't wrong so much as his understanding of what the terms are was decided by a different process. And he agrees that you are right that the terms are defined differently!
You aren't really disagreeing about as much as you think you are, because you and him both already agree that the ladder maker said the ladder was for climbing and he said the ladder was for falling over. When you find the contradictions, well, I'm not sure you are actually proving him wrong so much as you are agreeing with each other that a contradiction exists.
Well if debugging is the process of removing bugs from code, then logically the act of writing the code must be called "bugging" and the probability of having a bug in any new piece of code asymptotically approaches 1 with increasing SLOC ;^)
Every program has bugs, every program has inefficiencies. Therefore every program can be eventually reduced down to a single instruction, which will be incorrect.
It's a simple as that. If you know you pick A and then you will switch to B then it's the same as if you had just decided to pick B in the first place.
> you are given two identical envelopes [...] containing money. One contains twice as much as the other
> having chosen an envelope at will, but before inspecting it, you are given the chance to switch envelopes. Should you switch?
> because the person stands to gain twice as much money if they switch, while the only risk is halving what they currently have, a case can be made for switching the envelope
I dig this, but it seems like an unavoidable nonsense pattern that congeals in pools of otherwise practically useful logic / language systems.
Like Neo in the Matrix. Remember the Architect? That guy was cool.
There is a related puzzle which I've also heard called the Two Envelopes Problem. The premise is that you are handed two envelopes. Inside each envelope is a piece of paper with a number written on it. I have written down these numbers by sampling from some probability distribution, but you don't know what it is. All you know is that the two numbers are different. You get to pick one envelope and then look at the paper inside. You then need to guess whether or not the number in the other envelope is larger or smaller than the one you are looking at. Can you guess with better than 50% chance?
Is there actually some way to guess with better than 50% chance?
I'm pretty sure this could be reduced to the secretary problem where there's a pool of 2 candidates, which yields an optimal hire with probability 50%.
Maybe you play word games and say, "I guess the other number is not higher," or "I guess the other number is not lower." Since the number you observed was produced once, there is some non-zero probability that it was produced twice, in which case the other number is neither nigher nor lower.
The reduction is leaky in that it requires you to assume that there is nothing to learn from looking at the first number.
Here's a hint that is not a full solution. If you knew that the numbers in both envelopes were independently sampled from the same Gaussian (but not necessarily which Gaussian), then there is a simple strategy that wins more than 50% of the time: pick an envelope to peek at uniformly at random and guess that it is the larger one if and only if it shows a positive number.
Why does this work? If both numbers are positive, your strategy is equivalent to "pick a random envelope" and you're left with a uniformly random choice of envelope. If both numbers are negative, you're also left with a uniformly random choice of envelope. But if one is positive and the other is negative, you win 100% of the time.
What are the odds of the third case? Well every Gaussian has at least some mass on positive numbers and some mass on negative numbers. So there is some nonzero chance, even if you don't know what it is. So while the strategy might not always do better than a coin toss, some positive percent of the time you win every time. And so you get a distinct advantage over random guessing.
The problem is now how would one transfer such an approach to the original problem without the Gaussian assumption. Somehow the two problems are less different than they initially seem.
I didn't even think of the possibility of negative numbers. I can't ever imagine playing this x, 2x envelope game if there was any chance at all that the numbers would be negative. Like you open an envelope and it says I owe them either one million or two million dollars. Yeah, screw that game.
But couldn't you make the same argument for guessing that it's higher iff the number is greater than 42? Therefore, if the number is 9, both strategies tell you to do different things, yet in each case, your probability of winning is >½. There must be a hole in the logic somewhere…
> But couldn't you make the same argument for guessing that it's higher iff the number is greater than 42?
Yep.
> Therefore, if the number is 9, both strategies tell you to do different things, yet in each case, your probability of winning is >½. There must be a hole in the logic somewhere…
For any nonnegative value C, the probability of Gaussian draw x being larger than (iid) Gaussian draw y is strictly bigger than 1/2 when conditioned solely on x > C.
The particular choice of C might change which specific draws of x and y the strategy succeeds with, but as you noticed for every C a thresholding strategy of the above form does give you some pointwise nonzero advantage over random guessing.
A very general solution, which works regardless of how the numbers in the envelope are picked:
Choose a monotonically increasing function f with values between 0 and 1. Choose one of the envelopes at random. Then look at the number x in the envelope. Then, with probability f(x) say that you have the larger envelope.
I'm inclined to agree with you here. The solutions suggested here seem to depend on heuristics that depend on the type of distribution.
A tangential question: real numbers are uncountably infinite. Are probability distributions over the real numbers likewise uncountably infinite, or do they form a higher infinity?
First, a point of terminology: uncountably infinite refers to any infinite cardinality other than countably infinite, so if the number of probability distributions is at least the cardinality of the real numbers, then it's already uncountable.
As for what uncountable cardinality they form, it's the same as the real numbers [0]. Roughly, a probability distribution is determined by the countable collection of real numbers P(X <= q) with q rational. That means the cardinality is no more than R^Q, which is isomorphic to R. (the cardinality is at least R due to, say, uniform distributions on [0, x] for x real).
If we take any finite interval [0,a], there's a unique uninformed prior of the uniform distribution, and we get a (a-b)/a probability of gaining by switching under that distribution. If we drag a to infinity, we get with probability 1 that switching is better. Numbers are big yo
So, my ONLY guess as to how there can be a strategy is that, if the number you see is positive, you keep it, and if the number you see is negative, you switch. My reasoning is this:
1. If the number you pick is positive, then there are technically (although arbitrarily small) more negative numbers that can be chosen for the other envelope. So you have a slightly larger than 50% chance of winning.
2. The inverse is true if you chose negative, however.
That said - I think this would generally imply you should always switch, which is why I'm not convinced I'm right. Consider a purely positive set, for example. If my number shows up as "12" - then I know there are infinitely more numbers larger than it, but only 11 numbers lower than it. But this starts to feel like the original two envelopes paradox.
--
tldr; I'm not sure of the answer, although I think if I saw the answer I would have an "Aha!" moment.
Let's represent the poorer envelope by the value 0.5 and the more bountiful one by 1.0.
The expected value from the envelope draw is 0.75: 0.5 drawn with a 50% probability contributing 0.25 to the expected value, and 1.0 drawn with a 50% probability contributing another 0.5 to the expected value for a total of 0.75.
If you were to do a large number N of these draws, the value you will obtain will be from a distribution that is centered on 0.75 N. So far so good?
So the way to look at a draw is this: you're going to be getting the expected/average value of 0.75, together with either a 0.25 bonus if you pick the better envelope, or a else -0.25 penalty if you pick the poorer envelope.
So you see, those two situations are equal and opposite. By switching, you either turn a 0.25 bonus into a -0.25 penalty (- 0.5) or vice versa (+ 0.5) with equal probability.
If you need any more complex analysis than this, you need to be thwacked on the head.
The rhetoric about doubling versus halving the money is a distracting red herring. If you switch from the 1.0 envelope to the 0.5 envelope, you're down 0.5, and if you switch the other way, you're up 0.5. That's it.
Accounting is based on debits credits not halving and doubling. You would never update a ledger by crediting double some amount to one account, and debiting half the amount from another account. It's all purely additive.
You have to look at what you're potentially gaining or losing on its own not as a fraction of some guaranteed fixed portion that you're getting from either envelope.
You are missing the point. The problem of whether you should actually switch is obvious and trivial (you should not). The challenge here is explaining why the plausible 'proof' that you should switch is wrong.
I read that in the Wikipedia page. There is some obvious argument involving repeatedly switching sides and whatnot, and it has some subtle logical flaw. Why bother with that at all.
OK, but let's look at it:
1. Denote by A the amount in the player's selected envelope.
2. The probability that A is the smaller amount is 1/2, and that it is the larger amount is also 1/2.
3. The other envelope may contain either 2A or A/2.
OK, here is where this argument got seduced by the multiplicative thinking. The correct reasoning is: let's call the smaller base amount A and the larger one 2A. One envelope contains A, the other one 2A. That's it.
3. If A is the smaller amount, then the other envelope contains 2A.
4. If A is the larger amount, then the other envelope contains A/2.
Here, the argument is splitting into cases because of the IF, and in these two cases, A has a different meaning.
5. Thus the other envelope contains 2A with probability 1/2 and A/2 with probability 1/2.
6. So the expected value of the money in the other envelope is:
This is wonky. The overall two-envelope situation has an expected value, not the individual envelope.
The correct view is that if this is the richer envelope, it contains X more money. If it is the less endowed one, then it contains X less.
1 1 5
- 2A + - (A/2) = - A
2 2 4
7. This is greater than A so, on average, the person reasons that they stand to gain by swapping.
That calculation is wrong though, by itself. That issue should be attacked directly.
Separate calculations have to be performed for the two cases: A is the smaller amount and A is the bigger amount. For each of these cases, the expected value from both envelopes has to be calculated.
The way the above formula is doing the expected value calculation is off because it's equivocating on A.
It ends up working with two unrelated quantities: 2A and A/2. 2A is four times bigger than A/2. A is simultaneously denoting the smaller amount and the larger one.
8. After the switch, denote that content by B and reason in exactly the same manner as above.
Yes, and if you do, you will end up with 5/4B. So that can't be right and the whole remaining argument about infinite switching being irrational compared to just opening an envelope is superfluous.
Because the variable A refers to an amount, has to have a consistent meaning across the two cases, and so it has to be:
1. We have A; the other envelope has 2A; or else
2. We have 2A; and the other envelope has A.
In case 1, if we switch we gain A.
In case 2, if we switch, we lose A.
We have no idea which case we are in.
We cannot use the variable A to denote our initially selected envelope in both cases, because then A simultaneously represents the smaller amount and the larger amount, and we are absurdly mixing the quantities A/2 and 2A in the same calculation. No quantities in the actual problem are related by a factor of four!
People might be seduced into the A equivocation perhaps because they are initially thinking of A as being a label for the envelope, which is arbitrary and innocent enough. We have some envelope A and it has some money in it. Oh, but the other envelope then has either A/2 or 2A. At that point, A is no longer an envelope name, but a name for an amount. Then it has to pin down a specific number: it has to denote either the smaller value, or the larger one, and that value is found in different envelopes in the different cases; A cannot both be "name of initially chosen envelope" and "amount in that envelope".
>The puzzle is to find the flaw in the very compelling line of reasoning above. This includes determining exactly why and under what conditions that step is not correct, in order to be sure not to make this mistake in a more complicated situation where the misstep may not be so obvious. In short, the problem is to solve the paradox. Thus, in particular, the puzzle is not solved by the very simple task of finding another way to calculate the probabilities that does not lead to a contradiction.
I did solve the paradox: it emanates from falsely looking at the switch as doubling or halving the money.
If the envelopes contain, say, $50 and $100, it looks like a doubling/halving. But nothing materially changes if we add $1000 to each one to make $1050 and $1100. The probabilities don't depend on what is in the envelopes, and the win/loss is just a spread of the difference between their values.
Why would anyone, while solving some problem, worry about some unspecified more complicated situation which supposedly resembles that problem?
Give me the specific of that more complicated problem and I will knock it out of the ballpark using whatever reasoning fits that problem.
Only if each value is equally likely. If you see $1,000 but figure the envelope-filler is a lot more likely to have been willing to put $1,500 in than $3,000 then you should stick.
that's an excellent treatment. I'm curious: this video picks a particular example of a distribution of the amounts and shows that the expected profit from switching is not defined. Can one somehow prove that for all possible distributions, either the expected profit from switching is 0 or it is undefined?
the point of the video is that given a distribution, the total expected profit cannot be defined, because the infinite sum of the probabilities of each case don't have an order, and also adds up to positive infinity and negative infinity.
it doesn't really matter what the distribution of the amounts are, as long as there's an infinite number of possibilities in the distribution (ie., it's not a finite amount of possible envelopes).
> as long as there's an infinite number of possibilities in the distribution
how would one prove that claim? it's not obvious to me. it seems like the specific conditions under which the series of terms in the total expectation has a well defined sum is important, but I didn't fully grasp those conditions
For example if instead of halving the probability of each value as he does in the video we take 1/20 then we get the expected value as the sum of 9/20^2 - 9/20^2 + 90/20^3 - 9/20^3 + …
Here the series is absolutely convergent since the positive terms sum to 9/10 and the negative terms sum to 9/10.
I don’t think this explanation is very good. We could use a distribution where the expectation is defined and be back to square one. For example you can just tweak his distribution so that the ratio of envelopes over the ratio of successive probabilities is < 1 (he has 10:2 which is >1).
What an interesting trick. A starting point to intuit a resolution: yes you have a 50% chance of doubling and a 50% of halving. But the doubling only happens if you have x dollars, and the halving only happens if you have 2x. So you can see you either gain x or lose x, so your expected return is 0.
When the amount you multiply your money by depends on the amount of money you currently have, you have to factor in your initial money to each case to compute expected return.
Oh, that actually really helped me getting a grasp on the problem. I feel like other comments and sections of the article were trying to get at that idea, but your phrasing of it made it finally click for me. Thanks.
let M be the maximum amount of money that a envelope can contain (at worst it can be the total amount of money in the world). You don't know the value of this upper bound M.
Most of the time, you get 2*x or x/2 with 50% chance if you choose the other envelope (which on average is a win)
But when x=M you get M/2 instead of M and therefore you loose M/2 with 100% chance. And when it happens you have lost much more than in the other cases (because exponentiel...)
If you do the math, you can see that what you win for 1 2 4 8 16 ... M/2 is equal to what you loose for M.
In fact, the fake/paradoxical solution probably plays on the reader's familiarity with the Monty Hall problem. ("Ah yeah I remember this. Switching is better!")
And this is why we try to hammer into students' head that you can only manipulate series after you've established convergence / that the quantity actually exists in general.
And what is the solution? I would just take the money at point A, i.e. at the very beginning, you never know when the experiment supervisor decides that he has enough of potentially giving even more free money (in case I'm smarter compared to him/her). Plus, free money at moment t is better than potentially double free money at moment t+t1
At some before point memorise enough random generated entropy so that when you enter this situation you combine that with the Schelling point (the left hand envelope for example) to pick one at random.
Then whatever trick is being played you still have a 50-50 chance.
Back when I made videos for YouTube, I did a little video on this paradox. I used a pair of cash boxes instead of envelopes because I thought opening boxes looked better on screen than opening an envelope. I know I didn't explain the issue very well because a lot of commenters thought I was discussing the TV game show "Deal Or No Deal".
> Having chosen an envelope at will, but before inspecting it, you are given the chance to switch envelopes. Should you switch?
Normally we would assume there is someone who owns the envelopes of money. Now their incentive could be to offer to switch only if the envelope you chose contains more money (so they have a chance of keeping more). More like Monty Hall.
Even without that, after switching, should you switch back? The equation given argues you should switch back, so obviously the equation must be wrong.
It is easier to think: if I switch I have a 50% chance of making $X and a 50% chance of losing $X. Switching doesn’t change expected $.
I actually think the later problem in the Wikipedia article is more clever:
Two people, equally rich, meet to compare the contents of their wallets. Each is ignorant of the contents of the two wallets. The game is as follows: whoever has the least money receives the contents of the wallet of the other (in the case where the amounts are equal, nothing happens). One of the two men can reason: "I have the amount A in my wallet. That's the maximum that I could lose. If I win (probability 0.5), the amount that I'll have in my possession at the end of the game will be more than 2A. Therefore the game is favourable to me." The other man can reason in exactly the same way. In fact, by symmetry, the game is fair. Where is the mistake in the reasoning of each man? — "Martin Gardner: Aha! Gotcha"
> Even without that, after switching, should you switch back? The equation given argues you should switch back, so obviously the equation must be wrong.
Ah, that observation is very clever! Even if you let yourself be bamboozled by the logic for switching, there is no reason then not to switch again by using the same logic. And again.. and again!
hmm, I don't really see where the probability of 50% comes from? even if we assume that we don't have any contextual knowledge about what amounts would typically be carried in wallets, the probability of winning would surely have to depend in some way on the amount currently in my wallet? in the extreme case, if I had zero cash, my probability of winning (not just drawing) would be zero, since we can't carry negative amounts of cash.
My bad - then your probability of losing would be zero with zero cash, yes. But we still don't have a proof for the constant probability of 0.5 for winning.
The latter problem is more interesting, although as written they're meeting to compare wallets with this game, so both wallets should logically be empty.
But assuming a more surprise situation in which the game is suggested* the reasoning is pretty good for both if they're happy with a 50% chance at doubling their money + X. But I think maybe its the hidden prize that makes the logic work (or not work)? For one of them the prize is less then double their money (a prize they'd never win but still) so the deal isnt a good deal. Not sure how the math works for that, but a contest where you might win 2a or less then 2a is presumably a different reasoning.
*Where similar to the first point we should assume the suggesters wallet is empty.
The error is that the probability of winning is not 0.5, it’s a function of how much they have in their wallet. They probably don’t know the true distribution so they can’t calculate the actual expected value. But clearly your chance of winning is better if you have $1 than if you have $100000
So add a step: they flip a coin and swap wallets if it comes up heads.
Now what is the probability of each winning? Did adding that step actually change the problem?
Or reword the problem just like I did in my grandparent comment: call the amount in the larger wallet $X. If you lose then you lose $X, and if you win then you win $X. 50% chance and expected outcome $X/2 by playing.
I must admit I don't find the "compelling line of reasoning" all that compelling; to me it seems the "paradox" has more to do with the meaning of probability than with its calculation.
That is, the probability is 1/2. That's all that matters to the decision making. Calculating an "expected value" at all is completely useless, whether or not you do it "correctly".
Calculating an expected value is meaningful and regular and does affect decisions. If you were told that you could pay £1 to flip a coin and heads would pay out 50p and tails would pay out £2, then playing it you would expect to make £1.25 (£0.25 profit) per play, on average. Many people interpret the envelope problem as being the same situation.
> Many people interpret the envelope problem as being the same situation.
It just seems obvious to me that it's not the same situation. That is, I think the "paradox" is ultimately about the temptation to treat these as similar problems in the first place.
Right, the difference here is that the calculation that determines the expected payout is the same as the calculation that determines how much it costs to switch. With the coin, the cost to play is set and is lower than the expected outcome.
I was thinking more along the lines of there being no need to actually calculate anything at all because the only relevant information is the 50-50 probability of choosing the higher envelope. And since you gain no new information after choosing, there's no need to calculate anything afterward either. The "paradox" is in making one think there's something to be calculated beyond the 50-50 chance.
Let's play a game where I wager $10 on each coin toss and you're the casino. Heads, I pay you $5. Tails, you pay me my $10. Just like the envelope game, right? "The only relevant information is the 50-50 probability", right? So if I pay you $1 per toss, you'd be happy to play this game and take my money, right?
No, because you're not actually wagering anything in the envelope game, so it makes no sense to calculate probabilities as though you are.
An analogous coin toss example would be something like: I'll flip a coin, you call heads or tails. If you call it right, I'll give you $10. If you're wrong, I'll give you $5. (So either way you make $5, so I don't know why anyone would even offer this game!) After I flip the coin, I'll conceal it in my hand and give you a chance to change your choice of heads or tails. Well, of course there's no reason to change your choice, it's completely arbitrary! You've got a 50-50 chance of winning $10 regardless. The paradox makes you think you're wagering your unknown potential winnings, but you're not actually wagering anything.
It has nothing to do with calculating whether or not a specific wager is worth it given some set of probabilities. That's a different problem for a different sort of game.
> No, because you're not actually wagering anything in the envelope game, so it makes no sense to calculate probabilities as though you are.
If we consider the decision point where you can either keep the first envelope or switch it, you are effectively wagering the value of the first envelope. You're just muddying the waters when you're trying to make some kind of point about receiving the first envelope for free. Yes yes, you are only gambling your "winnings" that you won earlier, it doesn't make a difference here.
> An analogous coin toss example would be something like: [...] I don't know why anyone would even offer this game!
Look, you made the claim that expected value doesn't matter for anything at all, the only thing that matters is the probability of winning. I offered you a game where you have a 50% probability of winning on each coin toss and I offered to pay you $1 for each coin toss to incentivize you to play it. Can you please explain why you are refusing to play this game? Yes, you don't like the analogy, I get that, but you also don't like free money? I find that hard to believe. A more plausible explanation is that you don't actually believe the claim you are making. That's why you aren't willing to wager any money on it.
> Yes yes, you are only gambling your "winnings" that you won earlier, it doesn't make a difference here.
It makes all the difference because you don't know what those "winnings" are, and what you're potentially wagering them for depends on what they are. So ultimately you're not really wagering anything. (Again, I think this is the crux of the "paradox".)
> Look, you made the claim that expected value doesn't matter for anything at all
I never meant to claim it doesn't matter for "anything at all", just not for this envelope game. (Apologies if the meaning of "at all" was ambiguous in my original post.)
> I never meant to claim [expected value] doesn't matter for "anything at all", just not for this envelope game.
When you say "this envelope game", I assume you are also including minor variations of this game?
If you mean specifically this exact envelope game, where the expected value for all actions is zero, then it is strictly true that it doesn't matter how we make decisions - using expected value or not - though it's not a particularly interesting point to make.
I will proceed assuming you mean also including minor variations of this game.
You wrote this earlier:
> That is, the probability is 1/2. That's all that matters to the decision making. Calculating an "expected value" at all is completely useless, whether or not you do it "correctly".
We can make a minor variation to the game such that the probability will still remain at 1/2, but the expected value for switching can be changed. We can make the minor variation such that it will be profitable to switch, or such that it will be unprofitable to switch. We can do this while the probability remains at 1/2. According to you the probability is all that matters, and it's useless to calculate the expected value for a game like this? This would lead to making unprofitable decisions in a game with a small variation as described above.
> When you say "this envelope game", I assume you are also including minor variations of this game?
No; it seems obvious to me that the argument for switching presented in the original wikipedia article is meant to apply only to the evelope game as it's presented. Of course introducing variations could easily change the meaningfulness of the article's premise. That is, it wouldn't be considered a "problem" or a "paradox" if calculating an "expected value" were actually meaningful.
> then it is strictly true that it doesn't matter how we make decisions - using expected value or not - though it's not a particularly interesting point to make
True, but my point was more of a question: if it's obvious that calculating an "expected value" is irrelevant in this specific case (as the article says, "It may seem obvious that there is no point in switching envelopes as the situation is symmetric"), why is the argument presented in the article considered compelling? That is, either it's not actually compelling, or the "expected value" being meaningless in this case is not necessarily so obvious at first... but if so, why not? (Or, to put it another way, why is the argument in the article compelling enough to warrant such a long wikipedia page with such numerous proposed "resolutions"?)
Your question would be very easy to answer if it concerned a whole basket of "games like this". Now that you narrowed it down to this specific game, it becomes difficult to answer, because we can just throw our hands up in the air and say "it doesn't matter what we do". If we allowed slight variations in the game format, it suddenly _would_ matter what we do, and suddenly we would need to do expected value calculations to do well in these games.
Your question is akin to a driver plowing through red lights without an accident and concluding that the traffic light was useless in this specific instance. "I didn't cause a crash even though I ran a red light, so the traffic light was meaningless! In this specific instance I mean! It didn't matter if I stopped to the red light or ran across it, no accident would have happened either way in these very specific circumstances, so why are people even talking about traffic lights?"
> That is, the probability is 1/2. That's all that matters to the decision making. Calculating an "expected value" at all is completely useless, whether or not you do it "correctly".
I'd be happy to take you up on that offer. Let's play some dice or card games that are set up such that you'll always have 9/10 probability of winning. You'll play, regardless of the expected value, right?
If you treat this as a Bayesian problem, your context and experiences could lead you to specify a prior which has a finite mean, in which case for some values you will have a posterior probability of > 2/3 of it being the larger value.
E.g. if the maximum volume of currency that could fit in an envelope is $10k, it would be possible to have a uniform prior between 0-10k, at which point I would stick if my envelope had $5.01k.
A more extreme prior could be one capped at the largest ever academic research grant (I bet it's not more than $100m).
It could also be that you have a prior that has infinite mean, in which case it is not surprising that you will always want to switch if you draw a specific value.
If you have risk aversion, this further increases the range that you would like to stick, since switching could decrease your expected utility, even if it increases your expected return. There is a separate paradox in decision theory that we should be approximately risk-neural for 'small' amounts.
One mental model I apply to this and other similar problems is tracking what 'universe' you're in. In the flawed solution, the problem is that there's a universe in which the amounts are A and 2A and a universe in which the amounts are A and A/2, but they aren't the same universe, and the two possibilities will have different values for A and so you can't easily average A/2 from one universe with 2A from the other.
(The usual term is sample space; it's just more fun to split and eliminate universes than states.)
This line of reasoning isn't possible if you have specific amounts (eg £50 and £100) for the envelopes, which suggests that the variable A is being misused somehow.
`B = (2A if A=50, A/2 if A=100)`. Simplifying this to `B = 2A or A/2` loses important information: namely that when B is smaller, you expect A to be larger. Or alternatively, treating A as fixed (say A=100) conflates two different situations: one where the amounts are `100, 200` and one where they are `100, 50`. So you end up thinking B must have (200+50)/2 in it, which is incoherent. If you compute the expected B correctly then `A` is different in each branch.
It is subtle though. If this appeared in a paper, arguing something not obviously wrong, it'd be hard to convince everyone that the reasoning is bad.
TL;DR (Spoiler alert): the expected value of the amount of money you end up with is an infinite series whose sum changes depending on the order in which you add up the terms, and so you can choose an order that makes this value come out to be positive, negative, or zero.
Right away I was taken aback by the wikipedia article’s quick dive into mathematical solutions to the problem, when it seemed to me that what would be missing is consideration/framing of the initial setup.
That initial setup being, the act of starting the game with someone is(?) confers value in and of itself.
I do not like this explanation. In my opinion, the fancy mathematical argument involving infinite series is an unnecessary distraction from a much more fundamental and mundane mistake (i.e. incorrectly using a variable out of its scope).
When E[X], E[A], and E[X - A] are all well-defined, it is indeed the case that if E[X | A = a] > a for every particular value a, then E[X] > E[A].
What goes awry in this case is that E[X - A] is not well-defined (where X is the value in the unselected envelope and A is the value in the selected envelope). It is given by a conditionally convergent series, as noted.
Right. I mean the original, paradoxical line of reasoning – "the switching argument".
I see that the article also says "commonly one writer proposes a solution to the problem as stated, after which another writer shows that altering the problem slightly revives the paradox." But it doesn't seem to elaborate on where the simple resolution falls down, if it does.
It depends on how the envelopes are prepared. If the two envelopes have x and 2x in them and one is randomly handed to you, there's a symmetry between them that tells you there is no benefit in switching.
However, suppose x is placed in an envelope and handed to you. Then a fair coin is flipped. If it comes up heads, 2x is placed in another envelope and if it comes up tail, x/2 is placed in that envelope. Then you should switch.
It's a subtle difference in how the envelopes are prepared, but it makes all the difference.
The introduction of a second "stage" based on your action changes things in a non-obvious way. Your first action is less impactful than your second one, regardless of what you did.
At the end I constructed a super weird example where regardless of what you see in your envelope, you should switch (this seems to go against the symmetry argument, but is sort of saved by a divergence in the expectation value).
Once you pick an envelope, you no longer stand to only gain money.
You can lose money and that has to be reflected in the potential value of each envelope.
After the first selection, you must express the envelope value as the potential of what each envelope holds (the probabilities from the initial selection) which makes selecting again a wash.
Let A = 50
Envelope 1 is 100
Envelope 2 is 25
First selection
1/2(100) + 1/2(25) = 62.5
Great! Do it! 62.5 is bigger than 0, which is the expected value of not playing at all.
After that, it makes no difference to switch (the envelope value is recursive):
and the trick is that’s now the potential expected value you have (you now have 5/4A in your envelope), so switching is a wash. You never really “had” A as a value to compare against (by evaluating that 5/4A is bigger than A), we just get tripped up with the doubling and halving at the outset.
Said another way, the fact that 5/4A is bigger than A is irrelevant, no envelope contains A, they both contain the expected value of 5/4A.
No. The problem specifies that E1 is twice E2, but here you have E1 = 4 x E2.
A is the amount in one of the envelopes, so if A=50 then either E1 is 50 or E2 is 50, and the other E is 25 or 100. But under no circumstances can E1=100 and E2=25 at the same time.
Impossible. Schrodinger’s envelope, then. You can’t reflect the probability to be 2A AND 1/2A if A represents the value of an envelope.
The value of the other envelope must include the probability that it is the original A (not some sleight of hand new A’ that, itself, is based on an expected value).
> You can’t reflect the probability to be 2A AND 1/2A if A represents the value of an envelope.
Yes, that's right. That's exactly what I said: "A is the amount in one of the envelopes, so if A=50 then either E1 is 50 or E2 is 50, and the other E is 25 or 100. But under no circumstances can E1=100 and E2=25 at the same time."
317 comments
[ 0.21 ms ] story [ 1104 ms ] threadSee https://en.wikipedia.org/wiki/Monty_Hall_problem
https://dungeonsdragons.fandom.com/wiki/Monty_Haul
https://tvtropes.org/pmwiki/pmwiki.php/Main/MontyHaul
Or to put it another way how many people would stick with their chosen door if instead of opening a door and showing a goat they were simply asked "would you like to stick with your current choice of door, or switch and take the best result from the other doors?"
That is, effectively, what the choice boils down to. But that being said, since the problem is stated as only getting to pick one "other" door, the "extra information" received is knowing which of the other 2 doors has the best result.
2/3rds of the time, your initial choice will be a goat. In those cases the host deliberately avoids opening the remaining door with a car - thereby telling you where it is.
If you chose the car door initially, then indeed you get no new information. But that only happens 1/3rd of the time.
I am curious if that changes your opinion.
It's true that we can add other options (pick both doors) of equal likelihood, but it's because of our increased information that the reduced option of picking just one door isn't actually worse off.
I have a deck of a billion cards, one entitles you to the car, the others have a picture of a goat. You pick one at random but do not get to see it.
Of the remaining 999,999,999 cards, I discard 999,999,998 goats.
Do you want your initial card, or the remaining card in my deck?
Or to frame it another way, do you want the car iff you picked it first or iff you did not pick it first?
The way they put it, the strategy becomes obvious if you instead reformulate the problem like so: there are three doors with three different prizes. You get to pick one door, but are then given a choice: stick with your original choice, or choose to get the best prize from the other two doors.
The solution (switch!) is more obvious in the second formulation. I have no idea if it's easier to convince someone else of the reasoning in the first formulation, or to convince them that the second formulation is equivalent to the original.
you make your guess Me: Ok, I will reveal that the number is either $YOUR_GUESS or 452,871. You may keep your original guess or switch to 452,871.
Of course, you would switch, because the chances you guessed correctly originally were 1 in a million.
Personally, I'm not sure that it would be easier to convince someone that the second formulation is equivalent to the original problem, but next time I find a dis-believer I will give it a shot!
This problem is completely different, since the number of degrees of freedom remains fixed throughout the decision process. As someone else pointed out, it is trivially resolvable by realizing that no matter how many times you switch envelopes, you're really only making one choice, and doing so with 50-50 odds of picking the more profitable envelope. The offer to switch amounts to meaningless magician's patter, whether the offer is made once or a million times.
> Pigeons repeatedly exposed to the problem show that they rapidly learn to always switch, unlike humans.
The case where switching gives you 2A and the case where switching gives you A/2 describe 2 completely different universes, not two different actions in one.
Take the practical example of $100 and $200: A is either $100 or $200 depending on which envelope you receive first, so the equation is either 0.5*A + 0.5*2A, or it is 0.5*A + 0.5*(A/2). It can never be 0.5*2A + 0.5*(A/2)
The thing that helped me reconcile this was realizing that the A in the two equations are different values; then replacing them.
^ but the A in the left grouping (where it's 100) is different than the A in the right grouping (where it's 200).Replacing the As with their actual values
And, since one envelope has 100 and the other has 200, an expected outcome across both envelopes is, as calculated, 150.> If A is the smaller amount, then the other envelope contains 2A. > If A is the larger amount, then the other envelope contains A/2.
These are conditional probabilities so you can't simply add them up and compute an expected value of switching like they show in the example.
The big problem is that you can't have a uniform distribution on the natural numbers.
(And by extension, you can't have a uniform distribution on the rounded-to-integer version of a distribution on the real numbers.)
Imagine we have a distribution that satisfies that assumption, and then someone tells you they've sampled from that distribution and found that the result is of the form (say) 7*2^k, for some k > 0. That conditional distribution for k would seem to have to be uniform, right?
Of course, the lack of a uniform distribution on the reals or rationals presents the same problem.
Well, you can either assume that the envelope's numbers are specified only up to pennies. Or you can just round and say you don't care about sub-pennies.
Or you can go with your suggestion, and look at the lack of a uniform distribution over the reals / rationals.
But the "possible values" they put in are A/2 and 2A, which makes no sense. A is a random variable.
Don't forget that random variables are really functions on a probability space with an implicit argument, so (1) when multiple random variables appear in a formula, they're implicitly evaluated at the same argument omega, and the formula must hold for all omega from the probability space, and more importantly, (2) there can't be free-standing omegas in a formula for expectation because expectation is an integral (or sum in this case where the probability space is discrete) over all possible omegas.
In my opinion, the Wikipedia article is making a disservice to readers by mentioning unnecessarily complex mathematical arguments involving bayesian reasoning and infinite distributions. I believe all of these are distractions from the more fundamental typing error that invalidates the switching argument.
> in particular, the puzzle is not solved by the very simple task of finding another way to calculate the probabilities that does not lead to a contradiction
The reason the calculations are wrong are fundamentally related to why correcting the calculations leads to the correct probabilities. If you understand why the problem gives you the wrong result, you proceed to correcting it, and you get the right result - that doesn't mean you didn't understand. It means you did. You can tell you did, because you have the correct answer.
But if I show the calculations produce the correct answers, well, that is a step too far. We can now dismiss the understanding on the basis that it got the correct answer, apparently?
It's no wonder the author of the wiki writes such a bullshit claim - that no one can agree on a definitive conclusion. Their terms of engagement are self-defeating. Correctness is error, because correctness means you didn't /really/ understand. Its a no true scottsman fallacy - get the right answer, and you aren't engaging with the 'real' problem.
But its a flawed no true scottsman, because it defines something measurable: it tells us the goal, that it is to avoid this type of mistake in our thinking. And generalized algorithms for solving decisions problems that include this as a case which is successfully solved are many - and they just so happen to be in the imperfect information setting. And that setting has studied the problem of infinite recursion and has solutions for them. Which I can apply. To get the right answer. And, in general, avoid the problems they claim we aim to avoid.
More importantly and to the point of this thread - this isn't an intractable debate, because its been solved and used in production settings for literally decades. So what if some people are going to pretend it isn't? This isn't an unsolved problem. The actual game theory math is /well/ beyond this level of complexity. Its contending with things like environments where you have so much complexity you have to reduce to a blueprint abstraction, not stumbling at a decision problem that is quite literally simpler than rock paper scissors.
You can arrive at the correct conclusion either by pinpointing exactly where the original argument is incorrect, or you can come up with a completely different argument that does not have an error.
The puzzle challenges you to pinpoint the error because coming up with the correct solution is trivial (and the puzzle is deliberately set up this way).
This is not a "nonsense goal." If this came up in real mathematical research - two papers coming to contradictory conclusions - and no one could find where either paper went wrong, we would have a real paradox on our hands.
I think the paradox is that the algorithm equates the expected value of the contents of an envelope with the expected value of a policy choice for a player. When I correct what I feel is the root of the paradox, my solution drastically differs in fundamental ways such that the way the problem restricts to pointing out the wrong step feels disingenuous.
The entire structure is wrong, because even if you do correct the error that leads to the wrong EV for the envelope, you still haven't resolved the paradox. The right probabilities don't resolve the paradox, because they still imply that always switching has the same EV as not switching. If they were really equal, I could always choose switch, but I can't - so the paradox is still there.
My resolution ends up being so critical of their argument that the entire way they go about solving gets thrown out. I end up seeing, not just a specific wrong EV calculation, but a decision problem that is just fundamentally using an inappropriate algorithm to determine the policy function.
We can agree that "the entire structure is wrong" because the "entire structure" is giving a wrong formula for the EV and saying "this is the formula for the EV."
Yes, switching and not switching have the same EV and you can always switch.
To try and stress to you how big a problem this is, pretend you were playing the game and you wanted to find the right move that was going to make your EV the highest. Now, if you go with the logic of the problem, you are allowed to select not on the basis of your EV, but on the basis of some other EV. So instead of choosing the envelope, why not choose something else that has no relationship with our EV? Say, the weather in Alaska. If it is sunny, we like sunny. So switch. If it isn't sunny, we don't like that. So keep. It is crazy to do this, because there is no relationship between the EV you are using as a selection criteria and the EV your policy gets. This is the same situation as using the EV of the envelope. It sounds really crazy when you use the Alaska example, because it is so obviously unrelated. It sound so reasonable when you use the envelope example, because it isn't as obvious that they are unrelated. Yet for the policy of P(switch)=1, the ev of the envelope and the ev of the policy with respect to the game are not the same thing.
Now imagine the wikipedia article for the Alaska problem variant of the two envelope problem. Do you really think everyone would be so focused on the EV of the envelope as the step that was wrong? How could they? We have the same paradox still, but there is no EV calculation for the envelope included in the problem. If we can remove the EV calculation, yet still have the same paradox, it seems to me the paradox is not the expected value calculation.
So what is my solution? Well, to actually find your correct policy function you need to get the argmax of the policy with respect to the game. There are multiple ways to do this:
- Reinforcement learning does it by finding argmax pi with respect to Q_pi(s, a) = R(s') + P(keep)Q_pi(s',keep) + P(switch) Q_pi(s', switch).
- Game theory sets it up a bit differently. You define a similar graph using a different formalism, but simplified to operate over information sets. You can use a thing called regret matching; basically it turns out that if you play in proportion your normalized counterfactual regret, the average of those policies is the optimal best response.
In both cases you need to do something about the fact you're actually on an infinite graph. So the actual solution in the general case looks very very different from their way of solving the problem. It isn't just simple probability; I mean, it is, but taking the limit of an infinite sequence and taking advantage of the properties of markov chains aren't usually what I think of when someone tells me that something is simple probability. That is one formalism. In the other, we do have simple probability, but it isn't necessarily obvious that the central limit theorem gives us the optimal policy when we play in proportion to not just our regret, but our counterfactual regret. So yes, simple probability, but also, most people who know simple probability don't necessarily even know what a counterfactual is. So maybe not that simple after all.
But lets say we stick to the problem. We are here to learn how to avoid this problem, right? Nope. If you don't do things like this, you'll just be wrong in more complicated situations. Because the EV of the envelope is not the EV of the game with respect to your policy. This gets increasingly true as your imperfect information ga...
We aren't in agreement about this. I realize we have to fix this, but fixing it doesn't resolve the paradox. It is a red herring.
> Yes, switching and not switching have the same EV and you can always switch.
With all due respect, this isn't true and asserting this doesn't resolve the paradox. See my other reply for why it doesn't resolve the paradox.
But if only look at the formulat at bullet point 7 there "2A" and "0.5A" as amounts but not "A". It just doesn't describe the current state of reality
not trying to brag, just reformulating the simplest way for everyone to see
It's two versions of the reality in the same equation, which is wrong.
Use 100 usd / 200 usd instead of A and 2A and you'll see. If you never switch and if you always switch in both case the equation is
100×0.5 + 200x0.5
The problem is that they interchange A being the lowest and the highest in the same equation.
The right formula is changing either the 0.5A by A or the 2A by A.
Try with A being a real like 100 usd numbers to see it.
Basically you can't have "two realities" in the same equation. U havé to pick one.
No it's not, that's not how you make EV calculations. As I already said in my previous post, the parts in the EV calculation do NOT refer to the pair of envelopes. Both parts refer to the same object, not to different objects. You are trying to change the EV calculation such that the different parts refer to different envelopes. This does not make any sense.
> Try with A being a real like 100 usd numbers to see it.
...and that's not a way to prove / demonstrate the correctness of math. Yes, if you change these numbers in this way, you get EV 0, which is the correct answer. Just because the answer is correct does not mean the calculation is correct. In this case it isn't.
On Wikipedia they created two trees one with 2A.
And another with 1/2A Notice now that there is no game in which A is both 2A and A/2.If you don't think this is the error, you're not thinking in imperfect information terms. You're thinking in perfect information terms. This sort of splitting action is allowed in perfect information games! In imperfect information games, you are in multiple subgames at the same time. A has to be the same A in both subgames, because you can't discriminate between which game you are in. So you're not allowed to do this. It is wrong!
This gets at the core of what the question is trying to make people notice. You can't do this with imperfect information. You can only do it with perfect information. You can't do it over information sets and strategy spaces. You can only do it over states and action spaces.
But they do it. And then they combine them back into the same analysis when they say 1/2(Game tree 1 switch case) + 1/2 (Game tree 2 keep case). And you can't do that, because game tree one depends on game tree ones keep case and game tree 2 depends on its switch case! You can't tell which you are in, so you are in both - not one or the other. You can't combine the two different subgames from two different trees like you could in perfect information! You have to stick with the same game tree.
And that is what the person you are mistakenly correcting is saying. Pick one or the other, you can't have both.
PS: There are other errors in the problem, like that when you solve it with an MDP or the Bellman equations you actually /do/ end up with 100% Switch having an expected value of zero. So if you spotted some other error, that doesn't mean that the person you are correcting is wrong in their correction. There are multiple issues with the way the wikipedia article approaches the conclusion.
PPS: This exact confusion that people are talking about is actually something that happened in practice in the solving of imperfect information games! IIRC the Pluribus paper author corrected a reasoning error of this type when he introduced sub game reach solving? I might be misremembering here, since its been a while since I listened to his talk, but my fuzzy memory suggests that confusion about subgames interacting with each other was the root cause of a failure in some of the poker research for subgame solving in a blueprint abstraction.
This game tree only makes sense if you change the definition of A. In particular, you need to change the definition to "A = 1/3 of the total amount of money in the envelopes". If you define A like this, then your game tree is correct, and it corresponds to the "simple resolution" described on the Wikipedia page (where they use x instead of A, for clarity). This is an unsatisfactory answer, because it provides an "alternate path" to the correct answer, without demonstrating which step went wrong in the "incorrect path" that leads to the wrong answer. It is not particularly difficult to construct these "alternate paths" that lead to the correct answer. The difficulty of the puzzle is in demonstrating what goes wrong in the incorrect path, as stated on the Wikipedia page:
The puzzle is to find the flaw in the very compelling line of reasoning above. This includes determining exactly why and under what conditions that step is not correct, in order to be sure not to make this mistake in a more complicated situation where the misstep may not be so obvious. In short, the problem is to solve the paradox. Thus, in particular, the puzzle is not solved by the very simple task of finding another way to calculate the probabilities that does not lead to a contradiction.
> Notice now that there is no game in which A is both 2A and A/2.
I have no idea what you are referring to here. I never claimed that A = 2A in any circumstance (there is nonzero amount of money in the envelopes).
> If you don't think this is the error, you're not thinking in imperfect information terms. You're thinking in perfect information terms. This sort of splitting action is allowed in perfect information games! In imperfect information games, you are in multiple subgames at the same time. A has to be the same A in both subgames, because you can't discriminate between which game you are in. So you're not allowed to do this. It is wrong!
I agree with you that the crux of the issue is in the definition of A. In particular, "mixing" "two different A's" within the same computation, even though they sort of mean different things. This is a fuzzy explanation that gives a feeling "something about this explains the issue", but it's not a complete explanation that precisely pinpoints what is wrong.
> This gets at the core of what the question is trying to make people notice. You can't do this with imperfect information. You can only do it with perfect information.
This is false. We can easily construct imperfect information games similar to this one, but tweak a few things in such a way that a "naively constructed EV calculation" yields the correct results.
> And that is what the person you are mistakenly correcting is saying. Pick one or the other, you can't have both.
Nope! I was very clear about which very specific claims I was refuting. They were unrelated to the topics you are bringing up here. I posted a long, clarifying comment to a sibling comment here, you can look that up if you want to refute specific claims.
> So if you spotted some other error, that doesn't mean that the person you are correcting is wrong in their correction. There are multiple issues with the way the wikipedia article approaches the conclusion.
I didn't say "I spotted error X in Wikipedia which explains Two Envelope problem", I said "I spotted error X in this other poster's Hacker News comment". Please read the very specific comments I made about very specific claims made by the person.
Switching away from using A to avoid ambiguity.
> This is a fuzzy explanation that gives a feeling "something about this explains the issue", but it's not a complete explanation that precisely pinpoints what is wrong.
Yes it does, actually. Well, to me. I'll go more in depth so it less fuzzy for you.
The issue is that when you handle imperfect information correctly you have counterfactuals implied by your state-oriented reasoning, because you can't actually escape into perfect information world - you're not in it. When you combine the state-derived probabilities the - I don't know the word for this, so forgive me, but the - superposition of the two counterfactuals implied by those state-derived probabilities end up occupying the same space. They exist, because they are implied by the imperfect information game. They just aren't writing them out in the equations on wikipedia.
Edit: My 'spatial' probabilistic reasoning in my head is explained here, but I show you the identity rule that proves this in the next section.
> I never claimed that A = 2A
You aren't, but the counterfactual terms that get realized into the same position, but which weren't written out, claim that Z = 2Z and that Z = Z/2. The two cases were under two different games. So their implied counterfactual components don't agree with each other and we should never have been allowed to combine them.
We agree on this!
You're just more focused on the they come from different games part and not noticing the other part of what I'm telling you: you can equate the counterfactual components that they don't show between the two games, because there is an identity implied by being under imperfect information. Look at the set relationship closely across the two subgames and you'll realize that the set is defined to be equal to itself across all subgames where the set is used.
That is quite literally one of the logical contradictions. This is where they claim that 2=1. The sets aren't equal to each other after their operations. So by the identity implied by the set, they've claimed 2=1.
And yet when I try to claim this is a core aspect you throw out this...
> This is false. We can easily construct imperfect information games similar to this one, but tweak a few things in such a way that a "naively constructed EV calculation" yields the correct results.
This is a really dumb nitpick with very poor support. If you're going to be this fallacious, you might as well go all out. Why not say I'm wrong because first graders learning to add numbers aren't learning how to handle imperfect information subgames? After all, your argument for me being wrong is that the pedagogical value of problems is independent of the subproblems they contain.
> Nope! I was very clear about which very specific claims I was refuting.
You have a more subtle point than I thought you were making. I'll check out your other post.
> I didn't say "I spotted error X in Wikipedia which explains Two Envelope problem", I said "I spotted error X in this other poster's Hacker News comment". Please read the very specific comments I made about very specific claims made by the person.
Well, he does have another error; EV isn't defined over all policy choices. It is a problem on many of the wikipedia solutions too. He does see it - its why he mentioned other people talking about infinite series - but his correction attempt was more like "halt here because of type error" which is something that I as a programmer can respect.
> This is an unsatisfactory answer, because it provides an "alternate path" to the correct answer, without demonstrating which step went wrong in the "incorrect path" that leads to the wrong answer.
Eh, I mean, I guess I'm cheating by claiming an interpretation they aren't trying to use. This is why I detest the "no true scottsman" setup to the question. We can create an identity that maps their wrong step into my formalism where what they did is wrong in the way I claim it is. Even though they didn't "try to do it" doesn't mean I can't see that they did do it. But apparently - even though game theory notation neatly avoids these pitfalls - we need to stick to the footgun notation. It just seems stupid to me. If you want to avoid the problem, use the notation that trivializes avoiding the error. I really like the other person's analogy to type errors, because it is such a similar idea to what I'm saying, but they just use a different part of math to assert it. There are ways you can just make this decision fail to typecheck. If you don't want to have these type of errors? Be stricter about your typing so at to prevent them.
Nobody makes this claim, directly or indirectly. If you got this result by formalizing the plain English statement into a proof assistant, then the error was introduced during the step where you interpret the plain English into a formal statement. If the claim "Z = 2Z" was inherent to the plain English version of the problem, then it wouldn't be possible to run the wager as a simulation, but it is. The entire Two Envelopes problem - including step 7 - is possible to simulate in code: https://pastebin.com/jPyZVrkx
What the simulation demonstrates is:
- It's perfectly possible to define "expected value of the final envelope in relation to the value of the firstly-chosen envelope", as the Wikipedia page describes. No contradiction exists that would prevent this computation.
- It's also perfectly possible to define "expected value of the final envelope in absolute terms".
- These are different goals to maximize. A rational actor should maximize goal 2 ("expected value of the final envelope in absolute terms") in this variant of the game. It's also possible to construct another variant where a rational actor should NOT maximize goal 2, but should instead maximize goal 1 ("expected value of the final envelope in relation to the value of the firstly-chosen envelope").
- The error in Wikipedia's "compelling line of reasoning" appears to be at step 8 where they conclude that "they stand to gain by swapping". This implies that a rational actor should maximize goal 1, when in fact a rational actor should maximize goal 2 instead. This is the error in Wikipedia's line of reasoning.
> When we get to step seven we do a calculation that reintroduces our uncertainty. We multiply by the probability of being in each case. However, we don't actually know we are in one case and we don't know we are in the other case. So combining these two situations leads to the counterfactual part of subgames emerging. So we have {A, 2A} and {A/2, A}. The sets aren't equal. We can choose some parts of this, like grabbing A and comparing it to 2A. So now we have an identity rule that just let us say 2A=A. 2=1. Our contradiction is found.
I'm not a mathematician, so I don't understand expressions like "counterfactual part of subgames emerging" or "there is an identity implied by being under imperfect information". I appreciate you writing back at length, but the majority of what you wrote went straight over my head. The impression I have is that you formalized the problem in a manner that lead to a contradiction. If this contradiction was inherent to the original English problem statement, it wouldn't have been possible for me to simulate the problem. But it is. So it seems to me that there is no inherent contradiction in the English problem statement, it seems that the contradiction was introduced during the formalization step.
> I detest the "no true scottsman" setup to the question [...] If you want to avoid the problem, use the notation that trivializes avoiding the error.
I wouldn't describe the setup as a "no true scottsman" scenario, because the setup describes what counts as a scottsman: determine which step in the line of reasoning is incorrect, why and under what conditions. I appreciate that you tried to do this when you identified step 7 and explained what was wrong with it in your opinion. This is also what I tried to do in my explanation, with regards to step 8 and the comparison between the two goals.
Besides, it's very trivial to conclude that nothing can possibly be gained by switching (in absolute money terms, which should be the goal a rational actor chooses to maximize). It would be boring and...
Well, this is just a place where they do the math wrong. We can quibble about why, but it isn't a paradox. We both agree they do it wrong and we even have an overlap in the reason why - we both think they aren't comparing them relative to all the subgames they are in and we both think that this is required to reason correctly. So lets say we get to the correct relative EV of 0.
If both are worth the same, well, why not always switch? They are the same EV, right?
IMO this is the real paradox. We have a false equivocation. EV(policy, game) is not equal to EV(envelopes), because for the policy `ALWAYS_SWITCH` we get the logical contradiction that undefined = EV(envelope) or if you take the limit with a discount under a different formalism that 0 = EV(envelope)
When you tackle this problem starting there rather than at the other error the entire algorithm changes, because we need to modify the problem so that we can calculate EV with respect to policy. The algorithm changes enough so that it really annoys me, because I have this intuitive feeling that I'm changing the solution so much so that I'm no longer working within the spirit of the puzzle.
That is why I feel like "no true scottsman"; to use a metaphor, they gave me a coloring book and told me to color something beautiful but that I had to stay in the lines, but the lines of are a skunk and I want to draw a sunrise. I don't want to correct just the one particular step. I hate the framework that forces me into the confusion by falsely implying that if I get the right envelope EV, I know my EV.
I feel like this is a detour that we can avoid by adding a small cost to switching.
> I hate the framework that forces me into the confusion by falsely implying that if I get the right envelope EV, I know my EV.
I love it. I love it in the same way I love the trick of a magician who fools me. Also, the answer of "two EVs" is particularly satisfying for me, because it resembles a similar confusion in poker tournaments where you need to account for chip EV and dollar EV separately.
I don't think its a detour we actually want to avoid; the implications are really fascinating and help us to understand how to solve games more generally.
I agree with adding cost as a way to do it. Actually, that is why I've been saying it is zero or undefined. You know about dynamic programming right? Well, one of the reasons it was invented is kind of related to what we're talking about right now. There are these things called the bellman equations. They can look like this when you think of things in terms of a markov decision process.
https://wikimedia.org/api/rest_v1/media/math/render/svg/0e35...
Since they're recursively defined, you can compute them more quickly by caching the computation back to front then by computing them front to back. Anyways, putting aside that bit of trivia, do you see the symbol that looks a bit like a y? When we have an infinite sequence like we would if we kept switching you can set that term to be some constant, for example 0.9999999999999. That lets you take the limit, because the infinite sequence is obviously going to converge to zero.
Check out the formula again and notice one of the things that it does which the wikipedia article doesn't. Do you see the symbol for pi? That is talking about the concept of a policy function. A strategy, that way you compute what the agent would get if they played the game, rather than the envelope contents. Under this formalism we've doing an argmax over the policy function for the equation in order to get the policy that has the highest expected value.
> I love it. I love it in the same way I love the trick of a magician who fools me.
If you're actually interested in this sort of thing, I suggest checking out the poker research papers by Noam Brown. He had a talk at NIPS 2017 where he won best paper award. In 2019 he was runner up for best scientific advancement of the year. His work is about applying game simplification to poker to create a bluneprint of the game which is simpler, solving the blueprint using a modified form of counterfactual regret minimization, and then refining the solution during actual play with reach subgame solving. You might have heard of his work, because he was involved in the entire AI now better than humans at poker thing even in no limit breakthrough. I have some belief, but not certainty, that his paper contains a correction of someone who made this style of error. He mentions in the research paper that there was a mistake in another paper where they didn't account for the way subgames influence each other. I see that as the problem here, but I haven't read the other paper, so I don't know if it was the same category of error. Find the paper he referenced, read through it, and see if you're tricked. It is potentially a kind of real world two envelope problem to see if your tools for reasoning about these things is actually helping you to avoid the error in more complicated situations - though, since you disagree with me that this is about handling of imperfect information (or more precisely, the counterfactuals - there is a variant on wikipedia where they removed the probabilites, but counterfactual reasoning still applied to resolve the paradox) maybe you won't see it as related errors.
Yes, I've done dynamic programming in competitions. I'm not familiar with bellman equations, and the explanation you provided about convergence, policy functions, etc. went over my head, sorry.
> If you're actually interested in this sort of thing, I suggest checking out the poker research papers by Noam Brown [...] He mentions in the research paper that there was a mistake in another paper where they didn't account for the way subgames influence each other. I see that as the problem here, but I haven't read the other paper, so I don't know if it was the same category of error. Find the paper he referenced, read through it, and see if you're tricked. It is potentially a kind of real world two envelope problem to see if your tools for reasoning about these things is actually helping you to avoid the error in more complicated situations - though, since you disagree with me that this is about handling of imperfect information (or more precisely, the counterfactuals - there is a variant on wikipedia where they removed the probabilites, but counterfactual reasoning still applied to resolve the paradox) maybe you won't see it as related errors.
This sounds really interesting! It was a huge deal in the poker scene when the poker AI developed by Brown and Sandholm defeated pro human players. I read the first few pages of the paper now, but the paper becomes very math-heavy after that. I don't have the necessary background to understand the notation they use. That said, the "mistake" in the way "subgames influence each other" that you referenced, I suspect it was of a far simpler kind - the kind that Brown explains in chapter 2 which he concludes with:
This shows that a player’s optimal strategy in a subgame can depend on the strategies and outcomes in other parts of the game. Thus, one cannot solve a subgame using information about that subgame alone. This is the central challenge of imperfect-information games as opposed to perfect-information games.
Although Brown says "imperfect-information games" here, he actually means a specific type of imperfect-information games: the type where the opponent's strategy is not fixed. We're talking about games where your opponent can change their strategy in order to exploit weaknesses in your strategy. This property was a key requirement of the "coin toss" example game that he provided in chapter 2. If you modified the coin toss game such that the opponent's strategy was fixed, then the situation would change completely. Crucially, Two Envelopes Game is not one of those games where the "opponent" can adapt their strategy according to your strategy. The strategy of the opponent is fixed in Two Envelopes Game. That's why you can solve a single subgame in Two Envelopes game independently of other subgames, even though it's an imperfect-information game. If you are suspectful of this claim, we can verify it by simulations.
I was agreeing with you when I talked about convergence. You said that adding a cost would solve the problem and I agree that it does. I think you were probably thinking of a literal cost like "one cent". If you choose a cost like that then switching an infinite number of times has an infinite cost. You could instead think of the cost as a fraction of your expectation, the cost is 1% of whatever you end up getting back. Now if you switch an infinite number of times you end up having a cost of zero. That might seem counterintuitive, but recall that 1/3 is .3333 repeating. So when you sum 1/3 + 1/3 + 1/3 you get .9999 repeating. Yet 1/3 + 1/3 + 1/3 is equal to one. Infinitely close to something else is basically being the thing you are infinitely close to. Even though we never get the reward we know that the fraction is becoming infinitely close to zero. People call it "converging" when we have an infinite sequence we can sum to a real value. We know before ever seeing the value that we'll be multiplying it by zero. So we can refactor the equation to be 0*ev(switch) and then take advantage of the identity of 0x=0 to declare the result to be 0. Thus, the calculation converges to zero.
This isn't really the central problem of imperfect information. Consider that in a perfect information game, your opponent will also adjust their strategy to exploit weakness in your strategy. So if it was the central challenge, why does it happen in both? You can rule it out as what he was referring to, because it doesn't discriminate between the two types of games. The central challenge in imperfect information games is you have to play with respect to your information set, not the subgame you are in. So the policies and outcomes of other subgames influences the expected value of the subgame you are in. In perfect information, the only game in your information set is the subgame you are in. So you only have to play with respect to the subgame. That is what makes perfect information different from imperfect information. It discriminates between the two game types.
The type of issue I was referring does not occur in perfect information games.
As a practical example, consider the concept of "balancing your range" in poker. If you play poker without doing that - if you play with a purely exploitative strategy where you are only trying to maximize your EV for each hand - then your strategy will be very easily exploitable by an adaptive opponent. You will frequently end up in river situations where your opponent can deduce whether you have a strong or weak hand, so they can fold to your strong hands and bluff you out of weak hands. In contrast, if you attempt to "balance your range" - that is, consider all the subgames - then you won't end up in these situations as badly. For example, when you make a particular river bet, your opponent might deduce that you have a strong hand 70% of the time and a bluff 30% of the time (as opposed to 100% and 0%).
This issue does not exist in perfect information games. Yes, my definition of "adjusting your strategy to your opponent's strategy" was overly broad to define this issue. But if you think about the poker example, where a poker player will might make a decision like "I need to bluff with this part of my range, because I need to support my strong hands [other subgames] by having some bluffs in my range in this situation" - you won't find a corresponding example from perfect information games like chess. This class of problems is unique to imperfect-information games in which an opponent is allowed to adapt their strategy to yours. If you fix the opponent's strategy, the issue disappears. If you turn the game into a perfect-information game, the issue disappears. Both requirements must be present for this issue to exist.
I thought that Noam Brown was talking about this issue in chapter 2. He discussed a simple coin toss game where one player took a strategy, and then the other player adapted by taking the optimal (exploitative) strategy against them. Then the other player changed their strategy, and the other player again adapted their strategy. And then he described a balanced (GTO) strategy. Then he said this as a conclusion:
> This shows that a player’s optimal strategy in a subgame can depend on the strategies and outcomes in other parts of the game. Thus, one cannot solve a subgame using information about that subgame alone. This is the central challenge of imperfect-information games as opposed to perfect-information games.
I thought that this corresponds perfectly to my poker example. If it doesn't, and it means something completely different, ok, sure. I'm not a mathematician. I can't even read the notation that's used in subsequent part of the paper.
I think you understood his point very well. I just think you're making a mistake in trying to recast his claim from "imperfect information games" have this property to "this narrow subset of imperfect information games" has this property. Both imperfect games with an opponent and imperfect games without an opponent have the property of needing to play as if you are in multiple subgames, because the definition of the problem is that you don't know which subgame you are in. His claim was that the policy in one subgame could influence the EV of another subgame - and in this problem, it does. I believe you're thinking of the EV of the envelope when you think you are proving this game doesn't have that property via calculation.
To see his claim applies to this game consider the case where P(Switch)=1. Being able to calculate the EV of the envelope's contents is a bit different from solving the subgame. Here, we have two subgames E1 and E2. We can know a priori what the expected values of envelope's contents in E1 and E2 are. But if you change your policy in E1, it changes your expected value in E2. If you doubt this, remember the core of the paradox again - choose to always switch and your EV is no longer the EV of the envelopes. Ergo, the EV of the subgame E1 is dependent on the strategies and policies of another subgame, E2.
If we take the poker example and we modify it by "locking" our opponent's strategy, then this property is removed. Suddenly the optimal strategy for us no longer includes any GTO-like thinking such as "balancing our range", we should simply maximize our EV for each hand "in a vacuum" without any thought to other subgames. We no longer care how our range looks to our opponent, because they are no longer able to change their strategy.
> To see his claim applies to this game consider the case where P(Switch)=1. The policy you chose in one subgame just changed the EV of another subgame.
Sorry, but I don't understand this. This sounds to me like we fix the probability of switching to 1, which means that we end up in infinite loop and the outcome can not be computed. I don't understand this premise, nor its implications for the EV of the other subgame.
> This is a subtle distinction that I mentioned earlier - the EV of the envelope as in the wikipedia problem isn't the EV of the subgame. So you're not solving the subgame if you figure out the EV of the envelope.
The expression you use "EV of the envelope" is ambiguous in this context. I'm not sure if you mean EV relative to the value of the total amount of money in the game, or if you mean EV relative to the value of the firstly-chosen envelope.
When we're talking about "solving a game", we're talking about finding the optimal decisions within a game to maximize the expected value from the game as a whole. In the Two Envelope game we have just one decision: switch or not. So we need to find out if EV(switch) > EV(stay), where both EVs are relative to the total amount of money in the game (not relative to the value of firstly-chosen envelope). We have many ways to conclude that both of these actions have expected value zero. We don't have to be able to compute the "EV of the envelope". We only need to know the relative difference between the EV of these 2 actions. There's many ways of computing them, and all of those ways lead us to the conclusion that the EV of both actions is zero. I'm not aware of any "incorrect" way of computing those EVs such that we would get a nonzero result.
I guarantee you that your solution is going to have to incorporate a set somewhere that includes both subgames. You might forget it does, because you simplify to a scalar, but it is going to be there. In perfect information, it isn't there. In imperfect information it is.
I'm having a lot of trouble identifying what exactly it is that we disagree about. If you have identified what it is that we disagree on, can you please formulate the disagreement as a wager that can be simulated with code? That way we can easily resolve the disagreement (or conclude that we actually don't have a disagreement).
Your opponents strategy in poker is fixed; they will always play the nash equilibrium strategy, they will never play another strategy. Their strategy is fixed. They will never change it from this setting.
You've claimed that subgame perfect play can be calculated without respect to the subgame you aren't in, because you can make a choice on the basis of the EV of the subgame you are in without respect to the subgames you aren't in.
I disagree. I think you still need to account for every subgame you are in as if you are in all of them.
Let the subgame you are in be you having KK and your opponent having AA. However, obviously - you only know that you have KK.
Therefore, you should be able to compute the strategy which is the best response to 37 suited and according to your logic it should be equal to the best response to AA. After all, you have no means of determining which subgame you are in. So you have to have the same response in both subgames.
So compute the best response for KK to AA and prove that this is also the best response to 37 suited.
However, you've claimed you don't need to calculate this with respect to other subgames. So your computation of 37 suited and your computation for AA must not be equal to each other - if they are, then you share terms. You calculated them with respect to each other.
Let Br = Best response.
Write a program which shows Br(p1, p2, I[KK]) != Br(p1, p2, I[KK]) and Br(p1, p2, I[KK]) = Br(p1, p2, I[KK]) simultaneously. (That is to say, both your policy and your opponents policy are fixed)
My contention is that you can't do this. You claim you can. You are free to use a simpler variant of poker - Kuhn poker - so that the computation becomes more tractable.
My earlier point was that if an opponent's strategy is fixed, then I don't have to balance how I play that KK compared to how I play that 75o. That I can play each hand "in a vacuum", in a strategy where the EV of each hand is maximized. Note that this idea is only relevant if my opponent is not playing perfect GTO (contrary to your example). For example, my opponent may have a weakness where their preflop fold frequency is the same regardless of how whether I raise 3BB or 2BB. Against this weakness the optimal play would be to raise 3BB with KK and raise 2BB with 75o. Obviously, this would be a bad strategy if my opponent was allowed to adapt their strategy to me, but it's the optimal play if my opponent's strategy is locked.
To clarify further:
- "exploitative style" poker strategy is made at the level "My hand is 96s, and my opponent's range is presumed to be Y"
- "GTO style" poker strategy is made at the level "My range is X, and my opponent's range is presumed to be Y"
If the opponent's strategy is fixed, then our optimal strategy is to play 100% exploitative. If the opponent's strategy is not fixed, then the optimal strategy will be a mix with elements from both exploitative and GTO strategies. (Again, note that this only makes sense if the opponent is not playing perfect GTO. If the opponent is playing perfect GTO, then obviously the optimal strategy is to also play GTO.)
I thought that Noam Brown was making a point related to this concept. It seems that I was mistaken and his point was related to something else entirely.
Yes; consider a chess move, every move is a subgame - a subtree of that game. He just can't safely say tree, because there are games that are better described as a graph than a tree. Actually, Two Envelope is such a game. The action switch, in Two Envelope, leads to the subgame in which you are at I[null] and need to decide whether to keep or switch - the same situation as you started in. Switch again and you arrive at the subgame I[null]. It contains itself.
> If the opponent's strategy is fixed, then our optimal strategy is to play 100% exploitative. If the opponent's strategy is not fixed, then the optimal strategy will be a mix with elements from both exploitative and GTO strategies. (Again, note that this only makes sense if the opponent is not playing perfect GTO. If the opponent is playing perfect GTO, then obviously the optimal strategy is to also play GTO.)
I agree.
Obviously the EV of envelope 1 is the contents of envelope 1. If you know the probability of reaching it you can calculate the expected value of that envelope by multiplying by the probability of reaching it. But why are you multiplying by 1/2? Probability is defined in terms of sets. What are the set that makes it 1/2? Does that set contain only the subgames that are part of the subgame you are in?
# Defining Counterfactuals
Consider a fair coin flip. You have {Heads, Tails}. Lets assume you are going to get heads, take it as a given - that is what actually happens. It actually happens. It is factual. However, for the purposes of analysis, sometimes it doesn't really matter that we know what happened. We need to consider all the cases that didn't happen. Tails in our analysis would be the counterfactual. These two situations, the factual heads and the counterfactual tails, they're associated with each other. There is a set {HT} that contains both of them. E.g P(head) = |{H}|/|{H, T}|.
When I'm saying counterfactual I'm referring to the events that we didn't assume to be factual, but which we want to keep track of. Probability kind of drops these terms when it says "assume" because it says |{H}|/|{H, T}| becomes |{H}|/|{H}| which is equal to one. This is perfectly fine from a math perspective. The thing is that just like saying 5/x = y lets us move to 5=xy is valid, it has some assumptions built into it. Namely that x can't be zero. Our counterfactuals are a bit like the x, because they offer an easily hidden constraint on what it is valid to do. If you reintroduce uncertainty about whether or not you are in H, you have to do so in a way that takes you back to having the set {H, T}.
Let me show you a practical example of that to make the point very clear:
Let the value of heads be one and the value of tails be zero. P(Head) = 0.5 But assume heads on the same coin flip. P(Head|assumptions) = 1.0 But assume heads again on the same coin flip. P(Head|assumptions) = 1.0. Now since P(Head) is actually true with 1/2 probability: Ev(Head) = 1/2*P(Head|assumption) + 1/2P(Head|assumption)?
Well, it is certainty true that 1/2P(Head|assumption) is a correct term. So you can't say this is wrong on the basis of the assumptions alone. Being very precise, the problem is the neglected counterfactual wasn't handled. Every term leading up to the equation was technically true, but obviously we just did something really weird right? And to be very precise, we neglected the counterfactual associated with heads.
# Defining subgames
Most games can be written out as a game tree. I showed one in my previous post. When you move down the tree, you are in a subgame of that tree.
In a perfect information game like chess if you are in a subgame, a portion lower on the tree, the other parts of the tree don't matter anymore. Your results are independent of the rest of the game tree.
In imperfect information though, just because you are in a subtree doesn't mean you know which subtree you are in. Your actual view into the game is through the information you have. Just like in the {HT} case you have to consider the potential that you have both H or T, in a subgame you have to consider the potential you are in every subgame that is reachable given the information you've seen.
Check out this to get a better sense of what I'm talking about: https://www.youtube.com/watch?v=EbKmZLp5HvA
Since you seem to be making some authority arguments just know that I have a PhD in ML / stats and was top 30 in my country in olympiad level competitive maths so I DO know what I do and in this case with absolute certainty.
Look for the other guys answer if you want more details
Let's review what you're claiming with absolute certainty. This is the first claim that I'm contesting:
> they say "one envelope contains 2A and the other A/2".
When pointed out that they actually don't say this, you clarified your point:
> the "writing" doesn't say it but the written formula at bullet point 7 says it. That's the mistake, the formula is wrong and does not describe reality.
The second part of that quote is correct: bullet point 7 has the mistake, the formula is wrong and does not describe reality.
The first part of that quote is incorrect: the written formula at bullet point 7 does not claim that one envelope contains 2A and the other envelope contains A/2. It claims that the first envelope contains A and the second envelope has a 50% probability of containing 2A and a 50% probability of containing A/2.
You correctly identify where the mistake is, but you misidentify what the mistake is.
If you read bullet point 6, it is very clear: "the other envelope contains 2A with probability 1/2 and A/2 with probability 1/2". Notice that bullet point 6 does not claim "One envelope contains 2A and the other envelope contains A/2".
To illustrate my point, consider a simple coin toss game. If the coin comes up heads, you win $2. If it comes up tails, you win $0.50. How might we construct the EV calculation for this coin toss?
0.5 * $2 + 0.5 * $0.50
Notice how both parts of the formula refer to the same coin. They don't refer to different coins. You have one coin that you flip, and depending on how that one coin lands, there is some probability that you get $2, and some probability that you get $0.50. If you looked at that formula, you wouldn't think that the first part refers to different coin that the second part.
Similarly, here we have one envelope (similar to having one coin) and we have uncertainty about the what the envelope contains (similar to having uncertainty about how the coin will land). We have 50% chance that the envelope will contain 2A, and we have 50% chance that the envelope will contain A/2. Thus, we arrive at the (incorrect) formula:
0.5 * 2A + 0.5 * A/2
It's very obvious that both parts of the formula refer to the same envelope, just like in the coin toss formula both parts of the formula refer to the same coin. To be extremely clear, I am not claiming that the formula is correct. I am claiming that both parts of the formula refer to the same envelope, not to different envelopes. To be specific, I am refuting this claim that you made:
> they say "one envelope contains 2*A and the other A/2".
They don't say that.
Furthermore, you make an additional incorrect claim:
> The right formula is changing either the 0.5A by A or the 2A by A.
Here you are just randomly changing the formula in order to make it output 0EV (which is the correct answer). Just because you get a correct answer does not mean your computation was correct. In this case it is incorrect, because A was defined badly. In order to fix the formula you would have to fix the definition of A. If you keep the incorrect definition of A and just shuffle symbols around until you get the correct result, your computation is still incorrect.
But it's wrong! Try with literally any A.
> If you read bullet point 6, it is very clear: "the other envelope contains 2A with probability 1/2 and A/2 with probability 1/2". Notice that bullet point 6 does not claim "One envelope contains 2A and the other envelope contains A/2".
Sure but this bullet point is also dead wrong and possibly the start of the scam. If you say A is lowest value, Either it contain A with probability 1/2, or 2A.
Otherwise you are changing the reality of A midway through the sentence.
To meet you halfway, what you may actually want, or see in your head is the possibility that A (in the lowest value sense) may vary in the experiment, and try to compute the expectation of that new problem.
But that's a different problem. And if you want to compute the expectation of this problem with A varying in a range you have to write an integral over the p(A) around the (corrected) equation 7. But the equation 7, even in this integral, need to use the same consistency for A: either you have A being the lowest envelope value and you only write 2A ever, or the highest value and then you only write 0.5A
As I said, I agree that the formula is wrong. I disagree with you regarding where the mistake is and how to fix it.
> Sure but this bullet point is also dead wrong and possibly the start of the scam. If you say A is lowest value, Either it contain A with probability 1/2, or 2A.
I think maybe you have some typos here, because I don't understand what you're trying to say here. In any case our disagreement concerns whether the formula in Wikipedia refers to the same envelope in both parts of the formula, or different envelopes. You kept stating that it refers to different envelopes, and I wrote a long post to refute that. Now you came back saying that the bullet / formula / Wikipedia text is "wrong" and "possibly the start of the scam". Ok, sure, but the EV formula still clearly refers to the same envelope in both parts of the formula, despite the fact that you claimed otherwise with literally "absolute certainty". You were wrong about something that you claimed to know with "absolute certainty", so perhaps you should in the future adjust those estimates downwards with a bit of uncertainty added in?
> Otherwise you are changing the reality of A midway through the sentence.
Nope, A is defined as the "value of the firstly-chosen envelope" in the beginning of the sentence, and A is defined as the "value of the firstly-chosen envelope" in the end of the sentence. The definition for A does not change midway through the sentence.
> But that's a different problem. And if you want to compute the expectation of this problem with A varying in a range you have to write an integral over the p(A) around the (corrected) equation 7. But the equation 7, even in this integral, need to use the same consistency for A: either you have A being the lowest envelope value and you only write 2A ever, or the highest value and then you only write 0.5A
That's not the only way to compute the expectation for this problem. I actually ran some small simulations for this today, representing wagers on this problem. The simulations demonstrate the following claims:
- Steps 1 through 7 in the Wikipedia page are correct, in terms of calculating the "expected value in the other envelope, relative to the value in the firstly-chosen envelope". The expected value for the switch, relative to the firstly-chosen envelope's value, is in fact positive (+25%).
- At the same time, the "expected value for the switch in absolute terms" is zero.
- These claims do not contradict each other.
- The first error in Wikipedia's line of reasoning is step 8 that states "they stand to gain by swapping". This indicates that the player should try to maximize goal "expected value relative to firstly-chosen envelope", which is incorrect. The player should instead maximize goal "expected value in absolute terms".
If you disagree with any of these, let's formulate our disagreement in the form of a wager that can be simulated in code.
A man walks to a house with a ladder. He explains that ladders are climbing devices. He tries to set it up against the house, killing plants that he sets it on. The ladder falls over.
One person watches this happen and correctly explains that the ladder is a climbing device and says many very true things about the intentions and proves the genuine thoughtfulness of the ladder users approach. He isn't wrong.
Another person looks at everything from a different perspective. They are a time traveler and they experience the world in reverse chronological order. They say the ladder isn't a climbing device - it is a thing which falls on ground. They say it is a flower killing device. They claim that when the person explains that the ladder was for climbing, they were wrong, because it isn't. They fundamentally disagree about what the ladder is, because they viewed the ladder from a different perspective. He isn't wrong.
They start arguing about what the ladder is. The first person is trying to argue that the ladder isn't a falling device, because it isn't defined to be so. The other person is arguing - this is very critical by the way - not that it is a falling device, but that the way it got used meant it was one and that this disagrees with the statements about what the ladders purpose was. Now, because they aren't noticing this distinction, they argue and they try to prove that the ladder isn't what the other person is claiming it is. Perhaps they think they are winning the argument by pointing out the contradiction, but pointing out the contradiction is agreement that the contradiction exists.
You are using a programmatic understanding of what is happening. You are having to proceed forward from statements in order to get meaning. But there are different paradigms in programming type resolution which might help you understand why you get such sharp disagreement. In programmatic terms you can actually have this backward flow too. b [unknown]; foo(int); foo(b) -> b [int] is implied. You can propagate the type backwards from the function call.
Critically, this is extremely common in math. Mathematics has identities. The relationships between operations flow along those identities. When you let the identities flow backwards, the ladder you two are discussing stops being a ladder. When they make the choice to use a known algorithm, getting the expected value for a subgame under imperfect information, we know the relationships which flow backward from a correct usage. We know what was 'supposed to' be there. They had an incorrect usage. Their ladder wasn't intended to be for falling. But we're experiencing the world in a backwards direction. So we don't know that yet, like you know it. We're treating it as it was used, not as it ought to have been used - flowing what it is backwards from what they mean in the equations, not forward from how they are defined.
So he started trying to explain his thinking to you by moving backward and you noticed that he was saying the ladder was a falling over device, when the ladder maker told you it was something else. And you are right. He isn't wrong so much as his understanding of what the terms are was decided by a different process. And he agrees that you are right that the terms are defined differently!
You aren't really disagreeing about as much as you think you are, because you and him both already agree that the ladder maker said the ladder was for climbing and he said the ladder was for falling over. When you find the contradictions, well, I'm not sure you are actually proving him wrong so much as you are agreeing with each other that a contradiction exists.
Or code that “obviously” has a bug only after it caused an issue in production.
Would be nice if someone warned me before I do a PR: “ok, fyi there is a subtle but devastating bug in the new_feature.cpp”
Where is the bug in this programm?
Cognitively, the mind filters out quite a bit of information for performance reasons.
Are we biased in favor of reasoning that "sounds plausible"?
My "hot dice" say "yes".
> having chosen an envelope at will, but before inspecting it, you are given the chance to switch envelopes. Should you switch?
> because the person stands to gain twice as much money if they switch, while the only risk is halving what they currently have, a case can be made for switching the envelope
I dig this, but it seems like an unavoidable nonsense pattern that congeals in pools of otherwise practically useful logic / language systems.
Like Neo in the Matrix. Remember the Architect? That guy was cool.
I'm pretty sure this could be reduced to the secretary problem where there's a pool of 2 candidates, which yields an optimal hire with probability 50%.
Maybe you play word games and say, "I guess the other number is not higher," or "I guess the other number is not lower." Since the number you observed was produced once, there is some non-zero probability that it was produced twice, in which case the other number is neither nigher nor lower.
Here's a hint that is not a full solution. If you knew that the numbers in both envelopes were independently sampled from the same Gaussian (but not necessarily which Gaussian), then there is a simple strategy that wins more than 50% of the time: pick an envelope to peek at uniformly at random and guess that it is the larger one if and only if it shows a positive number.
Why does this work? If both numbers are positive, your strategy is equivalent to "pick a random envelope" and you're left with a uniformly random choice of envelope. If both numbers are negative, you're also left with a uniformly random choice of envelope. But if one is positive and the other is negative, you win 100% of the time.
What are the odds of the third case? Well every Gaussian has at least some mass on positive numbers and some mass on negative numbers. So there is some nonzero chance, even if you don't know what it is. So while the strategy might not always do better than a coin toss, some positive percent of the time you win every time. And so you get a distinct advantage over random guessing.
The problem is now how would one transfer such an approach to the original problem without the Gaussian assumption. Somehow the two problems are less different than they initially seem.
Yep.
> Therefore, if the number is 9, both strategies tell you to do different things, yet in each case, your probability of winning is >½. There must be a hole in the logic somewhere…
For any nonnegative value C, the probability of Gaussian draw x being larger than (iid) Gaussian draw y is strictly bigger than 1/2 when conditioned solely on x > C.
The particular choice of C might change which specific draws of x and y the strategy succeeds with, but as you noticed for every C a thresholding strategy of the above form does give you some pointwise nonzero advantage over random guessing.
You pick an arbitrary threshold, if your envelope is below the threshold, you switch.
(You can add randomness as necessary.)
Choose a monotonically increasing function f with values between 0 and 1. Choose one of the envelopes at random. Then look at the number x in the envelope. Then, with probability f(x) say that you have the larger envelope.
Both are equally baffling in how / why they work.
A tangential question: real numbers are uncountably infinite. Are probability distributions over the real numbers likewise uncountably infinite, or do they form a higher infinity?
As for what uncountable cardinality they form, it's the same as the real numbers [0]. Roughly, a probability distribution is determined by the countable collection of real numbers P(X <= q) with q rational. That means the cardinality is no more than R^Q, which is isomorphic to R. (the cardinality is at least R due to, say, uniform distributions on [0, x] for x real).
[0] https://math.stackexchange.com/questions/3698864/why-does-th...
1. If the number you pick is positive, then there are technically (although arbitrarily small) more negative numbers that can be chosen for the other envelope. So you have a slightly larger than 50% chance of winning. 2. The inverse is true if you chose negative, however.
That said - I think this would generally imply you should always switch, which is why I'm not convinced I'm right. Consider a purely positive set, for example. If my number shows up as "12" - then I know there are infinitely more numbers larger than it, but only 11 numbers lower than it. But this starts to feel like the original two envelopes paradox.
--
tldr; I'm not sure of the answer, although I think if I saw the answer I would have an "Aha!" moment.
Let's represent the poorer envelope by the value 0.5 and the more bountiful one by 1.0.
The expected value from the envelope draw is 0.75: 0.5 drawn with a 50% probability contributing 0.25 to the expected value, and 1.0 drawn with a 50% probability contributing another 0.5 to the expected value for a total of 0.75.
If you were to do a large number N of these draws, the value you will obtain will be from a distribution that is centered on 0.75 N. So far so good?
So the way to look at a draw is this: you're going to be getting the expected/average value of 0.75, together with either a 0.25 bonus if you pick the better envelope, or a else -0.25 penalty if you pick the poorer envelope.
So you see, those two situations are equal and opposite. By switching, you either turn a 0.25 bonus into a -0.25 penalty (- 0.5) or vice versa (+ 0.5) with equal probability.
If you need any more complex analysis than this, you need to be thwacked on the head.
The rhetoric about doubling versus halving the money is a distracting red herring. If you switch from the 1.0 envelope to the 0.5 envelope, you're down 0.5, and if you switch the other way, you're up 0.5. That's it.
Accounting is based on debits credits not halving and doubling. You would never update a ledger by crediting double some amount to one account, and debiting half the amount from another account. It's all purely additive.
You have to look at what you're potentially gaining or losing on its own not as a fraction of some guaranteed fixed portion that you're getting from either envelope.
OK, but let's look at it:
1. Denote by A the amount in the player's selected envelope.
2. The probability that A is the smaller amount is 1/2, and that it is the larger amount is also 1/2.
3. The other envelope may contain either 2A or A/2.
OK, here is where this argument got seduced by the multiplicative thinking. The correct reasoning is: let's call the smaller base amount A and the larger one 2A. One envelope contains A, the other one 2A. That's it.
3. If A is the smaller amount, then the other envelope contains 2A.
4. If A is the larger amount, then the other envelope contains A/2.
Here, the argument is splitting into cases because of the IF, and in these two cases, A has a different meaning.
5. Thus the other envelope contains 2A with probability 1/2 and A/2 with probability 1/2.
6. So the expected value of the money in the other envelope is:
This is wonky. The overall two-envelope situation has an expected value, not the individual envelope.
The correct view is that if this is the richer envelope, it contains X more money. If it is the less endowed one, then it contains X less.
7. This is greater than A so, on average, the person reasons that they stand to gain by swapping.That calculation is wrong though, by itself. That issue should be attacked directly.
Separate calculations have to be performed for the two cases: A is the smaller amount and A is the bigger amount. For each of these cases, the expected value from both envelopes has to be calculated.
The way the above formula is doing the expected value calculation is off because it's equivocating on A.
It ends up working with two unrelated quantities: 2A and A/2. 2A is four times bigger than A/2. A is simultaneously denoting the smaller amount and the larger one.
8. After the switch, denote that content by B and reason in exactly the same manner as above.
Yes, and if you do, you will end up with 5/4B. So that can't be right and the whole remaining argument about infinite switching being irrational compared to just opening an envelope is superfluous.
1. We have A; the other envelope has 2A; or else
2. We have 2A; and the other envelope has A.
In case 1, if we switch we gain A.
In case 2, if we switch, we lose A.
We have no idea which case we are in.
We cannot use the variable A to denote our initially selected envelope in both cases, because then A simultaneously represents the smaller amount and the larger amount, and we are absurdly mixing the quantities A/2 and 2A in the same calculation. No quantities in the actual problem are related by a factor of four!
People might be seduced into the A equivocation perhaps because they are initially thinking of A as being a label for the envelope, which is arbitrary and innocent enough. We have some envelope A and it has some money in it. Oh, but the other envelope then has either A/2 or 2A. At that point, A is no longer an envelope name, but a name for an amount. Then it has to pin down a specific number: it has to denote either the smaller value, or the larger one, and that value is found in different envelopes in the different cases; A cannot both be "name of initially chosen envelope" and "amount in that envelope".
If the envelopes contain, say, $50 and $100, it looks like a doubling/halving. But nothing materially changes if we add $1000 to each one to make $1050 and $1100. The probabilities don't depend on what is in the envelopes, and the win/loss is just a spread of the difference between their values.
Why would anyone, while solving some problem, worry about some unspecified more complicated situation which supposedly resembles that problem?
Give me the specific of that more complicated problem and I will knock it out of the ballpark using whatever reasoning fits that problem.
let's imagine the first envelope contained between 1 andand 1000 dollars, the second double the first.
if you open your envelope and it contains a whole number of dollars between 1 and 1000, then you should swap, id take "250 or 750" over 500.
http://wikibon.org/wiki/v/Prepare_three_envelopes
https://en.wikipedia.org/wiki/Monty_Hall_problem
https://m.imdb.com/title/tt6026160/movieconnections
this video explains the paradox properly, with a very reasonable explanation.
it doesn't really matter what the distribution of the amounts are, as long as there's an infinite number of possibilities in the distribution (ie., it's not a finite amount of possible envelopes).
how would one prove that claim? it's not obvious to me. it seems like the specific conditions under which the series of terms in the total expectation has a well defined sum is important, but I didn't fully grasp those conditions
For example if instead of halving the probability of each value as he does in the video we take 1/20 then we get the expected value as the sum of 9/20^2 - 9/20^2 + 90/20^3 - 9/20^3 + … Here the series is absolutely convergent since the positive terms sum to 9/10 and the negative terms sum to 9/10.
but then the probabilities don't all add up to 1.
When the amount you multiply your money by depends on the amount of money you currently have, you have to factor in your initial money to each case to compute expected return.
let M be the maximum amount of money that a envelope can contain (at worst it can be the total amount of money in the world). You don't know the value of this upper bound M.
Most of the time, you get 2*x or x/2 with 50% chance if you choose the other envelope (which on average is a win)
But when x=M you get M/2 instead of M and therefore you loose M/2 with 100% chance. And when it happens you have lost much more than in the other cases (because exponentiel...)
If you do the math, you can see that what you win for 1 2 4 8 16 ... M/2 is equal to what you loose for M.
And so on average what win from switching is 0.
https://en.wikipedia.org/wiki/Monty_Hall_problem
(except that problem has a more definite solution)
https://ycmusings.com/why-wont-we-switch/
Well no, one doesn't contain one fourth of the other.
Yes, B = A/2 or B = 2A with probability .5 each, but not for the same value of A.
Then whatever trick is being played you still have a 50-50 chance.
https://www.youtube.com/watch?v=0yjVv_iB1h0 (Interestingly, a lot of those DoND comments have gone. Did they delete them?)
Normally we would assume there is someone who owns the envelopes of money. Now their incentive could be to offer to switch only if the envelope you chose contains more money (so they have a chance of keeping more). More like Monty Hall.
Even without that, after switching, should you switch back? The equation given argues you should switch back, so obviously the equation must be wrong.
It is easier to think: if I switch I have a 50% chance of making $X and a 50% chance of losing $X. Switching doesn’t change expected $.
I actually think the later problem in the Wikipedia article is more clever:
Ah, that observation is very clever! Even if you let yourself be bamboozled by the logic for switching, there is no reason then not to switch again by using the same logic. And again.. and again!
But assuming a more surprise situation in which the game is suggested* the reasoning is pretty good for both if they're happy with a 50% chance at doubling their money + X. But I think maybe its the hidden prize that makes the logic work (or not work)? For one of them the prize is less then double their money (a prize they'd never win but still) so the deal isnt a good deal. Not sure how the math works for that, but a contest where you might win 2a or less then 2a is presumably a different reasoning.
*Where similar to the first point we should assume the suggesters wallet is empty.
Now what is the probability of each winning? Did adding that step actually change the problem?
Or reword the problem just like I did in my grandparent comment: call the amount in the larger wallet $X. If you lose then you lose $X, and if you win then you win $X. 50% chance and expected outcome $X/2 by playing.
That is, the probability is 1/2. That's all that matters to the decision making. Calculating an "expected value" at all is completely useless, whether or not you do it "correctly".
Am I missing something?
It just seems obvious to me that it's not the same situation. That is, I think the "paradox" is ultimately about the temptation to treat these as similar problems in the first place.
No, because you're not actually wagering anything in the envelope game, so it makes no sense to calculate probabilities as though you are.
An analogous coin toss example would be something like: I'll flip a coin, you call heads or tails. If you call it right, I'll give you $10. If you're wrong, I'll give you $5. (So either way you make $5, so I don't know why anyone would even offer this game!) After I flip the coin, I'll conceal it in my hand and give you a chance to change your choice of heads or tails. Well, of course there's no reason to change your choice, it's completely arbitrary! You've got a 50-50 chance of winning $10 regardless. The paradox makes you think you're wagering your unknown potential winnings, but you're not actually wagering anything.
It has nothing to do with calculating whether or not a specific wager is worth it given some set of probabilities. That's a different problem for a different sort of game.
If we consider the decision point where you can either keep the first envelope or switch it, you are effectively wagering the value of the first envelope. You're just muddying the waters when you're trying to make some kind of point about receiving the first envelope for free. Yes yes, you are only gambling your "winnings" that you won earlier, it doesn't make a difference here.
> An analogous coin toss example would be something like: [...] I don't know why anyone would even offer this game!
Look, you made the claim that expected value doesn't matter for anything at all, the only thing that matters is the probability of winning. I offered you a game where you have a 50% probability of winning on each coin toss and I offered to pay you $1 for each coin toss to incentivize you to play it. Can you please explain why you are refusing to play this game? Yes, you don't like the analogy, I get that, but you also don't like free money? I find that hard to believe. A more plausible explanation is that you don't actually believe the claim you are making. That's why you aren't willing to wager any money on it.
It makes all the difference because you don't know what those "winnings" are, and what you're potentially wagering them for depends on what they are. So ultimately you're not really wagering anything. (Again, I think this is the crux of the "paradox".)
> Look, you made the claim that expected value doesn't matter for anything at all
I never meant to claim it doesn't matter for "anything at all", just not for this envelope game. (Apologies if the meaning of "at all" was ambiguous in my original post.)
When you say "this envelope game", I assume you are also including minor variations of this game?
If you mean specifically this exact envelope game, where the expected value for all actions is zero, then it is strictly true that it doesn't matter how we make decisions - using expected value or not - though it's not a particularly interesting point to make.
I will proceed assuming you mean also including minor variations of this game.
You wrote this earlier:
> That is, the probability is 1/2. That's all that matters to the decision making. Calculating an "expected value" at all is completely useless, whether or not you do it "correctly".
We can make a minor variation to the game such that the probability will still remain at 1/2, but the expected value for switching can be changed. We can make the minor variation such that it will be profitable to switch, or such that it will be unprofitable to switch. We can do this while the probability remains at 1/2. According to you the probability is all that matters, and it's useless to calculate the expected value for a game like this? This would lead to making unprofitable decisions in a game with a small variation as described above.
No; it seems obvious to me that the argument for switching presented in the original wikipedia article is meant to apply only to the evelope game as it's presented. Of course introducing variations could easily change the meaningfulness of the article's premise. That is, it wouldn't be considered a "problem" or a "paradox" if calculating an "expected value" were actually meaningful.
> then it is strictly true that it doesn't matter how we make decisions - using expected value or not - though it's not a particularly interesting point to make
True, but my point was more of a question: if it's obvious that calculating an "expected value" is irrelevant in this specific case (as the article says, "It may seem obvious that there is no point in switching envelopes as the situation is symmetric"), why is the argument presented in the article considered compelling? That is, either it's not actually compelling, or the "expected value" being meaningless in this case is not necessarily so obvious at first... but if so, why not? (Or, to put it another way, why is the argument in the article compelling enough to warrant such a long wikipedia page with such numerous proposed "resolutions"?)
Your question is akin to a driver plowing through red lights without an accident and concluding that the traffic light was useless in this specific instance. "I didn't cause a crash even though I ran a red light, so the traffic light was meaningless! In this specific instance I mean! It didn't matter if I stopped to the red light or ran across it, no accident would have happened either way in these very specific circumstances, so why are people even talking about traffic lights?"
I'd be happy to take you up on that offer. Let's play some dice or card games that are set up such that you'll always have 9/10 probability of winning. You'll play, regardless of the expected value, right?
E.g. if the maximum volume of currency that could fit in an envelope is $10k, it would be possible to have a uniform prior between 0-10k, at which point I would stick if my envelope had $5.01k.
A more extreme prior could be one capped at the largest ever academic research grant (I bet it's not more than $100m).
It could also be that you have a prior that has infinite mean, in which case it is not surprising that you will always want to switch if you draw a specific value.
If you have risk aversion, this further increases the range that you would like to stick, since switching could decrease your expected utility, even if it increases your expected return. There is a separate paradox in decision theory that we should be approximately risk-neural for 'small' amounts.
(The usual term is sample space; it's just more fun to split and eliminate universes than states.)
`B = (2A if A=50, A/2 if A=100)`. Simplifying this to `B = 2A or A/2` loses important information: namely that when B is smaller, you expect A to be larger. Or alternatively, treating A as fixed (say A=100) conflates two different situations: one where the amounts are `100, 200` and one where they are `100, 50`. So you end up thinking B must have (200+50)/2 in it, which is incoherent. If you compute the expected B correctly then `A` is different in each branch.
It is subtle though. If this appeared in a paper, arguing something not obviously wrong, it'd be hard to convince everyone that the reasoning is bad.
https://www.youtube.com/watch?v=_NGPncypY68
TL;DR (Spoiler alert): the expected value of the amount of money you end up with is an infinite series whose sum changes depending on the order in which you add up the terms, and so you can choose an order that makes this value come out to be positive, negative, or zero.
That initial setup being, the act of starting the game with someone is(?) confers value in and of itself.
So I’m glad the real solution is wonky.
I believe mike_hock provides the clearest and simplest explanation in this thread: https://news.ycombinator.com/item?id=31567251
I also made this argument separately: https://news.ycombinator.com/item?id=31569991
When E[X], E[A], and E[X - A] are all well-defined, it is indeed the case that if E[X | A = a] > a for every particular value a, then E[X] > E[A].
What goes awry in this case is that E[X - A] is not well-defined (where X is the value in the unselected envelope and A is the value in the selected envelope). It is given by a conditionally convergent series, as noted.
I see that the article also says "commonly one writer proposes a solution to the problem as stated, after which another writer shows that altering the problem slightly revives the paradox." But it doesn't seem to elaborate on where the simple resolution falls down, if it does.
However, suppose x is placed in an envelope and handed to you. Then a fair coin is flipped. If it comes up heads, 2x is placed in another envelope and if it comes up tail, x/2 is placed in that envelope. Then you should switch.
It's a subtle difference in how the envelopes are prepared, but it makes all the difference.
I’m not sure I’d have naively realized that the fixed amount in your envelope makes so much difference.
For people confused like me, think about the outcomes:
When the two are prepared together, say $2 and $4, then when you exchange your options are +2 and -2 with 50:50 odds.
When the second envelope is prepared based on $2 in your envelope, then when you exchange your options are +2 and -1 with 50:50 odds.
(I had to diagram this all out; unintuitive to me!)
The introduction of a second "stage" based on your action changes things in a non-obvious way. Your first action is less impactful than your second one, regardless of what you did.
At the end I constructed a super weird example where regardless of what you see in your envelope, you should switch (this seems to go against the symmetry argument, but is sort of saved by a divergence in the expectation value).
You can lose money and that has to be reflected in the potential value of each envelope.
After the first selection, you must express the envelope value as the potential of what each envelope holds (the probabilities from the initial selection) which makes selecting again a wash.
Let A = 50 Envelope 1 is 100 Envelope 2 is 25
First selection 1/2(100) + 1/2(25) = 62.5
Great! Do it! 62.5 is bigger than 0, which is the expected value of not playing at all.
After that, it makes no difference to switch (the envelope value is recursive):
1/2(62.5) + 1/2(62.5) = 62.5
or
1/2(1/2(2A) + 1/2(A/2)) + 1/2(1/2(2A) + 1/2(A/2)) = 1/2(2A) + 1/2(A/2) = 5/4A
and the trick is that’s now the potential expected value you have (you now have 5/4A in your envelope), so switching is a wash. You never really “had” A as a value to compare against (by evaluating that 5/4A is bigger than A), we just get tripped up with the doubling and halving at the outset.
Said another way, the fact that 5/4A is bigger than A is irrelevant, no envelope contains A, they both contain the expected value of 5/4A.
No. The problem specifies that E1 is twice E2, but here you have E1 = 4 x E2.
A is the amount in one of the envelopes, so if A=50 then either E1 is 50 or E2 is 50, and the other E is 25 or 100. But under no circumstances can E1=100 and E2=25 at the same time.
The value of the other envelope must include the probability that it is the original A (not some sleight of hand new A’ that, itself, is based on an expected value).
Yes, that's right. That's exactly what I said: "A is the amount in one of the envelopes, so if A=50 then either E1 is 50 or E2 is 50, and the other E is 25 or 100. But under no circumstances can E1=100 and E2=25 at the same time."