MetaProbability Problem
Moderators: gmalivuk, Moderators General, Prelates

 Posts: 35
 Joined: Wed Apr 09, 2014 1:33 am UTC
MetaProbability Problem
You have a coin, which has an unknown probability distribution biased towards heads.
(or tails, but that doesn't change the theory behind this question)
What is the probability you will, over two tosses, in no particular order, get a head and tail?
This is not a homework question, it arose from a discussion my nerd friends had about unknown weighted probability of simple events.
(or tails, but that doesn't change the theory behind this question)
What is the probability you will, over two tosses, in no particular order, get a head and tail?
This is not a homework question, it arose from a discussion my nerd friends had about unknown weighted probability of simple events.
GENERATION 705  992 i: The first time you see this, copy it into your sig on any forum. Square it, and then add i to the generation.
Re: MetaProbability Problem
If you know nothing about the coin other than that it's biased towards heads, then the probability distribution for the coin is 2 on [0.5, 1] and 0 everywhere else. The probability of a heads and a tails given a probability of heads p is 2p(1p), so we get that the probability is int_{0.5}^1 2(2p(1p))dp, which I don't care to calculate but my intuition tells me is 1/3.
(∫p^{2})(∫q^{2}) ≥ (∫pq)^{2}
Thanks, skeptical scientist, for knowing symbols and giving them to me.
Thanks, skeptical scientist, for knowing symbols and giving them to me.
Re: MetaProbability Problem
This comes to the fuzzy edge of what we mean by "probability".
Ostensibly, coin flips are still independent and identically distributed, we just know that the probability of heads p is larger than 0.5. The chance of a head and a tail is 2*p*(1p), so we can immediately conclude that the chance of getting one head and one tail is less than 50%, since this is true for any value of p != 0.5. Beyond that, you can't conclude much: it depends on your prior distribution over possible values of p. In other words, it depends on what you believe about unknown unfair coins. Are they usually almost fair, or usually radically unfair, or what?
Once you pick a distribution for p, it's a straightforward calculation of E[2*p*(1p)]. The simplest assumption would be a uniform distribution on (.5,1) but there's no clear reason that this (or anything else) must be the correct distribution for an unknown unfair coin.
Ostensibly, coin flips are still independent and identically distributed, we just know that the probability of heads p is larger than 0.5. The chance of a head and a tail is 2*p*(1p), so we can immediately conclude that the chance of getting one head and one tail is less than 50%, since this is true for any value of p != 0.5. Beyond that, you can't conclude much: it depends on your prior distribution over possible values of p. In other words, it depends on what you believe about unknown unfair coins. Are they usually almost fair, or usually radically unfair, or what?
Once you pick a distribution for p, it's a straightforward calculation of E[2*p*(1p)]. The simplest assumption would be a uniform distribution on (.5,1) but there's no clear reason that this (or anything else) must be the correct distribution for an unknown unfair coin.
No, even in theory, you cannot build a rocket more massive than the visible universe.

 Posts: 35
 Joined: Wed Apr 09, 2014 1:33 am UTC
Re: MetaProbability Problem
Meteoric wrote:This comes to the fuzzy edge of what we mean by "probability".
Ostensibly, coin flips are still independent and identically distributed, we just know that the probability of heads p is larger than 0.5. The chance of a head and a tail is 2*p*(1p), so we can immediately conclude that the chance of getting one head and one tail is less than 50%, since this is true for any value of p != 0.5. Beyond that, you can't conclude much: it depends on your prior distribution over possible values of p. In other words, it depends on what you believe about unknown unfair coins. Are they usually almost fair, or usually radically unfair, or what?
Once you pick a distribution for p, it's a straightforward calculation of E[2*p*(1p)]. The simplest assumption would be a uniform distribution on (.5,1) but there's no clear reason that this (or anything else) must be the correct distribution for an unknown unfair coin.
Well one of the heads involved in the discussion claims that because every distribution has a "complementary distribution" (i.e. p <=> 1p)
then the averages of the infinite probabilities gives a uniform distribution.
I'm skeptical because of the way infinities are handled.
GENERATION 705  992 i: The first time you see this, copy it into your sig on any forum. Square it, and then add i to the generation.
Re: MetaProbability Problem
Paradoxica wrote:Well one of the heads involved in the discussion claims that because every distribution has a "complementary distribution" (i.e. p <=> 1p)
then the averages of the infinite probabilities gives a uniform distribution.
I'm skeptical because of the way infinities are handled.
I pretty sure it doesn't work like that. If I tell you that I have a biased coin, would you think that 0.5 > p > 0.6 is equally likely as 0.9 > p > 1 ? I wouldn't think so. If you have completely no knowledge of the biases of coins, then assuming a uniform distribution of probabilities isn't the worst thing you can do, but it's not really "correct" either. The beauty of Bayes' theorem is that your choice of prior distribution doesn't really matter after performing lots of experiments, and process almost certainly converges to the true probability if you assume a uniform distribution over all the possible p's. However if you need to estimate the probability of rolling one heads and one tails without performing any experiments first, the best you can do is bound this probability using the range of the possible values of p.
 doogly
 Dr. The Juggernaut of Touching Himself
 Posts: 5538
 Joined: Mon Oct 23, 2006 2:31 am UTC
 Location: Lexington, MA
 Contact:
Re: MetaProbability Problem
A uniform distribution of .5 < p < 1 makes sense to assume, because it is the simplest distribution that fits what you know (that it's somewhere between those two endpoints.) But you should attach little faith to this, start flippin coins and updatin dat prior.
LE4dGOLEM: What's a Doug?
Noc: A larval Doogly. They grow the tail and stinger upon reaching adulthood.
Keep waggling your butt brows Brothers.
Or; Is that your eye butthairs?
Noc: A larval Doogly. They grow the tail and stinger upon reaching adulthood.
Keep waggling your butt brows Brothers.
Or; Is that your eye butthairs?
Re: MetaProbability Problem
I read the question as "All you know about the distribution is that it's biased towards heads.". If you want to bring other "knowledge" into this about intuitions regarding coin manufacturing and whatnot, then that opens a can of worms that can't be sealed again. How do we know it's biased towards heads? What's our confidence in the observations that led us to that conclusion? The problem pretty clearly suggest a prior, so if you don't go with that one, then things get very messy very fast, and not in a way that leads to some greater insight.
(∫p^{2})(∫q^{2}) ≥ (∫pq)^{2}
Thanks, skeptical scientist, for knowing symbols and giving them to me.
Thanks, skeptical scientist, for knowing symbols and giving them to me.

 Posts: 35
 Joined: Wed Apr 09, 2014 1:33 am UTC
Re: MetaProbability Problem
Cauchy wrote:I read the question as "All you know about the distribution is that it's biased towards heads.". If you want to bring other "knowledge" into this about intuitions regarding coin manufacturing and whatnot, then that opens a can of worms that can't be sealed again. How do we know it's biased towards heads? What's our confidence in the observations that led us to that conclusion? The problem pretty clearly suggest a prior, so if you don't go with that one, then things get very messy very fast, and not in a way that leads to some greater insight.
Actually, that may be it.
Our discussion assumed an unknown bias towards both sides.
GENERATION 705  992 i: The first time you see this, copy it into your sig on any forum. Square it, and then add i to the generation.
Re: MetaProbability Problem
Paradoxica wrote:Well one of the heads involved in the discussion claims that because every distribution has a "complementary distribution" (i.e. p <=> 1p) then the averages of the infinite probabilities gives a uniform distribution.
I'm skeptical because of the way infinities are handled.
This seems like a non sequitur. All the 1p values would be biased towards tails, so we don't care about them in this question, and in any case it suggests symmetry, not uniformity.
But we can use a different argument to support the same point. The relevant thing isn't the complementary values, it's the indifference principle which states that without reason to favor any possibility over another, you should treat them as equally likely (until more information comes along). So in this problem as stated, with no outside information, a uniform is the most "natural" prior. It's not hard to pose slight variants of the same question where other priors are more natural.
There's really two answers to this question. Under the Bayesian idea of probability as subjective confidence, the probability of getting a head and a tail (or whatever other combination of flips) is determined by your information about the coin; when you have no information it's determined entirely by your prior. Then the probability will change as you gain new information about the coin, for example by flipping it a bunch of times.
Under the frequentist idea of probability as proportion in the limit, the probability of getting a head and a tail is some specific exact quantity determined strictly by physical properties of the coin, and at the outset you just plain do not know what it is. You cannot answer the question at all. But as you gain new information about the coin, for example by flipping it a bunch of times, you can determine that the true probability is located in eversmaller intervals with everhigher confidence.
And clearly, in the limit these two probabilities should agree. But in the beginning, the Bayesian blurts out "one in three!" and the frequentist elbows him in the ribs and says "you can't know that!"
No, even in theory, you cannot build a rocket more massive than the visible universe.
Re: MetaProbability Problem
Wait, why are you guys getting 1/3? I don't see it (Warning: I never had a stats class, so I might have outrageous misconceptions):
Let's call [;p;] the probability of getting heads in one toss, the probability for tails is [;1p;]
The probability of getting two heads is then [;p^2;], and the probability for two tails is [;(1p)^2;]
We can then find the probability of getting headtails or tailshead by subtracting the found probabilities from [;1;]:
[;P_t_h=1p^2(1p)^2=1p^21+2pp^2=2pp^2;]
Then (and this is where it gets shaky), we can find the average value of [;P_t_h;] from [;0;] to [;1;] by doing a simple integral:
[;P_a_v_e=\int_0^1 \! 2pp^2 \, \mathrm{d}p=2/3;]
Which doesn't seem right, where is the error?
EDIT: Found it, when you learn integrals you forget that 1+1=2
[;P_a_v_e=\int_0^1 \! 2p2p^2 \, \mathrm{d}p=1/3;]
Let's call [;p;] the probability of getting heads in one toss, the probability for tails is [;1p;]
The probability of getting two heads is then [;p^2;], and the probability for two tails is [;(1p)^2;]
We can then find the probability of getting headtails or tailshead by subtracting the found probabilities from [;1;]:
[;P_t_h=1p^2(1p)^2=1p^21+2pp^2=2pp^2;]
Then (and this is where it gets shaky), we can find the average value of [;P_t_h;] from [;0;] to [;1;] by doing a simple integral:
[;P_a_v_e=\int_0^1 \! 2pp^2 \, \mathrm{d}p=2/3;]
Which doesn't seem right, where is the error?
EDIT: Found it, when you learn integrals you forget that 1+1=2
[;P_a_v_e=\int_0^1 \! 2p2p^2 \, \mathrm{d}p=1/3;]
Re: MetaProbability Problem
Sorry for the double post, but I was thinking, what would happen if the probability p of heads followed a normal distribution? Would the probability change?
Re: MetaProbability Problem
Squarian wrote:Sorry for the double post, but I was thinking, what would happen if the probability p of heads followed a normal distribution? Would the probability change?
In general, yes. If you come up with a new probability distribution, you should expect that the integral of 2p(1p) dF will change also. However be careful that the domain of your probability function matches your knowledge of values p can take. Normal distributions are defined for all real numbers, so that would be like saying it's possible that p<0 or p>1 (or indeed 0<p<0.5 which you assumed is not the case). Maybe you could truncate the distribution to match the range of p, but then it stops being a normal distribution.
Re: MetaProbability Problem
You'd run into a bit of a technical problem, because normals are over the entire real line, while probabilities are restricted to [0,1]. You could use some kind of truncated rescaled normal, though, or something that approximates that.
Unlike the uniform, there's more than one possible (approximated)normal you could use here. In fact, your choice of mean and variance are pretty much unconstrained, other than the [0,1] thing. And that would definitely change the probability of "a head and a tail". You could make it anything you want, really: by picking a small enough variance, you can almost guarantee that p will be very close to whatever mean you pick.
Unlike the uniform, there's more than one possible (approximated)normal you could use here. In fact, your choice of mean and variance are pretty much unconstrained, other than the [0,1] thing. And that would definitely change the probability of "a head and a tail". You could make it anything you want, really: by picking a small enough variance, you can almost guarantee that p will be very close to whatever mean you pick.
No, even in theory, you cannot build a rocket more massive than the visible universe.
Re: MetaProbability Problem
Meteoric wrote:You'd run into a bit of a technical problem, because normals are over the entire real line, while probabilities are restricted to [0,1]. You could use some kind of truncated rescaled normal, though, or something that approximates that.
Unlike the uniform, there's more than one possible (approximated)normal you could use here. In fact, your choice of mean and variance are pretty much unconstrained, other than the [0,1] thing. And that would definitely change the probability of "a head and a tail". You could make it anything you want, really: by picking a small enough variance, you can almost guarantee that p will be very close to whatever mean you pick.
I should mention that if you assume 0.5 ≤ p ≤ 1, then the mean μ must lie in that interval as well. The BhatiaDavis inequality also puts a constraint on the variance by saying that σ^{2} ≤ (μ0.5)(1μ). Other than that, you are free to pick whatever mean and variance you like!

 Posts: 35
 Joined: Wed Apr 09, 2014 1:33 am UTC
Re: MetaProbability Problem
Regarding Normals, just apply the argument [imath]\tan{\frac{\pi}{2}x}[/imath]
It has both real infinities and is completely continuous.
Of course, there's always [imath]\tanh^{1}{x}[/imath]
As well as all the other functions that can be artificially constructed to be absolutely less than unity in domain but cover the entire real line in output.
It has both real infinities and is completely continuous.
Of course, there's always [imath]\tanh^{1}{x}[/imath]
As well as all the other functions that can be artificially constructed to be absolutely less than unity in domain but cover the entire real line in output.
GENERATION 705  992 i: The first time you see this, copy it into your sig on any forum. Square it, and then add i to the generation.
 gmalivuk
 GNU Terry Pratchett
 Posts: 26825
 Joined: Wed Feb 28, 2007 6:02 pm UTC
 Location: Here and There
 Contact:
Re: MetaProbability Problem
But there's no motivation for any particular choice of adjustment, and you'd still have to adjust the result so that the total probablility sums/integrates to 1.
Re: MetaProbability Problem
Paradoxica wrote:As well as all the other functions that can be artificially constructed to be absolutely less than unity in domain but cover the entire real line in output.
gmalivuk wrote:But there's no motivation for any particular choice of adjustment
Might I suggest we pick the inverse of the normal CDF?
Who is online
Users browsing this forum: No registered users and 11 guests