An Interesting Bit of Probability Theory
"There's a 0% chance that X will happen."
Most people interpret that to be equivalent to, "X will never happen."
But that isn't so. An example: If one were to toss an infinitely fine-pointed dart at the real number line and hit a spot between 0 and 1, for any particular number in that range, there is a 0% chance you will hit it. But you *will* have hit *some* number, so even though there was a zero percent probability that number would be hit, it was hit.
Most people interpret that to be equivalent to, "X will never happen."
But that isn't so. An example: If one were to toss an infinitely fine-pointed dart at the real number line and hit a spot between 0 and 1, for any particular number in that range, there is a 0% chance you will hit it. But you *will* have hit *some* number, so even though there was a zero percent probability that number would be hit, it was hit.
It's not actually zero, but positive infinitesimal.
ReplyDeleteI don't know, Huff, I've now seen four math PhDs assert it *is* actually zero, including two right below here, since an NYU econ degree is essentially a math degree.
DeleteAlright; if the math Ph.D.'s are on your side, I should probably wave the white flag. But I have a hunch that the proof of this involves the assumption that the probability must be a real number between 0 and 1 (inclusive).
DeleteDavid K.'s proof below relies on the assumption that the probability is a real number. But the position of the other side is that the probability is not a number (i.e., that the probability is undefined).
DeleteJust to clarify Gene, actually an econ PhD, even from NYU, is hardly a math PhD. The actual math people at Courant would blow the econ kiddies up, in terms of pure math.
DeleteHowever, I am knowledgeable enough to say that your post here is completely standard. Judging from the comments here, the authors of probability textbooks are all Keynesians.
Let me clarify: I regard your PhD as being in a branch of applied mathematics. Yes, I agree, that is still very different than a PhD in theoretical mathematics. But you had to learn a hell of a lot of math.
Delete@ P.S. Huff: "But the position of the other side is that the probability is not a number (i.e., that the probability is undefined)."
DeleteThe probability that the number hit by the dart will lie in a set E is well-defined if and only if E is measureable, and {0.5} is certainly measureable (I assume we are using the Lebesgue measure).
Yeah learning that kind of math stuff was one of the few things that helped me maintain my sanity in grad school. I remember one time I got saucy and tried to show off in front of the wiz micro professor in his office hours. He was referring to something that was "measure zero" in the probability space, and I said "Yeah these events are only countably infinite," and he corrected me, "Well no, you could have a set with uncountably infinite elements that still has measure zero." I think I offended him.
ReplyDeleteThey also say that two lines that are parallel are supposed to cross somwhere at infinity...
ReplyDeleteThis is presumably why people keep playing the lottery.
ReplyDeleteYour example is a nice illustration of the fact that measure is only countably additive.
I suppose that would be true if infinitely fine-pointed darts existed, but they don't, so the point you're making is, in any practical sense, essentially meaningless.
ReplyDeleteIf I was making a practical point, Neil, your comment would be spot on, but since I was making a theoretical point, it is essentially meaningless.
DeleteSo, if you told a person, "There's a 0% chance that X will happen," and then X happened one month later, would you attempt to argue that your prediction wasn't necessarily wrong?
DeleteWell, certainly, Neil: that an event occurs does NOT, as we have seen, make a 0% chance prediction wrong.
DeleteYou should definitely go into politics.
DeleteYes, Neil, anyone who speaks with mathematical accuracy is certainly a tricky political type!
DeleteBy the way, as Pedro notes below, this applies to continuous distributions: if the Knicks were playing the Lakers, and I said "There is 0% chance the Knicks will win," I would certainly admit I was wrong if they won!
Well in practice there exist no real continuous distributions (100% of the real numbers by the way would be impossible to even write down...). There is also a certainly non zero probability that the initial predictions of 0% was wrong witch makes it certainly wrong.
DeleteI'm not a mathematician but my intuition is that events with 0 probability can at least in theory happen, but only as long as you haven't enumerated them before the event.
Lets also not forget that maths is a tool, and you can make up any mathematical theory you want, which doesn't mean it needs to apply to the real world, it is just intuition and empirical evidence that make certain theories applicable, and thus certain axioms and their provable consequences useful.
It's certainly an interesting paradox to think about...
"Well in practice there exist no real continuous distributions (100% of the real numbers by the way would be impossible to even write down...)."
DeleteFela, what are you talking about? It's also impossible to write down all of the integers (in finite time). It is also impossible to write down a name for each subatomic particle in the universe. Do you want to contend that therefore those particles don't exist?!
"There is also a certainly non zero probability that the initial predictions of 0% was wrong witch makes it certainly wrong."
But this is not a prediction at all. We're talking axiomatic probability theory here.
Well, if I understand 'measure zero' properly, it refers to a set that is zero for practical purposes. Not *actually* zero.
ReplyDeleteAnd if I understand your post properly, you are arriving at the 0% by dividing 1 by infinity.
The answer, then, should be some infinitesimally small--but non-zero--number.
But then again, I don't have a PhD in math (or econ from NYU). So feel free to correct me.
Well, unknown, "1 in a quintillion" refers to something that, for practical purposes, is impossible, but everyone recognizes there is some chance it could happen. But not so for 0% probability.
DeleteSo, unknown, why don't you tell us what that itsy-bitsy number IS, and win a major mathematical prize in the process? Hint: you can't divide by infinity, but you can take the limit of 1/x as x goes to infinity. What is that limit?
"And if I understand your post properly, you are arriving at the 0% by dividing 1 by infinity."
DeleteHere's the usual proof:
By assumption, all real numbers between 0 and 1 have the same probability p of being hit by the dart. Suppose, for contradiction, that p is greater than 0. Then there is a natural number n such that n*p is greater than 1. Choose n real numbers a_1, ..., a_n between 0 and 1. Then the probality that one of the numbers a_1, ..., a_n will be hit by the dart is n*p. But a probality cannot be greater than 1. We have arrived at a contradiction. Hence, p is 0.
Thanks, David K. I actually tried to paste that proof in to my reply from another site, but I'm blogging from my iPhone right now, and Blogger doesn't seem to allow pasting from an iPhone into a comment box!
DeleteFrom what I remember from my high school and calculus 100 classes, what "as x approaches infinity, f(x) approaches 0" implies is that the closer x gets to infinity, the closer f(x) gets to 0. But, x never reaches infinity--so f(x) never reaches 0. That's why when you plot 1/x, the function asymptotes at 0.
DeleteI know 1/infinity is *defined* as 0. But that's only because, as you said, it's impossible, as far as we can tell, to divide by infinity. So to me, what it seems like you're doing is this:
1. 1/infinity is defined as 0.
2. Therefore, on an infinite line, the probability of hitting any specific point is 0(=1/infinity[or the limit of 1/x as x->infinity, whatever])
3. But the probability of hitting any point is 1
4. And a specific point is within the range of all points, you can hit that specific point--even though the probability is defined as 0!
So *your* scenario of a 0% chance event is *defined* as having a probability of 0 (1/infinity). Whereas most people, I think, when they say 0%--outside of hyperbole--refer to a situation where the numerator or denominator is zero.
Do you see where I'm coming from? Am I making any sense
PS, I don't know why Google's putting me as Unknown. I'm Ash.
Ash, see the proof offered by David K.
DeleteOh, and as noted, 1 / infinity is certainly not defined as zero!
DeleteI talked to my former econometrics prof today about this. He immediately agreed with you. When I told him I didn't get it, he showed me a neat way to think about it:
DeleteSay X is 0.5. Hitting anywhere on the number line 100%. But, at the same time, the probability of hitting anything-but-0.5 is *also* 100%, we have to conclude that hitting 0.5 is 0%!
I also showed him David K.'s example, and he was as confused as I was by it.
But even after the fact, you could not make the statement "the dart hit X", because X would be some trancendental number that would be impossible to specify exactly.
ReplyDeleteAsh:
Delete1) What does being able to say where the dart hit have to do with the problem?
2) Why do you think transcendental numbers are "impossible to specify exactly"? Both pi and e seem fairly exactly specified to me.
For the record, this is not me (Ash). I think it's a glitch with Blogger/Google.
DeleteHi, it's Antony, I made the comment.
DeleteMy point was that, because "X" cannot be specified, if you make the statement "the dart will hit number X", that will never happen, even if you throw the dart an infinite number of times.
We know that numbers like pi and e exist, and they have certain mathematical properties, but we can't describe them algebraicly, and we can only ever specify their value to limited degrees of precision. The same applies to any number the dart will hit.
If you're trying to figure out where the dart hit, you can imagine that you keep zooming in and zooming in, getting more and more precise ranges of where it is. But anytime it looks like it's near a number that can be specified, as you zoom in more and more, it will be slightly off to one side or the other. Or imagine that you are writing the number down, the digits after the decimal would go on forever with no predictable pattern, like pi or e.
OK, Antony, I think there are several things being conflated here: the "specifiability" of a number, our ability to write out the number in decimal form, our ability to measure precisely along the real number line, and a purely conceptual question in probability theory.
DeleteLet me specify (using the term the way you are) pi for you as precisely as you can specify 1: I ask you to draw a line of length one. Then I make a circle with that as its radius. I say, "As precisely as you made that line of length 1, I have made a circle of area pi."
Yes, I can never write out its full decimal expansion, but when it comes to measuring, you can't do that for 1 either: you will have measured 1.00000000000, or whatever, but you'd have to measure an infinite number of zeroes to say "It really is exactly 1."
So the transcendental number complaint here is a red herring. We can never measure precisely where we are on the real number line.
Now, please direct your attention here.
Gene, what you write is true if you're working with continuous distrbutions (which you are, because you're talking about real numbers). Then probabilities are calculated with integrals. But you can work in the discrete realm too, and then a probability of 0 does mean an impossible event. With continuous distributions, you calculate a probability by calculating the integral of the probability density function between the two points. If the two points are the same, the probability is 0, and yet the event is not impossible.
ReplyDeleteI suspect this is related to the definition of an integral and of probability as an integral. Something similar happens with series. For example, the sum of 1 + 1/2 + 1/4 + 1/8..., which never reaches 2, is defined as being equal to 2.
In the comments above, someone said that 1/infinity is defined as 0. It's not so. The limit of 1/n, when n tends to infinity, is 0. This is not the same as saying 1/infinity is 0. Infinity is not a number, and therefore, 1/infinity doesn't exist.
Sorry if the terms used in my explanations are not the ones that are commonly used in English. I didn't learn this stuff in English.