Is there any point to these simulations?
Here is John Hollinger's description of how he gets his NBA playoff predictions:
"As always, the output of a product is only as good as its input, so let's explain a little about how this is derived. The computer starts with the day's Hollinger Power Rankings. Then, in each of the 5,000 times it replays the season, it makes a random adjustment up or down to allow for the possibility that a team will play better or worse than it has done thus far."
I've been trying to think this through: why run simulations at all? The power rankings must establish some relationship between teams, such that, say, when a 94.5 plays an 86.7 it will likely win by three points, or something like that. Now you can produce some random wiggles and determine what the likelihood 94.5 will win is. Then use similarly derived likelihoods for all remaining games to get all of the teams' final records.
In other words, my first impression here is that the run of 5000 simulations is just reproducing the data from the power rankings in its outcomes, so why not move straight from power rankings to outcomes?
Or look at it another way: if I know I have a fair "coin" (which is perhaps a computer algorithm proven to produce heads and tails with equal probability), I don't need to run 5000 "simulations" to "see" what the likely outcome of repeatedly tossing it will be: it will be 50-50. Of course, if I do run the program 5000 times, the "answer" I get is likely to be pretty accurate, but that is because I had a fair coin to begin with. The runs have not told me anything new, and if they produce 47% heads and 53% tails, that does not mean that 47-53 is really the likely long-run outcome.
Does anyone have any idea as to why one would run simulations in a case like Hollinger's?
"As always, the output of a product is only as good as its input, so let's explain a little about how this is derived. The computer starts with the day's Hollinger Power Rankings. Then, in each of the 5,000 times it replays the season, it makes a random adjustment up or down to allow for the possibility that a team will play better or worse than it has done thus far."
I've been trying to think this through: why run simulations at all? The power rankings must establish some relationship between teams, such that, say, when a 94.5 plays an 86.7 it will likely win by three points, or something like that. Now you can produce some random wiggles and determine what the likelihood 94.5 will win is. Then use similarly derived likelihoods for all remaining games to get all of the teams' final records.
In other words, my first impression here is that the run of 5000 simulations is just reproducing the data from the power rankings in its outcomes, so why not move straight from power rankings to outcomes?
Or look at it another way: if I know I have a fair "coin" (which is perhaps a computer algorithm proven to produce heads and tails with equal probability), I don't need to run 5000 "simulations" to "see" what the likely outcome of repeatedly tossing it will be: it will be 50-50. Of course, if I do run the program 5000 times, the "answer" I get is likely to be pretty accurate, but that is because I had a fair coin to begin with. The runs have not told me anything new, and if they produce 47% heads and 53% tails, that does not mean that 47-53 is really the likely long-run outcome.
Does anyone have any idea as to why one would run simulations in a case like Hollinger's?
"…which is perhaps a computer algorithm proven to produce heads and tails with equal probability…"
ReplyDeletePRNGs aren't probabilistic, but I'm sure you already know that.
Is this relevant to the issue of the simulations?
DeleteWe could estimate final records fairly easily without the simulation, but computing the odds of making the playoffs, getting to the finals, etc. would be much more challenging to do analytically.
ReplyDeleteWhat is the probability that OKC will make the playoffs, i.e. end up with a better record than at least 7 other teams in its conference? The number of possible outcomes is so huge that it's probably more efficient to simulate 5000 times than to enumerate and sum the probabilities.
I think that must be it, Matt.
DeleteYes. A monte carlo simulation is often the best way to get a close solution to a difficult set of contraints or equations which have no closed form solution.
Delete