Probability

Recently, Nate Silver won acclaim by correctly predicting the electoral results for all fifty states. If one of Silver's predictions had failed, however, would that have shown that he was wrong? I mean, I take it that Silver's predictions amount to assignments of probability to different outcomes. Suppose that I claim that an ordinary coin has a 50% chance of landing head or tails. If a trial is then run in which the coin lands tails three times in a row, we wouldn't take this to mean that I was wrong. Along similar lines, then, would it not have been possible for literally all of Silver's predictions to have failed and yet still be correct?

Comments

Right, as Silver himself would be the first to agree. However, we might want to put it a bit differently. The projections could all be mistaken, but not because his methods or premises were incorrect. Here's a way to see the general point.

Suppose we consider 20 possible independent events, and suppose that for each, the "correct" probability that the event will happen is 95%. (I use shudder quotes because there's an interesting dispute about just what "correctness" comes to for probability claims, but it's a debate we can set aside here.) Then for each individual event, it would be reasonable to project that it would occur. But given the assumption that the events are independent, the probability is over 64% that at least one of the events won't occur, and there's a finite but tiny probability (about 1 divided by 1026) that none of the events will occur. So it's possible that all the projections could be reasonable and all the probabilities that ground them "correct," and yet for some or all of the projections to fail. A statistical prediction/projection doesn't amount to saying that the projected event is bound to happen.

In the case of Silver's projections, the details are more complicated. For example: he projected not just who would win each state, but by how much, and he attached margins of error to his projections. But the general point remains: his methods could be sound and yet since he was making statistical predictions rather than outright claims about what would happen, any number of projections could have failed even if his methods were sound. On the other hand, if his projections failed badly, that would be good reason to think the problem was in the model rather than the caprice of the election gods. Such is our fallible lot.

Read another response by Allen Stairs

Read another response about Probability