### Margins of error

Matt Yglesias writes:

The thing is that there's no such thing asthemargin of error, just different confidence levels you could be using in your analysis. 95 percent is traditional in the US but there's no reason handed down from the Lord on High that all statistics need to be done this way.

No, but there's no free lunch.

A poll is a small sample of a much larger population. We'd really like to know what every voter in America thinks, but we can't ask them all, so we select a random sample voters and ask them. Let's say that 51% of the people we surveyed say that they support Kerry and 49% for Bush.

The skeptic will say, "But you only surveyed .0001% of the voters. How do you know that the ratio of Kerry supporters to Bush supporters in your sample is the same as in the population?"

The statistician will answer, "First off, I selected a random sample--but that doesn't guarantee that the sample look exactly like the parent population. When you flip a coin, sometimes you'll get a run of heads. Maybe we just got a run of Kerry supporters in our sample.

Luckily, we know something about probability distributions. We can calculate how likely it is for a sample score to diverge from its parent population by such and such an amount purely by chance. In this (hypothetical) case, we expect that the sample score will be within +/- 5 points of the real score 19 times out of 20."

The skeptic says, "Why a margin of error of +/- 5? Why not set the margin of error to +/- 2.5?"

The statistician replies, "What would be the point? We *could* apply the +/- 2.5 criterion to these data, but then we'd only expect the sample score to be that close to the population score N times out of 20 [where N is less than 19, but I'm too lazy to do the calculations]. By the same token, we expect the sample score to be within +/- 5 points of the population score 19 times out of 20--after all, it's the same sample from the same population. So, we always use the 95% confidence interval because that's the standard way to present the data.

"

## Comments