Polls, polls, polls! Everyone loves polls! Leading up to elections, we get lots of the little suckers. Yet, it seems that most people don’t understand them. Especially the people who report them.
I am here to help, because this just isn’t that complicated.
The issue is the poll’s margin of error. I was just reading an article over at the every good P M Carpenter blog, The NBC/WSJ poll reveals that sometimes a cigar really is a cigar. In it, he states that in a new poll, Obama is beating Romney by 5%, which is larger than the margin of error. (To be fair, Carpenter clearly does understand polls.)
I’m afraid that to most people, the margin of error means too much and too little. The fact that Obama is beating Romney by more than the margin of error does not mean that more people are necessarily planning to vote for him. Just the same, if Obama were beating Romney by less than the margin of error, it would still mean that it was very likely that more people were planning to vote for him.
Most margins of error are reported as 95% confidence intervals. This means that there is a 95% chance that the real number is within the margin of error. It also means that there is a 2.5% chance that the real number is one margin of error lower than this value. And there is a 2.5% chance that it is that much higher.
The NBC/WSJ poll finds that 50% of likely voters support President Obama. There is a 3.3% margin of error. This means there is a 95% chance that Obama’s actual support is between 46.7% and 53.3%. But there is a 2.5% chance that his actual support is less than 46.7% and a 2.5% chance that it is more than 53.3%.
Now let’s suppose that Obama were only ahead of Romney by 1.65% (half the margin of error). This is the one-sigma confidence limit and it indicates 66% certainty rather than 95%. If Romney similarly polled at 48.35% (half the margin of error below Obama), there would be a 66% chance that Obama’s real number was between 48.35% and 51.65%. Plus, there would be a 17% change that Obama’s real number was above 51.65%. And a 17% chance that it was below 48.35%. Thus, there is still an 83% chance that Obama is beating Romney.
The take away here is that being ahead but not by as much as the margin of error is still a very good thing. It doesn’t mean that the two candidates are tied. But just the same, this is statistics and it is possible for any given poll to be way off—even if it is a bigger difference than the margin of error.
Afterword
Polling is very difficult. The math is all fine, but there are all kinds of sampling problems that can make the math irrelevant. The best known example of this is the 1948 Dewey Defeats Truman debacle. The problem was that the polls had been based on telephone surveys when only more affluent (i.e. conservative) people had them. Now, the biggest issue is for polling organizations to figure out what exactly a “likely voter” is. This is why election day polling is so accurate—unless George W. Bush is running.
