I’m a little embarrassed to be so fascinated by the argument between Nate Silver and Sam Wang. It was only three days ago that I wrote, The Ballad of Sam and Nate. But it just keeps on going. The most recent round took place on Political Wire, Nate Silver Rebuts Sam Wang. And then this morning, Sam Wang Factchecks Nate Silver. If I weren’t so interested in it, I would tell them both they ought to find a nice quiet boxing ring somewhere.
Today, over at The Daily Beast, Daniel Altman came to much the same conclusion I did over the weekend, Why Is Nate Silver So Afraid of Sam Wang? For most of this, I think you are better off just reading my article. One thing I noted was, “The truth is, at times, The Monkey Cage model has been a big outlier.” But Altman went one step further. He pointed out that at the time he was writing, Silver’s model (FiveThirtyEight) actually disagreed more with The Monkey Cage than it did with Wang’s model (PEC).
At that time, the FiveThirtyEight model predicted a 58% chance of the Republicans taking control of the Senate. The Monkey Cage predicted 77% and PEC predicted 42%. Thus FiveThirtyEight predicted 19 percentage points lower than The Monkey Cage and 16 percentage points higher than PEC. Today, it is even worse. FiveThirtyEight still predicts 58%, but the others have moved toward the extremes: 84% and 36%. Thus, FiveThirtyEight is 26 percentage points lower than The Monkey Cage and 22 percentage points higher than PEC.
I stick with what I said over the weekend: I like the inertia in the FiveThirtyEight model. And I tend to think that Silver has things right. And if we include the Daily Kos and Huffington Post models to provide the five primary Senate models, we end up with a median that is the FiveThirtyEight model (58%) and a mean that is closer to the FiveThirtyEight model than any other: 60%. Even if we throw out The Monkey Cage and the PEC models as outliers, we get almost the exact same conclusions. The median, of course, is still the FiveThirtyEight model (58%) and the mean is 59%.
So by using the wisdom of the crowd approach that Sam Wang uses with polls, but instead use it with the Senate models, the FiveThirtyEight model is the winner. Of course, that just makes Sam Wang look better. If he were not one of the “big five” models with his most basic — statistics only — model, then what the modeling community as a whole would be telling us would be way off. The median would be 63% and the mean would be 66%. That would be closest to the Daily Kos model, which currently gives the Republicans a 67% chance of taking the Senate. (I wonder about the Daily Kos model, however; it has been stuck at 67% for a very long time, during which time there have been major changes in the polls.)
Daniel Altman’s main point is the same as mine: if the Democrats manage to hold the Senate, it will make Nate Silver look bad, even though it doesn’t mean what people will think it means. And the fact that Wang is doing it with the most boneheaded model imaginable won’t speak well for wonderboy. It will say to many that there is a very simple way to separate the signal and the noise. But maybe this whole election cycle is teaching us something more important. Maybe it’s teaching us that Nate Silver has a really big and obnoxious ego. Just kidding; we already knew that!
What we may be learning is that the best way to model elections is for a lot of smart people to create models the best way they can think of. And based upon this, we not only get a good idea of how the election will turn out, we get a better idea of what matters and what doesn’t. And if Nate Silver gets his fee-fees hurt, so be it. The world is generally improved by large groups of people doing things; it is rarely improved because one Romantic hero steps up and shows us the way. And even if Nate Silver’s model is the best, it just shows that he’s picked the best of The Monkey Cage approach (fundamentals only) and the PEC approach (polls only). That’s the signal. The rest is noise.
Afterword
I screwed up and forgot to include The Upshot model, which currently gives the Republicans a 61% chance of taking the Senate. It is very similar to the FiveThirtyEight model. Adding it to the analysis changes the numbers very little. However, it does tip things toward The Upshot so that the mean of all the models is closer to it. If PEC is removed, the median is exactly The Upshot and the mean is still the Daily Kos. I think the reason I forgot The Upshot model is that I don’t think it is really any different from the FiveThirtyEight model.
