Andrew Prokop wrote an excellent article over at Vox yesterday, Why Election Forecasters Disagree About Who Will Win the Senate. It gives far too much attention to Nate Silver, but it still manages to provides little nuggets of information that I had been wondering about. For example, as I had thought, The Monkey Cage model used to be fully fundamentals based. Now that we are so close to the election, it is mostly polls based. That’s why a couple of months ago, it gave the Republicans an 86% chance of taking the Senate and now it only gives them at 53% chance.
But overall, the article doesn’t provide any surprising information. Basically, the differences in the conclusions of the models is based upon just how much each model uses fundamentals like the approval rating of the President. According to the articles, the modelers have looked at the effect of fundamentals in the past at certain points in the election cycle and they have found that they really do predict better than poll-only models. I think there is a major problem with this claim.
These modelers have not looked at the effects of fundamentals-based models before the fact. They’ve developed models based upon what they’ve seen in the past. Basically: they do multiple regressions and see how much each element affects the final result. So of course these models would predict the past better than the polls-only models. The fundamentals-based models have the conclusions built into them to some extent.
No one thinks that these multiple regressions are the truth. There is no theoretical basis for the belief that the President’s approval rating will affect the election outcome. It is simply a tendency that has been experimentally derived. And it turns out that generally the President’s approval rating is correlated with a “generic ballot” where people say who they would rather vote for: a generic Democrat or a generic Republican. Well, as Nate Cohn explained last week, these two have diverged. Obama’s approval rating is very low at about 40%, but the Democrats have a three percentage point advantage in the generic poll. For contrast, the Republicans had a three point advantage in 2010.
What I wonder about is if the fundamentals aren’t generally useless in off-year elections. Even in presidential elections, there is really only one important fundamental: economic growth. The change in GDP for the three quarters before the election is responsible for almost half of the variation in how well the parties do. My hunch is that all the other fundamentals are just added noise. Yes, they’ve correlated in the past, but it isn’t real. And that takes us to Sam Wang’s conclusion, “Because polls are a direct measure of opinion, and fundamentals are indirect, it’s generally a risky move to add fundamentals to a measurement.” In other words: fundamentals just add noise to the models.
It is important to remember that these are just models. What’s more, the polling data itself could be wrong. This isn’t a presidential election with nationwide polling. The competitive states are getting decent polling coverage, but they are hardly what we would want. And in the end, it will all come down to who shows up to vote. Right now, the polls are mostly of registered voters. Soon they will provide “likely voter” estimates. As a first brush, this tends to help the Republicans. Just the same, in 2012, many “likely voter” estimates overstated Republican voters because roughly half of the people who said they were unlikely to vote ended up voting anyway. So it is a muddled mess.
I’m back to what I thought a year ago: the Republicans have a 50-50 chance of taking over the Senate. And I’m kind of gloomy on that issue, because I’ve trained myself to be pessimistic in situations like this. Even in 2012 when I predicted that Obama would win 332-206 (which is what he won by), I wasn’t willing to say that Obama was certain to win. The truth is that we just don’t know. But we ought to know the evening of 4 November.
