The problem with predicting presidential elections

Even though there is data galore today, political oddsmakers actually don’t have much material to work with.

Here’s a New Year’s resolution for political pundits: Be very, very cautious when making predictions about the presidential election. The reason for caution is the “small-N problem.”

The letter “N” stands for the number of past cases on which you base your forecast. All other things being equal, the bigger the “N,” the more confidence you can have. When insurance companies set rates for collision coverage, they can analyze the experience of millions of people who have driven billions of miles. Accordingly, they can draw up accurate projections of claims, which is why they make so much money.

Political oddsmakers have a lot less material to work with. Since George Washington, there have been only 57 presidential elections in all. For the first nine elections, there are no reliable records of the popular vote, so we have tallies for just 48 cases. Most methods of election forecasting require data from public opinion surveys, which did not exist in their modern form until the 1930s. So if we start with 1936, when Gallup began its presidential election polls, we are down to 20 cases. Many models also involve economic data that are unavailable for election years before 1948, leaving a grand total of 17.

Resting on such a small number of cases, election forecasting models simply cannot be precision instruments. At best, they can give us some rules of thumb. If the economy is bad and the president’s approval rating is low – think of Jimmy Carter in 1980 – then the out-party will probably win. Conversely, if the economy and presidential approval are both strong – think of Ronald Reagan in 1984 – then the in-party is a heavy favorite.

Things such as economic performance and presidential approval are themselves hard to predict far in advance. At the moment, we do not know how the world will look in the summer and fall of 2016. Maybe the economy will surge and foreign-policy triumphs will push President Obama’s ratings well north of 50 percent. In that case, the Democrats have a good shot at holding the White House. Or maybe there will be pain all around, meaning that the advantage shifts to the GOP. Or maybe the news will be one big “meh,” in which case the models are not sturdy enough to tell us what will happen.

Predicting the party nominees is even trickier because the small-N problem is more severe. The current nomination process, consisting mostly of primaries and caucuses, dates back only to 1972. On the GOP side, incumbent presidents did not face major intraparty challenges in 1972, 1984, and 2004, meaning that there have been eight real nomination fights. Bill Clinton (1996) and Barack Obama (2012) had uncontested renominations, so the Democratic total is nine.

As Yoda would say, single digits do not an iron law make.

Consider the “next in line” theory of Republican presidential nominations. Without a GOP incumbent, the theory held, the party turns to the candidate “next in line,” somebody who has sought the nomination before. Examples include Mr. Reagan in 1980, George H.W. Bush in 1988, Bob Dole in 1996, John McCain in 2008, and Mitt Romney in 2012. In 2016, the two candidates with a claim to be next in line are Mike Huckabee (who finished second to Senator McCain in the 2008 delegate count) and Rick Santorum (the last candidate standing against Mr. Romney in 2012). But neither had enough support to appear on the main stage in the last GOP debate, and both seem headed for oblivion.

And then there’s the problem of predicting a nomination contest where a leading candidate has billions in personal wealth and universal name recognition stemming from years as a reality-TV star. The number of precedents is exactly zero, which is the ultimate small-N problem.

Jack Pitney writes his "Looking for Trouble" blog exclusively for the Monitor.