Why voter surveys don't agree

To casual consumers of campaign news - and that would be most voters - the past week may have been particularly perplexing.

First the Pew Research Center, a respected nonpartisan polling outfit, showed President Bush ahead of John Kerry by just one point among likely voters nationwide. Then Gallup, another major brand in polling, showed Mr. Bush ahead of Senator Kerry by 13 points, also among likely voters. Other polls showed either a dead heat or a Bush lead, but smaller than the Gallup result.

Pollsters and experts flooded the zone with explanations: Bush's convention bounce is fading. The electorate is more volatile than previously thought. And, most fundamentally, don't read too much into any one poll. As the cliché goes, polls are just a snapshot in time - and sometimes the lens isn't quite in focus.

It's also worth noting, pollsters say, that it's still too early to come to any firm conclusions about who will win in November.

In fact, many of the voters who may well determine the outcome - those who have not locked in with a candidate - aren't fully engaged yet in the race.

"I would urge most Americans not to pay attention to polls," says Karlyn Bowman, an expert on polling at the American Enterprise Institute in Washington. "It seems to me, most of them are being done for the elites in campaigns and for journalists."

For the voter who does want to pay close attention, Ms. Bowman advises going to a polling firm's website and looking at the methodology. How big is the sample? Over how many days was a poll taken? Do they weight for party identification, and if so, how? She also advises looking at the trend over time within one polling organization - and not to compare among different polls.

It's also debatable whether polls can become self-fulfilling prophecies. That is, if there's a major poll showing one candidate down by a significant margin, will that discourage his voters to the point where some may think it's not worth their effort to turn out?

While there's no statistical evidence to prove that this happens in America's long-drawn-out elections, campaigns don't want to take any chances.

The Kerry campaign has played down the latest Gallup poll, calling it an "outlier." In a conference call with reporters last Friday, Kerry pollster Tom Kiley looked at the seven polls released over recent days - including his campaign's own poll, which showed the president up by four points - and came up with an average that shows Kerry not doing badly.

"If we do not include the Gallup survey, the average of all these polls points to a two-point race, with the president ahead by two; obviously it's a very tight race," Mr. Kiley said.

From the Bush perspective, while a huge lead in any poll is welcome news, it's also not in the campaign's interest to buy into a message of "we're way ahead." All along, a Bush-team mantra has been that this race will come down to the wire, and the base must remain energized. The president can't win if his supporters become overconfident and stay home on Election Day.

The latest Pew poll provides stark evidence that the electorate may be more volatile than previously thought. The survey of 1,972 registered voters was taken in two waves. The first group, polled Sept. 8-10, gave Bush a whopping 54-38 percent lead among likely voters. But by the second group, surveyed Sept. 11-14, that lead had vanished, and the race among likely voters showed Bush up by just 1 point, 47 to 46 percent.

Pew describes crosscurrents in opinion that could push swing voters first one way, then another. The pounding that Kerry took during August and at the GOP convention in New York clearly hurt the Democrat, while Bush remains vulnerable on Iraq and the economy, the Pew report said.

But even with a poll whose top line shows a statistical tie, it's possible to give one side an edge, says Raghavan Mayur, president of the TIPP poll, which conducts surveys for this newspaper. His latest poll, released Sept. 14, showed Bush and Kerry tied at 46 percent each. But Mr. Mayur still sees Bush as slightly ahead, based on the intensity of his voters.

"Whenever there is a tie, what I do is look at the next layer of statistics to see how the wind is blowing," he says.

All polls vary in their methodology, but perhaps none is more subjective than weighting for party identification - a practice some follow and others don't. Over the past three decades, the Democratic lead in voters' self-identification with that party has declined compared with Republicans; some pollsters factor in evolving voter ID when they weight their samples while others don't.

John Zogby, whose eve-of-election polling accurately predicted the final outcome in both the 2000 and 1996 presidential races, does weight his samples according to his best sense of how voters identify themselves. "We all use artistry; the difference is I admit it," says Mr. Zogby. "I'm a historian and when you're dealing with people, it's not all statistics."

Another factor that has made polling increasingly difficult is the public's declining willingness to give pollsters the time of day. The new do-not-call registry does not apply to pollsters, but it has emboldened the public to ignore, or reject with expletives, their requests for interviews. Proliferation of caller ID also hurts pollsters.

The fact that cellphones are off limits to pollsters also hinders outreach into certain demographics, such as young people, who in growing numbers have no land lines.

But young people - as well as minorities - have always been more difficult for pollsters to reach than other groups.

Below are results of three recent nationwide polls of likely voters, showing only responses for George Bush or John Kerry.

The Christian Science Monitor/TIPP (Sept. 7-12)
Kerry: 46%
Bush: 46%

Harris (Sept. 9-13)
Kerry: 48%
Bush: 47%

Gallup (Sept. 13-15)
Kerry: 42%
Bush: 55%