Thursday, September 09, 2004

Poll Accuracy (National)

The question has come up on another site, about who is the most reliable poll. That’s a good question, so I decided to look into it. The National Council on Public Polls had a nice site up, where they have reviewed the poll results for Presidential elections since 1936, and they have posted their results. I have a slightly different methodology for counting accuracy, so my results are slightly different, but all in all the information produced useful trends.

OK, for this review, I am only going to count polls still in operation, since there’s already quite a few to check. For each election, I take the actual election result, and tick off a point for each point of deviation in the prediction in any direction on the final poll from each group.

With that plan, here’s the history:

2000: Actual election results, Bush 48%, Gore 48%

Zogby: Predicted 48-46 Gore, off by 2.
Gallup: Predicted 46-48 Bush, off by 2.
USA Today: Predicted 46-48 Bush, off by 2.
IBD/CSM/Tipp: Predicted 46-48 Bush, off by 2.
Pew: Predicted 47-49 Bush, off by 2.
Harris: Predicted 47-47 tie, off by 2 (same results for Harris Interactive),
ABC/WP: Predicted 45-48 Bush, off by 3.
NBC/WSJ: Predicted 44-47 Bush, off by 5.
Battleground: Predicted 45-50 Bush, off by 5.
ICR: Predicted 44-46 Bush, off by 6.
CBS: Predicted 45-44 Gore, off by 7.
Rasmussen: Predicted 49-40 Gore, off by 9.

(I notice I mis-remembered, Rasmussen was actually too low for Bush in his final poll!)


1996: Actual election results, Clinton 49%, Dole 41%

Zogby: 49-41 Clinton, off by 0.
Gallup: 52-41 Clinton, off by 3.
USA Today: 52-41 Clinton, off by 3.
CNN: 52-41 Clinton, off by 3.
NBC/WSJ: 49-37 Clinton, off by 4.
Harris: 51-39 Clinton, off by 4.
ABC News: 51-39 Clinton, off by 4.
ICR: 51-38 Clinton, off by 5.
Pew: 52-38 Clinton, off by 6.
Battleground: 45-36 Clinton, off by 9.
CBS/NYT: 53-35 Clinton, off by 10.


1992: Actual election results, Clinton 43%, Bush 37%

ABC News: 44-37 Clinton, off b y1.
NBC/WSJ: 44-36 Clinton, off by 2.
CBS/NYT: 45-37 Clinton, off by 2.
Harris: 44-38 Clinton, off by 2.
Gallup: 49-37 Clinton, off by 6.
USA Today: 49-37 Clinton, off by 6.


1988: Actual election results, Bush 53%, Dukakis 46%

Harris: 50-46 Bush, off by 3.
ABC News: 54-44 Bush, off by 3.
CBS/NYT: 52-43 Bush, off by 4.
Gallup: 56-44 Bush, off by 5.
USA Today: 52-42 Bush, off by 5.


1984: Actual election results, Reagan 59%, Mondale 41%

Gallup: 59-41 Reagan off by 0.
ABC News: 57-39, off by 4.
CBS/NYT: 58-37 Reagan, off by 5.
Harris: 56-44 Reagan, off by 6.
USA Today: 61-34 Reagan, off by 9.


1980: Actual election results, Reagan 51%, Carter 41%

ABC News: 46-41 Reagan, off by 5.
Harris: 46-41 Reagan, off by 5.
Gallup: 47-44 Reagan, off by 7.
CBS/NYT: 44-43 Reagan, off by 9.
NBC News: 42-36 Reagan, off by 14.


1976: Actual election results, Carter 50%, Ford 48%

Gallup: 49-48 Ford, off by 3.
ABC News: 48-47 Carter, off by 3.
Harris: 48-47 Carter, off by 3.
CBS News: 45-41 Carter, off by 12.


1972: Actual election results, Nixon 61%, McGovern 38%

Gallup: 62-38 Nixon, off by 1.
Harris: 61-39 Nixon, off by 1.


1968: Actual election results: Nixon 43%, Humphrey 43%

Gallup: 43-42 Nixon, off by 1.
Harris: 45-41 Humphrey, off by 4.


1964: Actual election results, Johnson 61%, Goldwater 39%

Gallup: 64-36 Johnson, off by 7.
Harris: 64-36 Johnson, off by 7.


1960: Actual election results, Kennedy 50%, Nixon 50%

Gallup: 51-49 Kennedy, off by 2.


1956: Actual election results, Eisenhower 58%, Stevenson 42%

Gallup
: 60-41 Eisenhower, off by 3.


1952: Actual election results: Eisenhower 55%, Stevenson 44%

Gallup: 51-49 Eisenhower, off by 9.


1948: Actual election results; Truman 50%, Dewey 45%

Gallup: 50-45 Dewey, off by 10.


1944: Actual election results, Roosevelt 54%, Dewey 46%

Gallup: 52-49 Roosevelt, off by 5.


1940: Actual election results, Roosevelt 55%, Willkie 45%

Gallup: 52-48 Roosevelt, off by 6.


1936: Actual election results, Roosevelt 61%, Landon 37%

Gallup: 56-44 Roosevelt, off by 12.




OK, here’s how they stack up, for length of record and aggregate accuracy:

1. Zogby: 2 elections, 1.00% error average.
2. IBD/CSM/Tipp: 1 election, 2.00% error average.
3. ABC News: 7 elections, 3.29% error average.
4. Harris: 10 elections, average 3.70% error average.
5. Gallup: 17 elections, average 4.82% error average.
6. USA Today: 5 elections, 5.00% error average.
7. ICR: 2 elections, 5.50% error average.
8. NBC News/Wall Street Journal: 4 elections, 6.25% error average.
9. CBS News/New York Times: 7 elections, 7.00% error average.
10. Battleground: 2 elections, 7.00% error average.
11. Rasmussen: 1 election, 9.00% error average.

Zogby, IBD, Harris, ICR, NBC News, and Rasmussen provide no demographic data.

ABC News, CBS News provide limited demographic data.

Gallup, USA Today, and the Battleground polls provide extensive demographic data.

This provides a background for reviewing poll releases. Note that there are eleven national polls out this year, which do not have a known track record, including TIME and Newsweek magazines, the American Research Group, NPR, the LA Times, Quinnipiac University, and Marist University.

2 comments:

Anonymous said...

I have also noticed that the actual winner is always picked more times by the poll than the loser. In other words it appears to be a trend that whomever is winning the most polls will be the winner.

luke.thmpsn said...

The polls all seem to have some error i would like to see a variance or standard deviation result because the averages don't say what might happen if the polls are all incorrect in one direction say, in this election an over active democrat base that might not vote in the election or a higher level of poll activity by certain groups

I beleive McCain is closer than everyone thinks