Poll Bias: What the Numbers Say

I started by trying to figure out the bias averages of the national polls (OK, I really wanted to try to figure out the bias in state polls, but there just wasn’t a large enough sample to do that). Along the way, though, I found out some other interesting things, and being the gabby sort, I decided to share. Some of it might be useful in examining the polls, as we enter the post-Convention stretch.


I began by noting the 2-way poll results from Polling Report and Real Clear Politics from February 3 to the end of July. I made a spreadsheet with the name of the group or agency which conducted the poll, the starting and ending dates of the polling, and the results for Bush and Kerry. From this information, I drew up another spreadsheet which produced the daily average for the polls. I then compared each poll against the average for the days polled, to see where each poll stood against the average, noting whether Bush or Kerry was getting more or less support than the average in a poll. I then was able to apply that information to determine the average bias, the aggregate variance, and essentially the stability of each poll.


I want to stop at this point, and make an observation about the fairness of these polls. As polls make statements which favor Bush or Kerry, it’s becoming common to see this poll or that dismissed as being unreliable or unbelievable. This is unwise, because in examining the polls presented, I was struck by how well, overall, every poll was conducted. While I have concerns about the demographics base used, and the weighting assumptions inherent in modern polling, the conclusions of each poll were consistent with other polls at the same time. Since thirty-one national agencies or groups have conducted national polls cited in the two sites, such consistency demonstrates either a massive conspiracy, or an impressive level of professional standards. However, within that acknowledgment, there were some clear trends and characteristics, which I think are worth mentioning.


As I said, there were thirty-one separate groups or agencies which conducted polls. Of these, eight conducted only one poll this year (A McKenna, Public Integrity Project, Mother Jones, McLaughlin, University of Connecticut, NBC News, Sacred Heart, and A McHenry) [Of those, U-Conn conducted polls with support from a number of news groups, and NBC News joined the Wall Street Journal for later polling]. Another three groups conducted only two polls (Time/CNN, which split from each other and reformed with other groups for polling, Insider Advantage, and the Battleground Surveys). I have not considered these groups for the purposes of comparative bias, because there just are not enough polls from them to determine a real trend or pattern.


That leaves another twenty groups, however, which have conducted a steady flow of polls. For this report, I will note the five polls which most favor Bush, the five which are least favorable to Bush, the five which are most favorable to Kerry, the five which are least favorable to Kerry, and the five which are, as an aggregate, the least likely to be off the average measure for that date. But before I get to that, I will note here five individual polls which are notable for unusual results.


The Rasmussen polls for April 12-14 and 13-15 were uniquely favorable to President Bush; on those dates the average poll was 40-43 in favor of Kerry, but Rasmussen’s polls for those dates were 46-45 Bush and 46-44 Bush, a bias in favor of Bush.

On the other hand, the Newsweek poll for July 30-31 showed 42-49 favoring Kerry, at a time when the average was 47-48 for Kerry. This was a 5-point bias against Bush.

The CNN/USA Today/Gallup poll for February 16-17 showed 43-55 for Kerry, at a time when the average poll was 45-47 Kerry, which was an 8-point bias in favor of Kerry.

The Battleground poll for March 28-31 showed 43-39 for Bush, at a time when the average was 45-45 tie, a 6-point bias against Kerry.


These five polls do not mean these groups are biased overall, but represent the extremes found when I reviewed the polls taken between February 3 and July 31.
Here are the top five in each of the five categories, then:

Bush Favorable
1. Democracy Corps (+1.8)
2. NPR (+1.7)
3t. NBC/Wall Street Journal (+1.6)
3t. CNN/USA Today/Gallup (+1.6)
5t. Time (+1.0)
5t. Harris (+1.0)

Bush Unfavorable
1. CBS News (no partner) (-3.5)
2. Investor Business Daily (-1.8)
3t. Zogby (-1.6)
3t. Fox News (-1.6)
5. Newsweek (-1.3)

Kerry Favorable
1. LA Times (+2.7)
2t. Time/CNN (+2.5)
2t. CBS News (no partner) (+2.5)
4. CNN/USA Today/Gallup (+1.9)
5t. ARG (+1.7)
5t. Time (+1.7)

Kerry Unfavorable
1. Harris (-2.0)
2. AP/Ipsos (-1.7)
3t. Fox News (-1.4)
3t. Investor Business Daily (-1.4)
5t. Marist (-1.0)
5t. McLaughlin (-1.0)

Aggregate Stability
1. Marist (1.5 avg total variance)
2t. NPR (2.3 avg total variance)
2t. CBS/NY Times (2.3 avg total variance)
4. ARG (2.4 avg total variance)
5. AP/Ipsos (2.5 avg total variance)


I will let the reader draw his/her own conclusions, after making three suggestions:

1. Notice which groups are favorable or unfavorable to both Bush and Kerry

2. Aggregate Stability is a near-unknown quality, but represents a reason to have great confidence in a poll. If a poll does not miss the average by very much, it represents a reliable place for quick vitals.

3. Most of the variance shown is within a reasonable Margin of Error. Without naming the obvious name, however, one group stands out for two reasons. First, this one group is hostile to Bush by more than its Margin of Error, and is favorable to Kerry by almost its Margin of Error, on a regular basis. This group is also unique, because instead of doing its own polls, then joining a partner, this group was teamed up with another media group, but then began doing additional polls on its own, apparently out of displeasure from its partner publishing internals and demographics, which this group conceals.

THE POLLS

Who..............Var Bush..........Var Kerry..........Agg

Marist....................+0.5.....................-1......................1.5
Quinnipiac................-0.5.....................-1.2.....................2.0
NPR.......................+1.7.....................+0......................2.3
CBS/NYT...................-1.1.....................+0.7....................2.3
ARG........................-0.7.....................+1.7....................2.4
AP/Ipsos..................+0.5.....................-1.7....................2.5
ABC/WP....................+1.......................-0.4....................2.6
NBC/WSJ...................+1.6.....................-0.6....................2.6
Pew.......................-1.......................+0.3....................2.6
Time......................+1.......................+1.7....................2.7
Zogby.....................-1.6.....................-0.4....................2.8
Rasmussen.................+0.6.....................+0.1....................3.0
Harris....................+1.......................-2......................3.0
Gallup....................-0.1.....................+1.7....................3.0
LA Times..................+0.......................+2.7....................3.3
Newsweek..................-1.3.....................+0.7....................3.4
IBD/CSM...................-1.8.....................-1.4....................3.7
Democracy Corps...........+1.8......................+2......................3.8
C/U/Gallup................+1.6......................+1.9....................4.1
Fox News..................-1.6......................-1.4....................5.0
CBS News..................-3.5.....................+2.5....................6.0

Maybe I should title this one "Chutzpah". I mean, it's a brash suggestion, I admit. On the one hand, Kerry supporters are not going to agree that the Senator from Massachusetts, leading (for now) in the race for the White House, has anything in common with the man most-often blamed for sullying the Oval Office (although WJ Clinton certainly made an effort to claim that distinction for himself). Republicans, on the other hand, are not going to be willing to consider John Kerry qualified to be compared to a 2-term President, who quite literally changed the nation and the world in his work. The more I look at these two men, however, the more I see a common road to both accomplishment and denouement, and I am not sure Kerry can or would do anything to change his course, even if he recognized the direction.

The similarities, when you look, are striking.

Richard Nixon was a commissioned Navy officer. John Kerry was a commissioned Navy officer.

Nixon served in the Pacific. Kerry served in the Pacific.

Nixon was elected to Congress, at a young age. Kerry was elected to the Senate at a comparably young age.

Nixon became known to many Americans for his role in the House Unamerican Activities Committee investigation of Alger Hiss. Kerry became known to many Americans for his testimony against the leadership and officers of the U.S. Military in the "Winter Soldier" antiwar campaign. As the years pass, both of these occasions are viewed by most historians as dishonorable behavior by the Congress of the United States.

Both men felt that the best way to accomplish strategic goals, required the United States to dialogue with their enemies. In Nixon's case, that meant sending private envoys for quiet talks. In Kerry's case, it meant personally meeting with America's enemies.

Both men, for sometimes good reasons, but also often foolishly, showed a stubborn streak, refusing to listen to anyone who didn't already agree with them.

Nixon believed that the North Vietnamese would honor the cease-fire, because the U.S. would support South Vietnam. Kerry believed that a North Vietnamese victory would be relatively peaceful. Both men were wrong. When Congress failed to maintain arms supplies to South Vietnam, North Vietnam invaded. Accurate totals are not kept, but the number of people killed in Southeast Asia during the first year after the Communists invaded is estimated at roughly 2 million people. Approximately 30,000 South Vietnamese troops were massacred the week after Saigon fell, on April 30, 1975.

It turns out that Nixon had heard of Kerry, though it's very unlikely the two ever met. Sean Robins documented a discussion between President Nixon and Chuck Colson, retrieved from one of those notorious White House tapes:

(April 28, 1971, 4:33 p.m. Telephone call from Charles Colson to Richard Nixon. )

"COLSON: This fellow Kerry that they had on last week (referring to a televised appearance by Kerry).
NIXON: Yeah.
COLSON: He turns out to be really quite a phony.
NIXON: Well, he is sort of a phony, isn't he?
COLSON: Yes. . . . He was in Vietnam a total of four months. . . . He's politically ambitious and just looking for an issue.
NIXON: Yeah.
COLSON: He came back a hawk and became a dove when he saw the political opportunities. "

Nixon's fall was anything but unpredictable. His first embarassment was the scandal in which the VP candidate was accused of taking bribes. While Nixon handled the challenge effectively and eloquently, the charge would be remembered later as the first indication of the man's character. Later, Nixon was affected by the collapse of Senator Joseph McCarthy, when many of his charges proved to be baseless. After he lost the Presidential race to Kennedy, Nixon failed to win the office of Governor of California, and he gave up politics. For a little while.

As for Kerry, his road has had its potholes, as well. When Kerry left Vietnam, he claimed it was because he had served valiantly, and his three Purple Hearts made it necessary to come home. But Admiral Zumwalt had a different impression. "We had virtually to straitjacket him to keep him under control", the CNO said, to prevent Kerry from killing noncombatant civilians and acting recklessly. Other Navy officers concur, saying that Kerry was "told" to leave Vietnam. When Kerry first ran for office, he lost miserably, largely due to his transparent ambition. Later, when Kerry ran for the Senate, he needed to avoid his antiwar past, and so claimed that the medals he announced he threw way in 1971, he in fact never gave up. Kerry's conflicts between his statements and his history are striking, and would require several posts of their own to do them justice. But like Nixon, Kerry never lost sight of the prize, in their case the White House.

Many people remember the Watergate scandal vaguely. But many do not realize that while the U.S. House of Representatives was preparing articles of Impeachment against Nixon, they never proceeded (because he resigned), and these nine articles were all built on Nixon's actions and words after the Watergate break-in. That is, there was never anything to be held against President Nixon, until he began to cover-up the break-in. If Richard Nixon had told the Justice Department, 'do your job', and had simply allowed the investigations to proceed normally, while he might have been embarassed by the results, there was nothing to implicate the President. However, Nixon's pride would not allow him to let such an investigation proceed. He was aware of his 'enemies', and he was contantly at war with them. In the end, Nixon's downfall came because he was unwilling to consider any opinion but his own, or to evaluate his actions on a more objective standard.

John Kerry is exhibiting many of these same characteristics. He claimed to speak for many Vietnam Veterans, as a member of a group which included fake veterans. But today, in spite of the evidence, Kerry won't admit this was a mistake. He claimed American soldiers had committed war crimes, with "fullawareness" by officers, but when confronted later with the falsity of these charges, he denied ever making them. Kerry voted against the first Gulf War, but now claims he supported it. These are not issues where a change of mind is in accordance with developed belief, but a stubborn refusal to admit errors and mistakes, and is exactly in line with the stiff-necked personality of Richard Nixon.

Maybe John Kerry will not win this election. Personally, I think he will lose. But his character is such, that I would not expect him to go away even so, and if it should ever happen that Senator Kerry becomes President Kerry, some little event, so innocuous as to be unbelieveable, will be waiting to trip him up. Or, to be more accurate, waiting for him to trip himself up.

The Marines Meet Yet Another Insurgent

"The next time I come in here, I expect to see French fries on that plate! And you better be using Heinz catsup!"

A Quick Ride to Work

The Good News: The wife said I could look, even take a test drive.
The Bad News: They don't finance, and it's only a 2-seater. Worse, my daughter insists on driving.

Snapshot Pre-D-Convention

Over the last couple weeks, I have been looking through the details in nine polls (the ones I don't mention are not willing to share their demographics and such) which took their information between July 8 and July 22, a nice two-week window into Pre-Convention Campaign 2004.

I took data from the July 19 Marist Poll, the July 21 Pew Poll, the July 22 Fox News Poll, the July 22 LA Times Poll, the July 17 New York Times/CBS News Poll, the July 24 Quinnipiac Poll, the July 28 USA Today/CNN/Gallup Poll, the July 21 NPR Poll, and the July 22 Wall Street Journal/NBC News Poll.

Here are the results from those nine polls, the detailed kind people like me find absorbing:

89.2% of Republicans say they will vote for President Bush, while 85.4% of Democrats say they will vote for Kerry, a 3.8% advantage for Bush if the turnout is even between the parties.

8.8% of Democrats say they will vote for President Bush, while 5.8% of Republicans say they will vote for Kerry, a 3.0% advantage for Bush if the turnout is even between the parties.

Men prefer Bush over Kerry, 47.0% to 43.5%.
Women prefer Kerry over Bush, 47.0% to 40.5%.

77.3% of Bush Supporters say their support is "strong". 70.5% of Kerry Supporters say their support is "strong".

Respondents liked Bush as a person better than Kerry, 45.3% to 40.7%.

77.3% of Bush Supporters expect Bush to win. 67.3% of Kerry supporters expect Kerry to win.
10.3% of Bush Supporters expect Kerry to win. 18.7% of Kerry Supporters expect Bush to win.

This explains the message and tone of Kerry's nomination acceptance speech to me. Kerry still needed to build up his base, to energize his support. To do this, Kerry basically had to abandon the swing voters on this one, hoping he'll get another chance later.

It will be very interesting to see the August polls from these sources, and compare the changes in these areas.