There Is No Alternate Universe

People throw out a lot of strange ideas at times. Some of that is because people can believe some very silly things, and some of that I blame on television and comic books. But it really gets strange when we see fantasy and illusion play out in politics, in the guise of fact. CBS tried to smear President Bush with documents they admitted were fakes, the claimed inexperience of a Vice-Presidential candidate is loudly harangued, while the weaker resume of the other major party’s Presidential nominee is ignored, and biased opinion polls are touted as objective sources of news. Truly Alice-in-Wonderland stuff, except that this is the real world, and we dare not risk trusting the Mad Hatter.

Opinion polling is fun in many ways, useful in certain aspects, but in the end should not be trusted as a guide or counselor for course decisions any more than one might trust a Ouija board. The bias is often missed, even though it is obvious once you know where to look. I have shown before that every poll is biased to some degree, and I have repeatedly made clear what you should expect from a valid poll:

• public access to internal data
• a consistent, publicly reported methodology
• weighting according to Census norms or reasonable objective standard
• archive data available for comparison to current polls

Alas, there are areas where even the best of opinion polls cannot meet the standard of true objectivity. The most common stumbling block comes when a poll weights its political party affiliation. Polling groups understand that a truly random system of contacting poll respondents will produce results which are, to some degree, at odds with the true opinion of the public as a whole. To correct for this, respondents are asked certain questions to determine key demographic data, and their responses are categorized according to those demographic keys and the overall results weighted so that the response pool model is in line with demographic norms. The demographic standard used is almost always based on the most recent US Census data, for reasons that this data is considered the most unbiased and reliable demographic data available. So, Census data is used for gender, race, age, education, employment, and economic strata norm determination. This makes a lot of sense, and I applaud the pollsters for that standard and diligence in its application. The same polls who are so careful to avoid bias in most demographic norms, however, get completely squirrelly when it comes to party identification.

Political party identification is a stronger factor in candidate support than any other cited demographic category. It is no shock, after all, that democrats overwhelmingly support the democrats’ nominee, while republicans overwhelmingly support the republicans’ nominee. The problem comes in, when the poll weights the response pool to match a desired party identification standard. I noticed four years ago when I looked at the opinion polls, that the polling groups changed their party identification weights, some every week! When I contacted the polls about this practice, most just ignored me, but a few did respond. I found their explanations troublesome, however. Pew, Gallup, and Rasmussen, for example, choose their party weighting by examining the self-reported party affiliation of respondents from a prior period, usually a month (the ones who change each week generally use a rolling average of respondents from the period reviewed). Unfortunately, the reasoning behind such weighting is directly contrary to the whole purpose of weighting in the first place. To see what I mean, let’s apply that logic to other demographic groups.

South Texas was hit hard by Hurricane Ike, and it is very unlikely that any polling group had much success calling anyone down here in the past week; those who had phone service restored had a lot to do, and no time for answering polls. But even as phone and power service was restored, this was done according to critical needs (like hospitals) and a plan to get the most service restored to the most people as quickly as possible, which in practical terms means that urban areas and recent construction would get priority. This is because of the concentration of population, and the relative ease of repairing lines which would be less likely to have serious damage to the lines and transformers (older neighborhoods tend to have trees and other growth obstructing lines, so that in a major storm older neighborhoods are much more likely to have substantial damage to the transmission lines). Consequently, for the next month, an overwhelming majority of poll respondents in South and East Texas will be urban areas and new neighborhoods, which would heavily skew the demographics of the polling, if those respondents were considered “normal” for the area’s demographics. Or consider another example, where more mid-week polls are taken. People who work or who are taking care of children would be less likely to be available for polls, which would skew the apparent demographics towards the youngest and oldest voters, and those whose lifestyles matched the tactics used to reach respondents, like visiting malls or urban centers. Polling groups know these areas are heavily skewed in their demographic types, with heavy oversampling of certain groups, and so the weighting is adjusted to match Census norms, precisely because anything else would invalidate the poll results.

Those are just two obvious examples of why using poll results for one period to establish the demographic weights for another period would be clearly invalid; it’s simply circular reasoning and any errors (and there are always errors to some degree) would be greatly magnified rather than corrected. What’s worse, the only validity for any poll whatsoever is movement within a poll handled by consistent and transparent methods – if you shift party identification weights, you invalidate all conclusions. Anyone who has annoyed their lab professor understands that controls exist for a reason, and fiddling with the weights between polls is simply unscientific.

Some of the pollsters I have spoken with, argue that they shift their weights because no useful and objective source exists for party identification weighting. That, however, is not true. The exit polling from prior elections is a very valid source. It comes not from “adults”, “registered voters”, or even “likely voters”, but from people who actually voted. And what’s more, we can look at the last few elections to see if there is any substantive change. I looked at national elections for the past ten years, and discovered an interesting pattern:

In 2006, 38% of the voters were democrats, 36% were republicans, and 26% were independents;
In 2004, 37% were democrats, 37% were republicans, and 26% were independents;
In 2002, 39% were democrats, 38% were republicans, and 23% were independents;
In 2000, 39% were democrats, 35% were republicans, and 27% were independents; and
In 1998, 39% were democrats, 33% were republicans, and 28% were independents.

That sure looks like a consistent pattern to me. On average for the past ten years, democrats have averaged 38.4%, republicans 35.8%, and independents 26.0%. If we kick out top and bottom outliers, it becomes 38.7% democrats, 36.0% republicans, and 26.3% independents. Those numbers have a solid empirical history and a thoroughly objective source behind them, yet the polling groups do not use them. You might well wonder why.

Polling groups exist for one of three purposes. They either serve the needs of a client (private polling companies), they do academic research (which serves the faculty running the polls), or they are done for public release (including those by colleges like Marist and Quinnipiac). Each group has a very specific, and different, reason for existing. You may want to consider that both Barack Obama and John McCain have hired private polling firms, and consider why they each spend that money, as indeed every major political candidate does. And if political allegiance is so flighty, then why do so few of us know anyone who has supported one party, then another, in the same election? To read the reports from the polling groups, there are a lot of folks who loved Obama, then McCain, then Obama again, or vice versa, yet I have not met even one such person. Instead, I have seen democrats who stay democrats, I have seen republicans who remain republicans, and I have seen folks making up their mind as the campaign progresses. There are times where you might see a new jump in support, as previously undecided voters choose someone to support, but that happens for a reason – the “bounce” that supposedly comes just because a party has a convention, well, that’s pretty bogus if you think it through. The democrats may have been more excited about Obama after the democratic convention, but I do not believe for a minute that it made many new democrats. I am quite sure that the GOP convention excited republicans, but again I have to say I do not recall hearing about a bunch of new republicans after the convention. So, when the polls increase democrats’ weighting after the one convention and increase the republicans’ weighting after the other, it’s frankly dishonest because neither party saw much change in affiliation. What’s more, does anyone really think that there were fewer republicans after the DNC, or fewer democrats after the RNC?

Please.

Party affiliation is not some ephemeral quality, which winks in or out of existence because of a speech by one candidate, the headlines of a single day, or the personal desire of a polling group to have a hot story for the newspapers and TV stations. Party affiliation is something developed over a period of time, and while it does change over time, it does so gradually, taking into account the evidence of history as well as events of the moment. There were tens of millions of republicans after Bob Dole’s loss in 1996, just as there were tens of millions of democrats after Walter Mondale’s thrashing by Reagan in 1984. Support for a candidate is a thing apart from party identification. When it does change, it is spurred by a significant event of meaning with regard to the values and ideals of the party This is not merely a belief, but the evidence of American electoral history and the employment of common sense.