The Business and Utility of Opinion Polls

[].
[].
[].

I have written about the validity of National Opinion Polls, as they relate to politics, and a key statement summarizing any poll is this; “it should be understood that a poll is nothing more than the combined effect of its components”.

It’s very important to understand the demographic support, especially the base. During the 2004 campaign, I was aware that while George W. Bush had to work to convince Independents and Undecideds, his base was more solid than Kerry’s, which indicated not only a better share of his party support, but also forecast better Republican turnout than the Democrats, which turned out to be the case.

Back in August, I examined the demographics from past Presidential elections, to observe and note the ones which were the most relevant and effective to the outcome. Singularly effective among the opinion polls in this regard are the ones from the Gallup Organization, which I continue to regard as the most reliable and therefore the most influential poll group.

After the election, the poll focused on other questions than the candidates, and I noticed a shift in how the methodologies were used. The most significant changes were the abandonment of using Registered Voters and people known to have voted in the last election (the post-event version of the “Likely Voter”), with simple count of Adults, and a sometimes chaotic fluctuation in Party Identification.

In late May, I published an article which was critical of the weighting used by the poll groups, especially in terms of party identification. At that time, I noted a Pew analysis indicating that Republicans have nationally achieved a rough parity with Democrats in party identification. This led to an exchange of e-mails and a long telephone call, with an executive at one of the major polling groups. I was asked not to reveal the source’s name or position, or certain details provided for background about the group. The source I spoke with considers bloggers a vital resource for polling’s future, but notes that the controlling interests in polling groups are somewhat suspicious of the standards and professionalism of blogs, and further notes that if it were revealed that one blog had a contact with a specific polling group, other blogs would demand equal access, which would disrupt the polling group’s normal work. Also, it can hardly be denied that while some blogs are consistent and scholarly in their diligence, others can hardly be bothered to check their facts and support their contentions with any sort of rational foundation.

This created a conflict for me, however. In the first place, I have always desired to support my information with a source, ideally a link. In the case of direct interview, the link is not possible, and when a source wishes to remain anonymous, the attribution which would establish the fact is also removed from possibility. In such a case, I must ask the reader not to accept my word implicitly, but to weigh the uncited source in terms of his own judgment, qualifying the matter by reminding the reader to always consider multiple claims before believing too wholly in any one.

But I also had to sort out whether what I was told, was accurate in the general application. Even if what my source said was completely in the group for which he worked, it may be that his group was different in the main from other groups. To consider the matter more fully, I contacted two other sources I know in polling groups. From the results of those discussions, I satisfied my own curiosity about the priorities and restrictions under which polling groups work, and confirmed some similarities and differences.

Polls have existed for far longer than most people realize. They were first recorded in the United States during the campaign for the 1824 Presidential Election, which also was the first to allow Popular Votes. Polls were largely unscientific, however, until after the 1936 Election. The Literary Digest blew their prediction of the election so badly, that some scholars believe that’s why the magazine went out of business. But the new Gallup, Roper, and Crossley polls hit the mark rather closely, and this established that even with a small sample, a poll which used relevant data could be effective in reflecting the national mind.

Polls initially were created for the clients who paid for them. In the case of elections, candidates learned that knowing the areas which needed attention and the groups which could most easily be swayed, were critical to their chances for election. And news groups recognized that in the absence of sources, they could print poll results which demonstrated a demand for change or reform.

With the rise of 24-hour news services and the Internet, Polling claimed a much higher profile. No longer did polls only need to be produced in relation to a coming election or in response to a movement already underway; with demand in constant flow, regular polls became part of the news diet and the appetizer for talk shows nationwide. It’s not commonly noted, but one reason Harry Truman’s win in 1948 over Tom Dewey was such a shock, was because polls were sometimes months apart during that campaign. During the 2000 Campaign, most of the major polls published standings only 3 or 4 days apart in the last month, and in the 2004 Campaign, there were daily polls available from Super Tuesday straight through the Election.

That’s the good news for polling groups; they have a hungry and growing audience. The bad news is, with all that interest, the attention also brings questions about their methodology and its consistency. Polling groups generally try to follow the known demographics, most often basing their weights on the last U.S. Census for gender and racial composition. Weighting is taking the raw results and adjusting them to better reflect the actual balance between men and women and the different races. It is more controversial, when weighting is used to change more fluid elements, such as party identification, or when a poll which was based on Likely Voters or Registered Voters, decides to use random adults instead.

This brings us to the war room of a polling group. There are many theories of polls, some based on the mathematical anaylsis of statistical probability, others developed through analysis of historical evidence. The problem with trends and opinion, however, is that trends and opinions change, and it can be very difficult to note the difference between a momentary slowdown and a true shift in direction, much less to sort out the cause and scale. If you work for a polling group, you want to establish consistent, reliable rules for your polling, not only for the immediate results but to make your name in a growing industry. Where there were once less than a half-dozen polling groups of note, there are now more than thirty polling agencies which publish national releases, not touching the private groups which work only for their clients. This creates real pressure for pollsters to not only be accurate, but also find a way to set themselves apart from their competitors. I suspect this is why John Zogby made his boneheaded statement last year, showing himself an advocate of John Kerry in mid-year in a sequence of moves that ended up shredding his credibility; he was losing market share and became desperate. The reader may recall that Zogby followed up his declaration for Kerry, with the decision to mix his online and telephone respondent pools, and so corrupting the analysis and results for both poll methods. Zogby became to polling what Amy Carter was to Foreign Policy.

The debate about using National Adults in place of Registered or Likely Voters appears to have come down to money. It takes longer and costs more to ask people if they are a Registered Voter, or voted in the last election, and so the polling groups have all pretty much reverted back to just taking a somewhat human voice on the line. I understand the cost but cannot agree to the argument; while all three sources said they don’t believe the use of Adults instead of real Voters changes things, I cannot agree to the notion that a respondent who does not vote, is still somehow a reliable barometer to how voters see an issue, but until I get my own polling agency I will have to accept this condition, since it prevails across the board.

The real fireworks start, when one considers the question of political party identification. The first thing I hear when I raise this question, is that many people in polling do not consider party identification to be a true demographic, or even a salient indicator of support. For instance, the overwhelming majority of voters in Oklahoma consider themselves to be Democrats. Yet they are unyieldingly conservative on most of the issues, and have voted consistently for Republican Presidential candidates. If one considered only Oklahoma and New York voters, one would conclude that Democrats were more conservative than Republicans, but also that the population considered itself more heavily Democrat than Republican in party identification. Also, it is a plain fact that people think differently of their National, State, County and Local officials. Here in Houston for example, the city is weighted such that Democrats, especially liberals, usually win the City races, yet the County is predominantly Republican. That said however, it seems reasonable to me that when people know they are speaking to a national poll, they will identify themselves as they see the national parties.

The issue is complicated by the fact that the raw party identification breakdown changes rapidly, enough to raise the question of whether the variance reflects changes in the methodology. During the debate about the erroneous exit polls by Mitofsky which came out late morning of Election Day, it was revealed that the pollsters methods may well have deterred Conservatives and attracted Liberals; a clear breakdown of party identification would have shown the error much earlier than was later revealed. I do sympathize, however, with the difficulty a polling group has in establishing where a party weight should rest. But to me, that makes it all the more important to keep track of raw party weights, to be able to determine where it’s settling.

One difficulty the modern polls face, is a true evolution in political opinion. When Gallup and Roper came into existence, the Democrats enjoyed popular support under FDR, and the nation largely considered itself Democrat in the main through Truman, then Kennedy, Johnson, even through Carter. Eisenhower and Nixon were no more than a change of pace, and Watergate hurt the Republicans in every demographic sector. The Reagan Revolution convinced pollsters that Ronald Reagan was popular, and the Contract With America in 1994 convinced them that the New Deal was finally wearing out its influence. The close count in the 2000 Election misled pollsters into missing the clear signs of Republican growth, but the 2002 Midterm surprises finally snapped attention towards the undeniable trend. The last campaigns merely confirmed the results from the last 25 years; America is becoming far more Conservative, and the Republicans are now far more representative of most Americans’ opinions than the Democrats. Until the polls understand the numbers in that context, they will be fighting the tide.