Gallup and New Coke

The polls are wrong this year, very wrong. I have been saying this for months, and I have backed up my claim with both statistical and anecdoctal support. The claims I have made have inspired some, caused others to laugh in derision, and brought others to test their assumptions and revisit the hard data. Along the way, there have been a lot of questions about how and why the polls could be wrong. The most common complaint, is that for all of the polls to be wrong, there would need to be some sort of conspiracy, or else an incredibly stupid decision made across the board. Well, I am not a big believer in conspiracies, but I do think that the polling groups have fallen into a groupthink condition. I wrote earlier about the fact that of the major polling groups handling national and state polls, all of them are based deep in pro-Liberal, anti-Conservative territories.

Here’s that list of headquarters again, just to punch in that point again:

Poll Headquarters
ABC News 77 W 66th St, #13, New York City, New York
CBS News 524 W 57th St, New York City, New York
FOX News 1211 Avenue of the Americas, New York City, New York
Gallup 901 F St NW, Washington DC
Hotline 88 Pine St, 32nd floor, New York City, New York
IBD 12655 Beatrice St. Los Angeles, California
The Los Angeles Times 202 W 1st St, Los Angeles California
Marist Institute 3399 North Rd, Poughkeepsie, New York
Mason-Dixon 1250 Connnecticut Ave #200, Washington DC
Newsweek 251 W 57th St, New York City, New York
The New York Times 1 City Hall, New York City, New York
Pew Research Center 1615 L St NW, #700, Washington DC
Quinnipiac 275 Mount Carmel Ave., Hamden Connecticut
Rasmussen 625 Cookman, #2, Asbury Park, New Jersey
Reuters 3 Times Square, New York City, New York
Survey USA 15 Bloomfield Ave., Verona New Jersey
TIPP 690 Kinderkamack Rd, Oradell, New Jersey
Washington Post 1150 15th St NW, Washington DC
Zogby 901 Broad St, Utica, New York

As I wrote then, it needs noting that all of the major polling organizations are based in locations where liberals are strongest and conservatives weakest, where ‘democrat’ and ‘republican’ take on meanings wildly different from the rest of the country. The people making the executive decisions at these polls, most likely including the wording and order of polling questions, whether to focus on urban or suburban areas, the weighting of political affiliation, and the definition of ‘likely voter’, are most likely in regular contact and association with the most liberal factions of politics. It does not mean that they have deliberately skewed their decisions to support Obama, but it is obvious that there is an apparent conflict of interest in their process modality.

I want to stop here and direct the reader back to the ethics of polling. The National Council on Public Polling is, and I got this from their site’s welcome page, “an association of polling organizations established in 1969. Its mission is to set the highest professional standards for public opinion pollsters, and to advance the understanding, among politicians, the media and general public, of how polls are conducted and how to interpret poll results.”

NCPP members identified on the front page include ABC News, CBS News, Gallup, Hotline, Ipsos, the Los Angeles Times, Marist, NBC News, Pew, Princeton Survey Associates, and Survey USA, along with many others. This effectively promises that the major polls will abide by NCPP rules, something you should consider when matching the principles against the polls’ actual statements.

The NCPP has also posted a list of “20 Questions a Journalist Should Ask About Poll Results”, which I strongly recommend every one to read and memorize. Those questions include these very important queries, that I fear most people do not often consider:

2. Who paid for the poll?
In many cases, the poll we see in the papers and on television, was paid for by an agency known to be biased. For example, does anyone really expect CBS News or the New York Times to be even-handed, especially in light of their behavior since 2002?

7. Who should have been interviewed and was not? Or do response rates matter?
This is a sore spot for polling groups, because frankly most people do not have the interest to stop and take an 8-to-10 minute interview, especially from someone they do not know calling them up when they are likely to be busy doing something else. It’s been established as well, that democrats in recent years are more willing to take part in polls than republicans, possibly due to perceived bias on the part of the media. But it is quite important to know if the pollsters were getting one person in ten to take the poll, or only one person in fifty, because the people not interviewed matter just as much as those who do participate. Yet I have never yet seen a poll this year that publishes response rates.

14. What questions were asked?
This is a big one that a lot of folks miss. I have noticed in the details, that all of the polls are asking about the public’s opinion of the economy, and of their opinion of President Bush, even though he is not running this time. Also, I have noticed that many polls ask a question about John McCain just after asking about the voter’s opinion of President Bush, subtly linking the two men. For comparison, no questions have been asked about approval of the specific performance of either Majority Leader Reid or Speaker Pelosi, and no other politician is linked to Barack Obama in the same way that polls link President Bush to John McCain. This is a clear violation of the NCPP’s guidelines, yet it is done in absolutely every poll I have seen. Further, polls taken since Labor day have not mentioned foreign policy at all. There are no questions regarding Russia’s invasion of Georgia, nor of Iran’s nuclear weapons programs, nor about China’s intentions viz a viz Taiwan, even though these are current events which have great significance in a presidential race, yet all of the polls are ignoring them. Again, the economy-only focus betrays a bias which violates the principles of the NCPP.

I have already written extensively about polling groups manipulation of demographic weights, so I will only summarize here that in addition to party affiliation, various polling groups this year have produced polls out of demographic balance with Census norms for urban/suburban/rural participation, minority race representation, age, employment status, and income range. It should not be difficult to imagine how these manipulations might invalidate the results published by the polling groups.

When people reach this point in the discussion, an obvious question comes up; surely the polls want to be accurate, and they would have to understand that this fiddling with internal data to create a false image would destroy their credibility? And the answer to that can be phrased in a two-reminder of just how stupid people can be – “New Coke”.

In 1985, the Coca-Cola company dominated the beverage industry around the world, and it’s flagship product was its first, the Coca-Cola soft drink, literally an icon of Americana. It would seem to be the most obvious of strategic decisions, to leave the base of the company alone. Instead, in a move never explained let alone justified by the company, Coca-Cola announced that they were eliminating Coca-Cola, and replacing their number 1 product with a new formula, called “New Coke”. Everything about the promotion was an unmitigated disaster, and later that year Coca-Cola re-introduced what they claimed was the “original formula”, named “Coke Classic”. The company tried to push “New Coke” on a public that never wanted it, and eventually gave up the next year. The “New Coke” strategy and promotion have become textbook lessons on the worst possible way to listen to customers and meet their expectations. Pretty much everything was done the wrong way, especially the arrogant way that Coca-Cola assumed their customers would accept the elimination of their favorite drink. Near as I can figure it, the essential problem came down to the fact that the company’s marketing people made all the key decisions internally, without once stepping out into the real world to test their assumptions. What seemed a great idea in development, failed miserably in Reality. Obviously, Coca-Cola never wanted to enrage its customers, to drive them to Pepsi, or to put a bullet in their stock value, but that all happened because they made an incalculably stupid strategic decision, and they lacked an effective Deming loop to test assumptions and correct the process.

This is actually not all that uncommon in business. So many people saw Enron as a company made up of crooks, that they failed to notice that Enron did have a Code of Ethics; the problem was similar to Coke, in that too many people never tested their assumptions, and by the time they realized something was wrong, it was too late to repair the damage.

This brings us back to the polls. The thing most folks forget about polls which get published in the media, is that the polls’ first need is not to accurately reflect the election progress and report on actual support levels; it’s about business. A poll needs clients to survive, and the media – always – wants a good story more than they want facts. So polls sell that story, and what would actually be a gradual development of support, with modest changes brought about as the public learned about candidates’ records and positions, is instead sold as an exciting roller-coaster race, careening madly all over the place. If a candidate appears to be popular and charismatic, he might be allowed a strong lead, or the poll might tighten things from time to time just to keep attention on the polls. That’s where that whole “bounce” thing after the conventions comes from – do you really think republicans or independents got more excited about Obama because of his convention, or that democrats and independents were more likely to vote for McCain because of the GOP convention? When you think about it, it should be obvious that these bumps are artificial unless there is a clear cause to show a change in support. And when you take apart the polls and drill down to the raw data, what you find is a close race with a gradually declining but still large pool of undecided voters, which is consistent with the known facts and actions we see from both campaigns.

Obviously, though, the polls want to finish as close as possible to the actual results, but this year they have a problem. There has been unprecedented manipulation of demographics, corrupting even the raw data to the point where effective resolution of public opinion is doubtful. This might be described as an honest mistake, if one is willing to accept greed as an honest motive. Gallup, for example, who has more experience than any other polling group and who therefore should have known better more than anyone else to fiddle with the weights. In several past elections, Gallup and other polls have learned from operational blunders.

In 1948, Gallup screwed with the weighting, assuming the republicans would turn out much in much larger numbers than the democrats, but they were wrong, and badly miscalled the election. In 1952 Gallup assumed the other way, that the race would be tight and down to the wire, but they blew that call as well. In 1976, Gallup assumed the opposite, that democrats would overwhelm republicans because of Watergate, but when it became obvious that republicans would vote anyway, Gallup had to change its model to show their participation more accurately. In 1980, Gallup called Carter ahead until the very end, when they grudgingly granted Reagan a small lead, yet another case where Gallup’s assumptions were well off the mark. In 1996, Gallup overstated Clinton’s support and understated Dole’s support throughout the campaign, and in 2004 Gallup called the race too close to call. This year, trying to gauge the effects of Barack Obama’s ‘rock star’ charisma, Gallup decided to abandon historical norms and overweight urban and youth voters, and to over-sample democrats all campaign long. This model, dubbed the “expanded voter”, has proven a disaster for Gallup, so much so that the group reintroduced a more historically balanced model, which they call the “traditional” model. The problem for Gallup, however, is that their methodology became so skewed throughout the campaign up to now, that it may be impossible for Gallup to correct its procedures before the final election poll. In the light of past blunders, this year missing the call may not be unreasonable at all to expect.

So OK, Gallup is having a bad year, but what about the rest? Well, there the phrase to consider is follow the leader. Gallup has been doing this stuff for longer than anyone else, and the other polls have often fallen into the habit of chasing what they see Gallup do. But for an objective look at their performance, I direct you to another of my past articles, where I noted the NCPP’s record on poll accuracy. From what I see here, if Gallup is having problems, it’s likely just as bad or worse for everyone else.

So, could I be wrong? I have to be honest and admit that I could. But in that case, we’d have to ask why the polls do not generally agree with each other, why Gallup is trying to spin three different models at the same time to get a grasp of the picture, why McCain and Obama are both so interested in Pennsylvania, yet neither is working very hard in Ohio right now. We’d have to explain why McCain-Palin rallies are now attracting thousands more people than Obama-Biden rallies, why Letterman suddenly found it cool to have McCain on his show and SNL decided they wanted Palin on theirs. We’d have to explain why there are not a lot of Obama signs visible, but we hear about his army of lawyers getting ready. We’d have to explain why McCain and Palin appear to be so relaxed while Obama and Biden look like they’re worried.

What I think is happening, is this – the polls’ headquarters were based deep in liberal territory, where the assumption was that Obama’s candidacy would actually create a groundswell of pro-democrat voters unseen in the country since 1932. That McCain is more experienced with the key issues than Obama was ignored, that the historical significance of the debates shows that the effects appear several weeks later was also ignored. That the economy could be as reasonably blamed on the democrat-controled Congress as on the republican President was never considered. That character would be a salient factor in the decisions of voters was rejected out of hand.

The polls are wrong. Make your own mind up, because your vote will matter.