The Sad Case of John Zogby

.

By now, a lot of people have read Zogby’s latest poll, which purports to support the claim that most servicemen in Iraq want to call it quits and go home. A lot of bloggers have weighed in, on both sides of the aisle, the best in my opinion coming from Bruce Kesler at Democracy Project and Hugh Hewitt. My own opinion on Zogby should already be obvious.

Back in 2004, I caught him cheating on basic methodology and basically excommunicated him from the ranks of serious pollsters. That time, I discovered that Zogby was mixing telephone and online results without announcing the fact, and worse, had apparently been doing this for some time. Zogby also used some of his prior polls to drive demographic weighting for future polls, something frowned upon by the NCPP and the AAPOR.

I have not written on polls very much since the 2004 election, because in my opinion the current polls are not managed at the same level of standards as the election-year polls. This is due to several key factors, not least the fact that a lot more people are interested in opinion polls and the major issues during an election year, which means a 2006 pollster is going to have a much harder time getting a large enough respondent pool using RDD methodology to validate the results. Second, no one seems to be looking for the “likely voter”, on the argument that there is no such thing right now. I would remind people that there is always an election at some level coming up (primaries, anyone?), and pollsters could also ask whether someone voted in the last election. But it appears that another look at polling could be beneficial.

Political opinion polling is a business. A lot of people forget that. And what’s more, Zogby is a kid on the block. A short, arrogant, egotistical kid. Normally, I don’t pick on newcomers, as blogging is a pretty new medium and I am a newcomer amongst the bloggers, but in Zogby’s case, it actually appears that Zogby has something of a psychological problem with his newness, and he is overcompensating in a lot of ill-chosen ways.

I think it all comes back to 1996. Since the Republicans took over Congress in the 1994 elections, which was unexpected by the Dino-media, the conventional wisdom had swung to believe that President Bubba was going to be in trouble getting himself re-elected. As the election drew near, John Zogby’s polls continued to show Clinton leading Dole, which was what all the polls did by late October, but Zogby was the first to put Clinton clearly in front, and his 1996 final poll most closely matched the actual popular vote results. John Zogby was “made” as a professional pollster.

This was a big deal. Polling has been around in the United States for more than a century, with Gallup basically leading the parade since 1936. Interestingly, Gallup got it’s liftoff by predicting what conventional wisdom said would not happen, so there’s that common link, sixty years apart. The difference between the two polls is, of course, also those sixty years of distance, during which time Gallup has worked to review its results and see where they missed a target, in order to improve their process. In recent years, Gallup has not had to change their basic methodology at all to get reasonably good indicators of the public mood. They’re not exactly coasting, but it’s very difficult to see how someone else can pass Gallup as the leading media-quoted opinion poll.

Returning to Zogby then, we see the problem. Zogby had a good year in 1996, and wanted to parley it into a grand slam. But to do that, Zogby had to travel down one of four roads:

1. Be the most accurate poll. In Zogby’s defense, I believe this was his initial goal. None of the polls I see out there can be described as always accurate, much less perfect. So every pollster dreams of finding some missing element which raises the credibility of his poll above the rest, and I really do think that some of Zogby’s ideas, like the interactive online poll, were intended to discover possible resources which tapped the national mood. Zogby is aware, for instance, that the daytime call to a home telephone is not going to produce a representative response of the demographic cross-section, but almost no one wants a cell call from a pollster, and businessmen are unlikely to take a call during working hours anyway. I believe that somewhere after the 2002 Elections, Zogby came to the sad conclusion that he does not possess a means to be substantially more accurate than Gallup,and since Gallup is the #1 poll, they remain in front.

2. Be the strongest private poll. One thing which came out during the 2004 election, was that the Bush and Kerry campaigns did private polling. The fact that political candidates feel the need for private polling has suggested to me that the nominal public opinion polls are not up to the job. Certainly Zogby could make a nice pile of money, and presumably does, on contracted assignments for special clients. In Zogby’s case, however, his habit of using information from one poll as part of a driver of methodology in another poll, suggests to me that John might have poisoned his well of likely clients.

3. Be the first to ask the questions on everyone’s mind. Relevance and timing are critical components to poll success, and a sure road to media attention. The mistake made by Zogby, is that he has let his focus slip, so that he asks the kind of questions and phrases the wording, in the way he personally thinks. This is a critical error. I know from a couple helpful phone calls, that Gallup develops their questions and wording through a conference process, which tries to avoid both the fact and the sense of partisanship. Either Zogby writes the questions himself, or he has a staff which is expected to think as he does. The questions are evidence in themselves that Zogby has abandoned objectivity, and as a consequence, his polls are less and less likely to capture the mind and mood of the public.

4. Be controversial. Jerry Springer knew this; loud noise and conflict attract an audience. Zogby has apparently decided that he cannot be the most credible poll, nor attract the most influential clients, nor can he tap into the national mood at large better than other polls, so he has resorted to scripting his polls to produce a result which will generate headlines and debate. Having the words “Zogby Poll” included in episodes of “Face The Nation” or “Meet The Press”, as well as the nightly MSM newscasts would be good advertising, he seems to believe. This only demonstrates yet another Zogby error; polling is very interesting to bloggers, especially quant wonks who can take a poll apart and see how it was built. Refusing to admit and correct an error which invalidates a poll is bad enough, and of course deliberately rigging a poll to get desired results is very stupid these days, as you will, sooner or later, get caught, but Zogby fails to understand that bloggers will not simply forget his deception in 2004, nor fail to catch him out when he tries to lie through his teeth as he did in the latest ‘army wants to quit’ poll.

John Zogby may have a future in cable television, but not in professional polling. He’s become a media whore, and has crossed the line beyond which I do not believe he can return to credibility.