Exclusive Excerpt: Lost in a Gallup
Here’s an excerpt from “Lost in a Gallup: Polling Failure in U.S. Presidential Elections” (University of California Press), a new book by W. Joseph Campbell that tells the story of polling flops, epic upsets, unforeseen landslides, and exit poll fiascoes in American presidential elections.
Over the past 80 years or so, polls and poll-based forecasts have misfired in many ways in U.S. presidential elections, leaving pollsters, journalists, and pundits baffled or humiliated, and often without immediate explanation as to what went wrong.
Polls in presidential elections do not always go wrong. Or dramatically wrong. But they have been wrong often enough to invite skepticism and wariness. Indeed, it is a rare election that does not produce polling controversies of some sort. As Mack C. Shelley and Hwarng-Du Hwang wrote nearly thirty years ago, “The accuracy of presidential election polls has been argued in every presidential election year since polls first gained wide recognition” in the 1930s. Their observation is relevant still.
Just as no two presidential elections are quite alike, no two polling failures are precisely analogous. Not all polling failures are akin to the shock result of 2016. Their distinctiveness notwithstanding, polling failures tend to produce broadly similar effects—surprise, anger, bewilderment, and frustration at their failing to provide the American public with accurate clues about the most consequential of all U.S. elections.
Such reactions are hardly surprising, given that polls drive, color, and help fix news media narratives of presidential elections in the United States. They set expectations. They are central to how journalists, and Americans at large, understand the dynamics of presidential campaigns. Polls are critical to shaping conventional wisdom about the competitiveness of those races. And they have long been recognized as such. Years ago, in a series of articles about polling, the New York Post recalled that in “the fall of 1948, the press of the nation, acting on the advisory of the political pollsters, in effect recorded an event that never took place—the election of Thomas E. Dewey as President of the United States.” Or as George H. Gallup, opinion polling’s tireless evangelist, said about that polling failure, “We gave birth to a monster in 1948, the year when all of us pollsters elected Tom Dewey,” who lost to President Harry Truman.
This is not to say that polling failures are typically on the order of the epic fiasco of 1948 or the shock of 2016. In fact, some political scientists have argued that the record of election polling has been admirable in the United States and elsewhere. But polls have been in error often enough, and are beset by so many variables and potential contaminants (which pollsters don’t always discuss), that treating them cautiously is sensible. Election polls are not always accurate prophesies, and they certainly are not beyond challenge.
This book addresses in detail polling’s checkered record in U.S. presidential elections since 1936 — a record that rarely has been considered in depth and never collectively. It is a history that is not especially well known.
American journalists and, indeed, the American public are largely oblivious to the cata- log of polling flubs and miscalls. They may be faintly familiar with the “Dewey defeats Truman” polling debacle of 1948. They may have a nebulous sense that election polls were in error in 2016. But little else. This unfamiliarity— along with the certitude that can attach to polls and poll-based forecasts— surely contributed to the immense surprise that greeted Trump’s split-decision victory over Clinton. He handily won the Electoral College; she clearly won the popular vote, and yet lost the election.
This book does not dwell on trivial shortcomings. The polling failures considered here—whether spectacular or somewhat more modest—were all surprising and controversial when they occurred. They were much commented on at the time.
The book also addresses, and offers a fresh assessment about, the intriguing, intricate, and sometimes exasperating interplay between pollsters and journalists, a complex relationship that over the years has given rise both to sustained collaboration and to expressions of bitter disdain.
The virus of poll-hatred is not especially potent in the news media nowadays, but for decades many prominent journalists reveled in their contempt for opinion polling and poll-takers. And yet, paradoxically, there has never been a time when polling was not of some fascination to journalists—a fascination that persists, even if journalists are not entirely at ease with the intricacies or dynamics of survey research.
This book revisits prominent cases of polling failure while illuminating and presenting fresh insight into some of the characters, colorful and otherwise, who have shaped election polling and how it has been covered. Among these figures are George Gallup, the prickly founding father of public opinion research, who did much to promote and solidify expectations about the accuracy of election polling; Elmo Roper, a one-time retail jeweler and contemporary of Gallup, who by 1948 thought that election polling had little left to prove; and Warren J. Mitofsky, a brash yet brilliant innovator whose admonition to pollsters rings true across generations: “There’s a lot of room for humility in polling. Every time you get cocky, you lose.”
Once-prominent journalists also enter the narrative; they include David Lawrence, Haynes Johnson, and Jimmy Breslin—all of whom were practitioners of “shoe-leather” journalism, an intensive and revered kind of out-of-the-newsroom reporting that frequently has been considered a response or alternative to indulgence in election polls.
The book does not consider at length the well-documented troubles afflicting contemporary survey research—notably, the sharp and sustained declines in would-be respondents’ willingness to answer polls conducted by telephone, which once was the industry’s “gold standard” methodology. Since 1999 at least, pollsters have been experimenting with and incorporating internet-based approaches, generally with mixed results. Nor is the book steeped in the jargon and the opaque methodological arcana that pollsters and polling experts are keen to invoke, and for which they occasionally have been rebuked. Gallup, for example, once was taken to task for promoting impenetrable terms like “quindimensional analysis.” …
Polling failures and controversies arise from no singular template. Pollsters have forecast tight elections when landslides have occurred. They have pointed to the wrong winner in closer elections. The work of venerable pollsters has been singularly and memorably in error. Exit polling has thrown Election Day into confusion. Failed state-level polls have upended widely anticipated national outcomes. Poll-based data aggregators have miscalled elections. Narratives like these make for compelling accounts of expectations dashed and hubris exposed. …
This book argues that polling’s uneven, messy, and controversial past merits being addressed collectively and in its sweep. It proceeds in the recognition that polling failure often is correlated with journalistic failure. The correlation was pronounced in 2016, when poll-based prediction models and key state polls anticipated victory by Hillary Clinton.