WAS: What Adults are Saying about Polls as News--Part One
The humble science of polls is badly contorted and confusing
Whether election year or not, imagine being a news and information consumer without encountering polls. You really can’t. Poll driven “news” has become the 21st century replacement to the new journalism of the 1970s. No longer are reporters “embedded” with their subjects. Instead they summarize reams of boiled-down data which “speaks to the views” of the people. Poll ubiquity has become a tool to drive coverage of every news topic imaginable but especially campaign politics. Across conventional and new media platforms, their information is leveraged by many practitioners chortling as a supreme example of credible vox populi—as though they held a recording of God visiting Mount Sinai.
But for all their widespread use, polls and what they can legitimately infer are terribly misunderstood by many in the data and information food chain. Then, once put into the hands of those who foremost want to maximize viewership, clicks or interest, or to support a vested interest (such as a political candidate), “polling data” soon become a polenta-like mass than can contort to explain just about anything. By no means does this invalidate polling. Instead it means those using, applying and digesting polling outcomes combine their often unintended ignorance with content development priorities to create this. Let’s take a look how.
Any poll lives under a set of methodological premises. Principally this means polls measure what we might call “opinion” in numeric terms and those sampled for measurement are meant to represent a broader population of people like them. Any poll novice is startled by this latter claim. It is intuitively challenging to consider that a mere 1200 hundred people in the US can, within certain limits, accurately predict current national voting preferences. A basic concept to which many of us have been exposed illustrates how.
A poll is comprised of a sample of persons possessing characteristics meant to represent a broader population (such as “registered voters). Think of the “bell” or “normal” curve as a graphic representation about any topic that interests you. Though you are now cleaning the mental cobwebs to remember its properties just know the vast majority of human behavior and characteristics are normally distributed. That is, when measured and graphed, that “shape” will look like a slightly flattened “bell” or normal, curve.
The premises behind this speak to the “probability” of all things. A weather forecast, a stock price, yards thrown by Aaron Rogers, school grades, height of Americans of Italian descent, IQ scores, degree of law observance when driving in school zones—you name it. Virtually all measured human behavior and characteristics are normally distributed. (BTW: Wealth distribution is a famous exception to this premise). This so called “normality” is what permits a poll sample to infer about a larger population with certain caveats and is fundamental to any type of statistical inference. When things vary from that normally shaped expectation is when statistical significance is found. So much for that!
You most likely encounter these “caveats” in the form of what is typically reported as margin of error. In a style repeated daily you will hear “candidate A holds a slim lead over candidate B though it is within the margin of error.” A report like this means two critical things related to consuming poll “news”:
In this example there is no difference of results between candidate A or B. One candidate is either ahead or not—there is no “in between”. This extremely common sentence is self-contradicting—having a “lead” within the margin of error is nonsense, and
far, far, far more importantly, all news organizations (big or small) have voracious time and/or space content holes to fill. Polls result analysis is a quick and cheap way to do this and endlessly contributes to the horse race of political campaigns that was discussed previously.
All of this conspires to undercut clarity when it may be available. Add in the volume of polls, their intent (not all are done for mere “statistics”), wide ranging levels of quality, at least a half-dozen or so methodological challenges faced by any pollster on any topic, time and money pressures and by gauging “opinion” one presumes to predict voting booth “behavior”, one quickly sees that polls require very careful consumption.
Most who use polls in a news context don’t know much of this and increasingly, don’t care. Big organizations such as a national network will have professional pollsters to aid and educate but they invariably are far more concerned about finding poll data to produce news “nuggets” then evaluating poll quality. What’s worse some of them strive to become part of the “news as entertainment” complex as seen in Steve Kornacki at NBC or the ridiculous Frank Luntz who further confuses things by over reaching with Focus Group results.
This leaves poll data consumers in a “buyer beware” place where they end-up either confused and distrusting all of it or cherry pick the results that fit personal biases. Neither are helpful if there once was a legitimate aim to use polls to enlighten news audiences about what actually is going on.
In Part Two you will learn how a fundamental premise of American media ensures this is not likely to change any time soon and what you can do to survive poll-mania this election season. Stay tuned….