mead cohen berger shevtsova garfinkle michta grygiel blankenhorn
Three Kinds Of Lies
U.S. Poll Statistics, Now Even Less Reliable

Response rates to telephone polls are falling, causing the accuracy of many widely accepted surveys to fall with it. The rate of response to the average telephone poll has fallen from 36 percent in 1997 to 9 percent in 2012—and is possibly still dropping. Moreover, those who are responding tend to be disproportionately older than those who aren’t. At the Upshot, David Leonhardt reports on the consequences of this trend:

A new academic paper suggests that the unemployment rate appears to have become less accurate over the last two decades, in part because of this rise in nonresponse. In particular, there seems to have been an increase in the number of people who once would have qualified as officially unemployed and today are considered out of the labor force, neither working nor looking for work […]

Yet the research also relates to a larger phenomenon. The declining response rate to surveys of almost all kinds is among the biggest problems in the social sciences. It’s complicating our ability to understand how people live and what they believe. “It’s a huge issue,” says Alan Krueger, a Princeton economist and one of the new paper’s three authors. (Mr. Krueger, who recently spent two years as the chairman of President Obama’s Council of Economic Advisers, founded the Princeton University Survey Research Center in the 1990s.)

Read the whole thing. Telephone polls have always been skewed by ordinary problems with surveys like dishonest responses or questions whose phrasing trips up the respondents. But if low response rates are distorting results even further now, we have more reason to be skeptical of statistics that come from telephone polls. This doesn’t mean surveys are useless, but it’s a good reminder to always take them with a grain of salt.

Features Icon
show comments
  • Andrew Allison

    Polls of any sort are inherently unreliable. Many respondents give answers of which, for a variety of reasons, they think the pollster will approve; and many polls are designed to elicit the desired response. Polling provides revenue for the pollsters, headlines for the media, and not much else of value.

  • FriendlyGoat

    Well, “Is America on the right track or the wrong track?”

    With famous polls like these, respondents should have hung up on the whole enterprise decades ago.

© The American Interest LLC 2005-2016 About Us Masthead Submissions Advertise Customer Service