Response rates to telephone polls are falling, causing the accuracy of many widely accepted surveys to fall with it. The rate of response to the average telephone poll has fallen from 36 percent in 1997 to 9 percent in 2012—and is possibly still dropping. Moreover, those who are responding tend to be disproportionately older than those who aren’t. At the Upshot, David Leonhardt reports on the consequences of this trend:
A new academic paper suggests that the unemployment rate appears to have become less accurate over the last two decades, in part because of this rise in nonresponse. In particular, there seems to have been an increase in the number of people who once would have qualified as officially unemployed and today are considered out of the labor force, neither working nor looking for work […]Yet the research also relates to a larger phenomenon. The declining response rate to surveys of almost all kinds is among the biggest problems in the social sciences. It’s complicating our ability to understand how people live and what they believe. “It’s a huge issue,” says Alan Krueger, a Princeton economist and one of the new paper’s three authors. (Mr. Krueger, who recently spent two years as the chairman of President Obama’s Council of Economic Advisers, founded the Princeton University Survey Research Center in the 1990s.)
Read the whole thing. Telephone polls have always been skewed by ordinary problems with surveys like dishonest responses or questions whose phrasing trips up the respondents. But if low response rates are distorting results even further now, we have more reason to be skeptical of statistics that come from telephone polls. This doesn’t mean surveys are useless, but it’s a good reminder to always take them with a grain of salt.