Those pesky robo-polls

A new issue of Survey Practice is out and among the short articles is one by Jan van Lohuizen and Robert Wayne Samohyl titled "Method Effects and Robo-calls." (Some colleagues and I also have a short piece on placement of navigation buttons in Web surveys.) Like most people I know I have little regard for the accuracy of robo-calling as a competitor to dual frame RDD/cell phone using live interviewers and this article provides some grist for that mill. The paper looks at 624 national polls and the specific issue of Presidential approval. I'll just quote their conclusion:

" . . . while live operator surveys and internet surveys produce quite similar results, robo-polls produce a significantly higher estimate of the disapproval rate of the President and a significantly lower estimate for 'no opinion', attributing the difference in results to non-response bias resulting from low participation rates in robo-polls."

So far so good. But it reminded me of a report I'd recently seen (via Mark Blumenthal) about the latest NCPP report on pollster accuracy. In this study of 295 statewide polls in the 2010 cycle the average error in the final outcome was 2.4 percentage points for polls with live interviewers versus 2.6 for robo-polls and 1.7 for Internet. Of course, accuracy on Election Day is not the same as accuracy during the course of the campaign. As even casual observers have noticed there is a tendency for all polls to converge as the election draws near. As this excellent post by Mark Mellman spells out, robo-polls may do well on Election Day but not so well in the weeks prior. I won't speculate as to the reasons.

But I take comfort in all of this. It's always nice to have one's prejudices confirmed.