Automated Telephone Surveys

I recently had someone ask me about the effectiveness of those automated surveys where the phone number is computer dialed and the respondent is immediately put into an IVR interview.  This is a pretty inexpensive methodology and the obvious question is whether these surveys are as "good" as a standard telephone interview with a live interviewer.

The companies who do these surveys claim that their results are every bit as good as those that use live interviewers.  (One of the loudest companies making this claim is www.surveyusa.com )  The real bread and butter for these guys is political polling and their main clients are media companies, especially TV stations.  They typically use RDD sample, a local on-air talking head to record the interview, dial away, and then apply standard demographic weighting techniques on the back end.  They will show you lots of data to indicate that this methodology generates survey estimates and response rates (well, maybe a little lower) that are comparable to what you get with a live interviewer.  So what's the downside?

• IVR puts a lot of limitations on interview length and complexity.
• Without an interviewer you have the classic problems of higher breakoff rates, respondent verification (you get whomever answers the phone), no ability to explain difficult questions, and (probably) higher missing data rates.
• While using a on-air personality may improve the response rate there also may be unmeasured bias precisely because the individual is known to the respondent and he or she may have strong positive or negative feelings about "the interviewer."
• It's not clear what kinds of response rates these surveys get outside of political polling or when an on-air personality is not the recorded interviewer.
• Without getting into a lot of detail they don't supply, we just have to take their word on the response rates.

OK, so that's a nice reasoned approach and one that takes pretty much at face value what these companies say about what they do and the kind of results they get.  But what does your gut tell you?  Mine tells me that I'm not at all likely to do what of these things should I fall into their sample.  If you can't even go to the trouble to have a human being call me, why should I got to the trouble to answer your questions?  Call it social exchange theory or human nature, but I just don't see this methodology as a serious competitor to a well-designed and executed CATI survey except at the very low end.