Mode Effects: Part Two
October 31, 2005
Arguably the biggest difference between Web surveys and phone surveys is the absence of an interviewer. (There's also that visual vs. aural stuff, but more on that in another post.) And when people see evidence of Web studies producing different results than phone surveys they often jump to "social desirability" as an explanation. The theory is that we all want to present ourselves in as favorable a light as possible when we interact with other people, even survey interviewers. So, for example, when an interviewer asks, "Do you smoke cigarettes?" we might be inclined to answer "No" because we don't want the interviewer to know that we have so little will power that we cannot give up a habit that in all likelihood will end our lives prematurely. We answer "No" even though we smoke a couple of packs a day. But when the question is on a computer screen with no one there to judge us we might be more inclined to fess up. Or so the theory goes.
In truth, there is lots of research to show that self-administered surveys yield higher estimates of things like drug use, abortion, extramarital sex, and alcohol consumption than do interviewer-administered surveys. It seems pretty clear that on socially sensitive issues, social desirability is at work. But on questions about satisfaction with your electric company? Or with your insurance agent? While we might see differences between Web and phone on these kinds of questions, is it really about respondents wanting to appear positive and optimistic to telephone interviewers?
I don't have the data to prove it but my hunch after looking at a lot of these studies is that social desirability is a major issue for many of our healthcare surveys and converting them over to Web will be problematic. But for much of our satisfaction work I don't see a similar problem. I suspect that the differences we see there, which often are quite small, are more about seeing a scale displayed on a screen rather hearing it read over the phone. As I said, it's mostly a hunch. More on that later.