More on Social Desireability

Last week I sat through a very large panel discussion at a CASRO meeting and heard a lot of industry people express their views on the pressing problem of declining respondent cooperation.   There was, of course, discussion about online as well as mixed-mode and there was no shortage of people prepared to argue that online is better because people don't always tell the truth to interviewers.  I came home to the latest issue of POQ in which there is an article reporting on something called "T-Acasi" which stands for Telephone Audio Computer-Assisted Self-Interviewing.  Among the authors is Charles Turner from the Research Triangle Insitute who was one of the early proponents of offline Acasi.  In this original approach in-person respondents listened to questions played through earphones from a laptop and took a CAPI interview without the interviewer.  In T-Acasi, the interview begins on the telephone and at one point in the interview the interviewer disappears and the interview becomes sel-administered via IVR.

The research record on self-administration whether by paper or Acasi has pretty clearly demonstrated that you get better answers to sensitive questions about things like drug use, sexual behavior, and abortion.  "Better" in this context means higher reports of these socially sensitive behaviors.  It's not at all suprising that these results replicate for T-Acasi.

The question for us, and the subtlety that escaped those CASRO panelists, is whether this finding translates to the kinds of questions we ask in MR such as satisfaction, propensity to purchase, or smoking.  The experiments we've done suggest that the effect, if any, on satisfcation surveys is pretty minor and manageable, but we may have a bigger problem with behavioral issues like smoking.  But the smoking issue has been clouded (no pun intended) by the fact that comparisons have involved RDD telephone vs. Web panel sample, so the sample bias is a major confounding factor.

The bottom line is that social desirabilty effects are real and we need to be alert to them, but they may not be as major or consistent across different behaviors or attitudes as some people might think.  The unfortunate answer here as in so many of these methodological issues is, "It depends."