Some things we should already know

Earlier in the week Jeffrey Henning (IMHO the best of the MR bloggers) served up a much-praised and frequently retweeted post on why respondents abandon Web surveys. His post does what most of the respondent engagement debate does not do and that is get down to some basic facts about what it is in surveys that turns off people who already have volunteered to do surveys. Mostly that debate has started with the Flash imperative and never looked back.

I have taken the liberty of posting Jeffrey's chart below. It shows that the biggest single cause of abandonment is uninteresting subject matter. Media downloads are a bit of a red herring because the majority of surveys don't rely on multimedia so let's put survey length as the second biggest factor.  Now we can generalize that the main causes of abandonment are subject matter and length. Those grids that are so roundly condemned amount to a mere 15% (sorry, Andrew). I would extend Jeffrey's argument just a bit and suggest that outright abandonment is an extreme behavior and that many more complete boring and long surveys in a half-hearted way just to get the incentive.

What strikes me most about all of this is that it seems to be news to way too many people. The influence of topic and burden on survey participation has been talked about for decades. For one especially relevant discussion have a look at Nonresponse in Household Surveys by Groves and Couper. Now I know it's mostly about in-person surveys so it can't possibly teach us anything about Web surveys, but if you were to read the chapter about how survey design affects participation you might appreciate that, regardless of mode, topic salience and perception of the burden of participating (a.k.a length) are key. (Other important elements of survey design such as incentives are bridges that panel respondents already have crossed.)

The question in my mind is why don't we already know this stuff? Why do we have to keep relearning the basics? And why do we let ourselves get distracted by all of this talk about engaging people with cute gadgets and eye candy? What respondents really want is shorter surveys on topics they find interesting. The Web may change a lot of things but this isn't one of them.