The Survey Geek

More data on cell only households

While reports of the death of telephone interviewing continue to be premature it certainly is true that the double whammy of high rates of nonresponse and deterioration of the sample frame due to wireless substitution are serious problems in need of serious solutions. As I've noted in previous posts, current estimates put the percent of US cell only households at around 20 percent with perhaps an equal number of "cell mostly" households, meaning that they take most of their calls on their cell phone and therefore are very difficult to reach in the usual telephone survey design. Numbers like these have made it all but mandatory for a telephone research study to include calling cell phones if for no other reason than to maintain some level of face validity.

With this as backdrop Nancy Bates from the US Census Bureau has an interesting little piece out on Survey Practice. She has looked at the differences in preferences for online surveys versus mail surveys for households that are difficult to reach by telephone. While she finds only modest differences in Internet use across a continuum of cell only, cell mostly, cell sometimes, and landline only the preference for an online survey versus a mail survey across this same continuum is very striking. She reports that roughly 48 percent of cell only respondents report a preference for online versus just 13 percent for landline only.

Now there are multiple caveats here. She is asking about intent and it's not clear how behavior might differ. While I'm not going to try to dredge up the cites (you will just have to trust me) I have seen a handful of studies report that when given a choice between mail and Internet a significant plurality of Rs of all ages and stripes choose mail. In fact, we have a study going on right now that's driving home that very point. Nancy also is looking at filing our Census forms, not typical surveys that may be longer, more complex, and, frankly, less compelling. Finally, at some point we have to consider the possibility of mode effects using either of these approaches in a survey that also includes telephone interviewing.

Nonetheless, while I am offering lots of caveats I think this is a bit of research worth thinking about while we watch the telephone survey prognosis get darker and darker. But I also must confess that for the kind of work we do in MR simply calling cell phones may be the better path. Other sectors of the research industry may prefer another path.

If we knew what we were doing

At one point yesterday my wife showed me a wall plaque in a Signals catalogue with a quote from Einstein that read, “If we knew what we were doing it wouldn’t be called research.”  For some reason, it struck a chord.  And then this morning I was reading a review of the new Malcolm Gladwell book in the Times.   Now I confess that I pretty much loathe this whole genre of pop science (including other books like The Wisdom of Crowds and How We Decide) that has taken over the non-fiction side of the best seller lists all the while presenting itself as something we need to take really seriously.  And there seems to be no shortage of people in MR doing just that.  So when I read Steven Pinker’s review of  What the Dog Saw I literally jumped from my chair with glee.  He nails not just the Gladwell book but the entire genre when he describes the reasoning in Outliers as “cherry-picked anecdotes, post-hoc sophistry and false dichotomies.”  Sure, it’s entertaining stuff.  But it’s also lazy, shallow, and maybe even dishonest.  And then I thought about all that is being written and said about “the new MR” and I began to wonder whether we might be heading down this same road.  I hope not.

Einstein also said, “It’s hard.  That’s why we call it research.”

Let's hear it for conventional wisdom!

The post begins, "Contrary to conventional wisdom, U.S. consumers age 65 and older are most likely to shop online (77%). Consumers ages 45-64 and ages 30-44 follow at 71% with those ages 18-29 at 70%, a new report finds."  So 77%of persons aged 65 or older shop on line?  That's especially astounding because by most estimates only about 45% of persons 65 and older are online.  And presuming this study was done online (the post doesn't offer that detail) we need to also accept the reasoning that seniors on somebody's online panel are just like all online seniors in their use of the Internet.

Someone (The Cable and Telecommunications Association for Marketing) is actually selling this stuff!  One only hopes that nobody is buying it.