Previous month:
October 2005
Next month:
December 2005

Posts from November 2005

The Next Big Thing(s)

The current issue of Social Science Computer Review (Winter 2005, vol 23, no. 4)has a piece by Mick Couper titled, "Technology Trends in Survey Data Collection,"  in which he describes the latest technology-related trends in survey research.  The viewpoint is from one of the world's foremost survey methodologists and, as you would expect, there is a slight bias toward academic and government research. But his observations are interesting and useful:

  • Interview Administration to Self-administration. He sees two things are driving us in this direction: (1) interviewer administration is getting too expensive and (2) self-administration delivers more accurate answers on socially sensitive issues.
  • Auditory Communication.  IVR data collection is gaining in popularity and experiments have shown that test to speech (TTS) can be used to make set up of these applications more efficient as well as customize voices to respondent characteristics.  It's turning out that respondents do not object to computerized speech.
  • Visual Communication.  With the advent of Web-based surveys more and more visual information is finding its way into surveys.  But it's also proving to be a two-edged sword.  While images can be used to illustrate and stimulate, they also can produce unintended effects. We need more research to use them wisely.
  • Mobility.  Ten years ago the first laptops made CAPI the rage and smaller devices including PDAs have many applications in surveys.  As we become a cell phone oriented society there are new opportunities but also imposing challenges to capitalize on cell phones for survey data collection.

All in all, it reminds one of that old Chinese proverb: "May you live in interesting times."  For survey researchers, these are exceptionally interesting times.

The Psychology of Survey Response

The is the title of a book I have mentioned before in another context.  But before I go further I acknowledge that the lead author (Roger Tourangeau) is an old friend; one co-author (Ken Rasinski) is a former colleague; and the other co-author (Lance Rips) just has a very cool name for a guy in the survey biz.  None of this has anything to do with why I like this book.  I like this book because I firmly believe that if everyone read it we would be designing much better questionnaires that would do a much better job of getting our clients better answers to the questions they have.

The goal of the book is to advance "a theory about how respondents answer questions in surveys."  It deals with memory, with avoiding bias in question writing, with how to elicit both factual and attitudinal information, and how mode affects response.  It reviews the relevant psychological literature and its application to surveys.  In a nutshell, it places questionnaire design into the context of what we know about how people think and react.  Good stuff that we all should know and very readable to boot.

Cell Phones

Chet Bowie just sent me a publication from the Committee on National Statistics titled, "How Can We Conduct Telephone Surveys in a Cell Phone Age?"  This is a report from a two day meeting of experts in Washington in October.

At this point it's a somewhat reassuring story, although there no doubt is trouble ahead.  Despite what you may hear, the current best estimate of the percent of the adult population in cell phone only households (and not reachable by our standard telephone survey practices) is about seven percent.  And it's a very biased seven percent heavily over represented by the young and well educated in small, childless households.  In other words, just what we suspect and the same people we have had a tough time getting even before cell phone only households became an issue.  People who have attempted to survey these folks by cell phone have experienced higher refusals and lower response rates than landline studies.

So the conclusion here is that this seven percent is not yet a problem and the fact that we miss them in our standard telephone study is not a major bias issue unless you are especially interested in this subpopulation.  Exhibit A at this conference was a paper from the Pew Center pointing out the accuracy of 2004 election polling in which cell phone only  households were ignored. If you are interested in this subpopulation and we need to survey cell phone users, be prepared for a tough and expensive study.

Everyone expects that seven percent to grow over time and in that connection it's worth keeping an eye on what the Europeans are doing to cope with it.  A number of countries there, most notably Finland, have higher cell phone penetration than here in the US and have been thinking about this problem for years.