For some while now I have advocated against the use of drop downs in Web surveys. That's been based on a number of things--some research we've done with our UM colleagues, an emerging consensus in the usability literature, and my own personal experience.
Benjamin Healey from Massey University in New Zealand has just provided what I hope is the last nail in this coffin. He tested drop downs against standard radio buttons and has published his results in in the current issue of Social Science Computer Review ("Drop Downs and Scroll Mice: The Effect of Response Option Format and Input Mechanism Employed on Data Quality in Web Surveys."). His findings are straightforward:
- Drop downs produce higher levels of item nonresponse than radio buttons.
- It takes respondents longer to answer using a drop down than with a conventional radio button.
- A majority of respondents (76 percent) using scroll mice accidentally changed an answer at least once.
- Respondents in the drop down condition were more likely to choose answers toward the bottom of the answer set than those with radio buttons.
So higher respondent burden and lower quality data. It makes you wonder why we continue to see these things in surveys.