More from 3MC

Several of the sessions on the last day of 3MC were devoted to the general topic of response styles in multinational research. While I was aware that the problem of people from different cultures responding differently (especially in scale usage) this was the first systematic discussion I had heard on the issues. Given that we typically want to standardize measurement and produce reasonably comparable results across countries, understanding response styles and how to correct them either through survey design or in post survey analysis is critical if we are to do good quality international research.

A response style is generally defined as the systematic tendency for people to respond consistently to a range of questionnaire items on some basis other than the content of those items (see, for example, Paulhus, D. L. (1991). "Measurement and Control of Response Bias," in J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of Personality and Social Psychological Attitudes (pp.17-59). New York: Academic Press.).

Response styles are most often reflected in the use of ordinal scales of agreement, satisfaction, favorability, etc. They are best understood by example. The table below describes the seven response styles identified by Baumgartner and Steenkamp (Baumgartner, H. and Steenkamp, J (2001). "Response Styles in Marketing Research: A Cross-National Investigation, " Journal of Marketing Research, 38, 2, p. 143-156).

Style	Description
Acquiescence	The tendency to agree with items regardless of content
Disacquiescence	The tendency to disagree with items regardless of content
Net acquiescence	The tendency to show greater acquiescence than disacquiescence
Extreme	The tendency to endorse the most extreme response categories regardless of content
Response range	The tendency to use a narrow range of response categories around the mean response
Midpoint	The tendency to use the middle scale category regardless of content
Noncontingent	The tendency to respond to items carelessly, randomly, or nonpurposefully, a.k.a. satisficing

 

There is a considerable body of research that shows different response styles predominate in different cultures. So, for example, American respondents generally show a stronger tendency toward extreme response styles (ERS) than do Asians. Respondents from Mediterranean countries often exhibit stronger acquiescence and extreme response styles than do those from northern Europe.

As survey researchers we have two options. The first, and hardest, is to use analytic techniques that help us to score individual respondents and then make statistical adjustments. This is tricky stuff, at least for me. The easier path would seem to be to design surveys to be response style neutral. One frequently advanced solution is MaxDif. While there is strong statistical support for use of MaxDif we've done some research that shows it's a tough technique for respondents to use, at least in Web surveys. Another approach involves experimentation with scales to identify those designs that seem less susceptible to differences in response styles. Number of scale points and full labeling versus endpoint labeling are the typical features of scales that get varied in experiments, but as near as I can tell there is not a whole lot of agreement as to best practice.