Previous month:
August 2009
Next month:
October 2009

Posts from September 2009

ESOMAR Congress

I've just returned from the ESOMAR Congress in Montreux.  This is  a great event to meet up with colleagues and partners, current and hoped for, from around the world.  But in all honesty, the content is not always as great as the networking opportunities.  I'm not going to comment on the overall content this year because I had a number of long meetings that caused me to miss to too many presentations.  But based on what I did see and conversation with other attendees it seemed to me that there were three overarching themes.

First, It was hard not talk about the economy, how bad it's been and when its' going to get better. The sub themes here are the same ones you hear everywhere--2010 a transition year; slow growth even after that in the US and Europe; faster growth in China and India; win business through innovation and efficiency; uncertain impacts of government regulation, etc.  All in all there were confirmatory messages with nothing especially new and startling.

Second, corporate social responsibility (CSR) is in.  At least ESOMAR believes it is because it was the main announced theme of the Congress, "Leading the Way: Ethically, Responsibly, Creatively."   A number of speakers asserted that the anxieties produced by Great Recession and the increasing emphasis on accelerating climate change are causing consumers to look more carefully at how brands are behaving relative to these issues and to factor that into brand choice.  While there was widespread agreement about the central concept there was considerable debate about the magnitude of the impact, especially from industry to industry.

Finally, there was this almost constant undercurrent of concern about two related issues that seem central to the European perspective on MR. The first of the issues is how we deal with respondents in a respectful and socially acceptable way.  This is about privacy, confidentiality, and the special problems of research with children.  The second is the need to continually define what we do and how we do it in ways that distinguish research from marketing.  I used to wonder why ESOMAR spends so much time on something like developing a definition of market research and now I get it.  It's ultimately about fending off regulation of the sort aimed at direct marketing.  This is a huge concern in Europe and researchers recognize that if we don't behave in an ethical and responsible way the freedoms we need to do our work could be seriously infringed upon.





This is worth your time

One of the links over on the right is to an online journal called Survey Practice.  Rather than me trying to describe their editorial focus and policy it's probably best to check it out firsthand here.  I point all of this out to you because the current issue is focused on nonprobability sampling and while I've not actually read any of the pieces yet a quick peruse suggests they are worth a look.


Disclaimers

Our knowledge manager sent me this article on Twitter the other day.  She 's always pushing Twitter at me because she knows I don't get it and views that as some sort of character flaw.  This article tries to answer the question on everyone's lips: why don't more teens use Twitter?  The search for the answer causes the author to do a survey of teens with an N of 10,000.  Well, actually it was even a few more than that.  He shows us a bunch of graphs and some commentary.  Then at the end he says this:

Disclaimer: Here is some more info on the panel of teens we surveyed. We don’t claim the 10,000+ survey results represent the definitive survey of teens in the US. We do, however, claim that our users look very much like the users of other social networks and that our audience overlaps significantly with MySpace, Facebook, and Twitter, and that the insights of myYearbook teens may be useful to this analysis.


I thought, Wow!  He gets it.  And he's not even a survey guy, as far as I know.  It made me think about the piece in a completely different way because he wasn't trying to make it out to be something it's not.

By telling us what it is he helps us to understand it better and ultimately, he makes it more useful to us. Now we can believe what he tells  us and evaluate it.   I think there is a lesson here about transparency,disclosure, and the usefulness of research results.


To tell you the truth . . .

Those of us who are old enough to remember the early days of online may also recall that one troubling finding was the disparity between customer sat ratings across modes, that is, online vs. phone. Online always seemed to be lower. There were multiple hypotheses advanced but once the empirical work got going it seemed to come down to one of two mode effects: good old fashioned social desirability bias or simply differences in the way people use scales in a visual mode like Web as compared to an aural mode like telephone.

I admit to having always been skeptical of social desirability as an explanation when it comes to something like satisfaction with someone's product or service. The literature on social desirability generally has focused on really heavy issues like drug use and abortion. Then I saw some stuff from Harris Interactive that claimed to find effects on behaviors as mundane as how often one brushes one's teeth or goes to church. And now an article (with too many authors to cite) in the summer issue of POQ goes even further and seems to show that subjective items such as attitudes and opinions also are affected by social desirability bias. The experiment was not with Web, but rather using a method they call TCASI (for Telephone Computer-Assisted Self-Interviewing). In this approach a person is called and the interview started with conventional telephone. At one point the respondent is switched over to IVR and the interview moves from interviewer administration to self-administration. In the experiment respondents were randomly assigned either to full telephone or to TCASI. Respondents in the TCASI treatment were more supportive of traditional gender roles and corporal punishment, less supportive of integrated neighborhoods and same –gender sex, and more likely to agree that occasional marijuana use is harmless and to describe themselves as attractive. In other words, the telephone interviews were more likely to deliver socially tolerant and politically correct answers than were the self-administered interviews.

The thing I like about this study is that it doesn't confound the issue with other factors in the way a straight Web-telephone study might. There is no visual dimension; it's all aural. There also is no issue of comparing panel sample (non probability) with probability.

Of course, the key issue in these discussions always comes down to the simple question: what is the truth and therefore the better method for getting at it?