Previous month:
January 2008
Next month:
March 2008

Posts from February 2008

Blind Studies on the Web

The CASRO Code of Standards and Ethics for Survey Research has recently been amended to allow for conducting blind studies on the Web.  This comes up fairly often in situations where a client has a list of their customers  they wish to survey anonymously in the belief that they will get more unbiased information if respondents don't know the survey sponsor.  The specific change in the new version of the code is:

The practice of “blind studies” (for sample sources where the sponsor of the study is not cited in the email solicitation) is permitted if disclosure is offered to the respondent during or after the interview. The respondent must also be offered the opportunity to “opt-out” for future research use of the sample source that was used for the email solicitation.

There are other changes to the code that I am very unhappy with.  They have to do with allowing the collection of very sensitive information such as SSNs, bank account numbers, passwords, etc.  But this one relative to blind studies is a positive step.

Big Brother Watching

I'm here at the CASRO Panels Conference and have just heard an interesting paper by Jeff Miller from Burke and Andrew Jeavons from Nebu.  They were inspired by the work of Bateson, Nettle, and Roberts (2006) showing that posting sets of eyes in a university lounge increased the generosity of faculty members depositing money in a box after helping themselves to a cup of coffee.  Miller and Jeavons designed an online experiment to see if including a pair of eyes in the logo on the upper left of the screen mediated undesirable panelist behaviors (e.g., straightlining, overly fast completion, etc.).

Their results were interesting.  They managed to produce effects in the expected/hoped for direction although they tended to be minimal and the absence of significance testing makes it hard to draw convincing conclusions.  They also looked at the flip side, that is, an increase in social desirability reporting.  They claimed there was none but the graphic was hard to read. 

I had two reactions:

  1. Banner blindness may have muted any effect.  It would be interesting to see if the results change when the eyes are somehow displayed in the question space.
  2. It seems counterintuitive that the eyes would create a social presence effect on the one hand (mediating bad behavior) but not also lead to increased social desirability reporting on the other.

In short, this was kind of neat but there seem to be some obvious flaws.  It would be interesting to see it replicated with a stronger design and some good old-fashioned significance testing.