One of the downsides of being a compulsive conference goer is that you tend to hear essentially the same paper, although by different people from different companies, on more than one occasion. One of the ways in which the MR research on research world differs from that of academic methodological research is the complete isolation in which people seem to work. In academia, researchers are expected to be familiar with and to cite the work of others doing similar work. Market researchers seem to feel no such obligation and so it's not unusual to see similar research designs executed and reported on by different people at different points in time with no point of reference in terms of previous research findings by others.
It's in that vein that I report on some work presented by a researcher from Lightspeed at the recent CASRO Panels Conference looking at how panel response rates vary based on the day of the week when the initial invitation is sent. While the authors do not reference previous work, having seen a number of other similar presentations over the last few years I think there is a consensus evolving around this issue. That consensus seems to be that the invitations mailed earlier in the week generally produce a higher response rate than those mailed later in the week.
Now I admit that mostly I have ignored this topic because a response rate on a convenience sample is a pretty meaningless thing. Why does it matter? More recently it's occurred to me that it does matter if you are trying to access a low incidence population or the panel is known to have a limited number of panelists, say, in a smaller geographic area. So there are conditions under which panel response rate does matter because it ultimately could determine whether you can meet quota.
The impact of day of invitation as reported by Lightspeed was not dramatic, falling gradually over the week from a high on Monday afternoons of 39 percent to a low on Friday afternoons of 28 percent.
As a practical matter, how important is this difference? Well, let's assume that the panel only has 3000 potential respondents for the demographic of interest. A Friday launch will get you about 330 respondents fewer than a Monday launch. Suppose further that you have other qualifications that are going to yield you an incidence of 30%. Under those conditions, the Monday launch would yield 351 completed surveys while the Friday launch just 252 surveys.
In most instances there is a lot of panel sample available and we don't need to worry about this issue. However, when we are worried about whether a panel can provide us with enough sample the day of the week could well be the difference between hitting a quota and falling short.
One final note. None of the studies I've seen look at Saturday or Sunday launches because panels in general discourage it. But the trend in these data suggest that we would see still lower response. It would seem that there is little advantage in launching on Saturday rather than waiting until Monday afternoon