The indomitable Simon Chadwick has a a little piece in the June issue of Research World in which he reports on the results of Cambiar's annual survey of the research industry--clients and suppliers. (You can get a topline of the report from the Peanut Labs site here.) One of the ongoing themes of this survey since Simon launched it way back in 2005 has been concern about the quality of online sample, and that concern is front and center again in 2009. The concern has changed some over the years from one of worry about recruitment sources and procedures to those of respondent engagement, representivity, and questionnaire design. No surprise there. But what is something of a surprise to me at least is his finding that "Clients . . . appear to be rather less concerned about data and sample quality than research companies are." Ummm. It seems to me that one underlying premise of the industry frenzy on this issue has been that clients are worried about online quality and we'd better get some good answers and get them fast. But Simon's data suggests that may not be the case. Further, he points out that despite concern about online quality clients use of online and even expansion into new online methods continues unabated. But what does it mean? Do clients not require the level of precision and certainty that we think they do when they make business decisions? Or are the issues so arcane that they just don't understand them or care about them?
We say that MR is important as a means to help clients understand their markets and their customers. Findings like this make we wonder how well we as researchers understand ours.
Simon is right and, perhaps, wrong. Client researchers don't care about data quality nearly as much as do suppliers, but that has been the case over the past three years. I have personally discussed how to elevate data quality with specific client-side researchers, but in general, they don't understand what is wrong. In general, they are not inquisitive about what it going on. In general, they believe it is their suppliers' problem and, more or less, sole responsibility. I wrote two years ago that the industry and its associations must come together and engage in a re-education program to raise awareness and build a demand for action in support of superior data quality. We at RFL Communications are going to continue pushing that and related agendas. Remember that you heard it here first and join us!
Posted by: Bob Lederer | July 13, 2009 at 07:10 PM
This is the "indomitable" Simon Chadwick - I have to trade mark that description! Are we really surprised that clients are less concerned? Yes, we had a flurry of concern among high-profile clients stirred up by Bob Lederer, but are the bread and butter clients with whom we deal on a day to day basis really that concerned? Are they trained enough to be concerned? Another data point in our survey was that clients are increasingly investing in their own panels and communities, and in bringing online research in-house.
It is time to be afraid, my friends. The fox is in the hen house and knows nothing about laying eggs.
Posted by: Simon Cjhadwick | July 10, 2009 at 01:42 AM