Is the online data quality issue fading?
December 08, 2008
So asks Inside Research in its December issue. The cited evidence is from the latest so-called "Research Industry Summit" hosted by RFL Communications where attendance was off steeply from previous conferences and there just was not a whole lot new being said. For some while now we have been seeing panel vendors and researchers alike focused on the major abuses that the industry has been talking about for three years now--false panel registrations, false qualifying for surveys, heavy survey taking, speeding and satisficing, etc. All of these issues are being pretty effectively addressed across the industry which in turn is putting a lot of people's minds at rest.
But is that enough? The obvious remaining question is whether these problems are the real problems or are there more fundamental issues with online panels that has produced the kind of inconsistent results that started the discussion? I suspect there are and that they soon will evidence themselves. The industry continues to treat panel sample much like it has always treated probability samples. Until we recognize that they are a much different beast and that we need a whole new way of dealing with them I believe we will continue to see inconsistent results that will cause clients to doubt their validity. There is much work yet to done.