The Survey Geek

What about B2B?

Virtually all of the chatter across the industry these last three years or so about panel data quality has been focused on consumer sample.  Those of us who don't do a whole lot of online consumer work have now begun to wonder, what about B2B?  Is there a data quality problem there as well and, if so, is it being addressed by the plethora of quality initiatives around the industry?  The answer seems to be both yes and no.

Brad Bortner at Forrester has been paying attention to the B2B issue and overviews some of the concerns in this post, but I'm not sure he has hit the key one which, to my mind comes down to whether these respondents have the jobs and the responsibilities that they claim to have.  It is becoming increasingly common for panels to validate their members by bumping their member info up against big databases like Acxiom, but that only confirms that a person with this name lives at this address and maybe has this SSN.  It doesn't confirm, for example, that she also is general manager of a manufacturing company employing 750 employees and the person responsible for making decisions about the company's energy supplier.  No doubt some panel companies will disagree with this, but it's not at all clear to me that respondents in B2B surveys always are who they say they are and have the responsibilities they say they have.  And given the generally higher incentives for B2B, the temptation to overstate one's qualifications may be too compelling to resist.

One notable exception is physicians where the price of admission for any panel company is a rigorous validation process.  Of course, that's one reason that it will cost you around $75 before incentive to get an oncologist into your survey but maybe only $15 to get a CFO at a small business. 

Inside Research tells us that just over 15 percent of US online is B2B.  That, along with the CPG focus of the major client company players in the panel data quality debate, may explain why the issue has been getting so little attention.  But you have to believe that its day is coming and it may prove to be a tougher nut to crack than consumer has been.  Many people believe that professional social networking sites such as LinkedIn are an important part of the solution, but at the moment these folks seem a bit lost as to how to leverage the resource they're sitting on.

Cell only households continue to rise

CDC has just updated its estimate of cell only households from the National Health Interview Survey.  For those of you not paying attention, this is pretty much the gold standard for tracking the growth of cell only households.  CDC now reports that as of June of 2009, 22.7 percent of US homes have at least one cell phone but no landline.  Another 14.7 percent report having a landline but mostly using their cell phone(s) to place and receive calls.  Putting the two together, this means that a standard RDD telephone survey is likely to miss about 37 percent of households unless it is augmented to include cell phone sample.  While it adds time and expense, calling cell phones is rapidly becoming a standard feature of US telephone surveys.

I'd like to find a silver lining in this but can't.

It's all relative

Yesterday, for the first time in many months, I visited my local audio store to check out the latest equipment and to see how they've been weathering the recession.  When I asked how business had been the owner told me that last year people were saying, "Flat is the new up." He says that this year they're saying, "Open is the new up."

This same sort of relativism seemed to be at work in this press release last week from Clarabridge, the text mining people, and Harris Interactive.  It compares results from an online Harris Poll on US healthcare reform with online chatter during the same period.   In its survey Harris found that support for the current healthcare legislation had fallen to 40 percent by mid-November.  The Clarabridge analysis of social media pegged support at 65 percent during the same period.   What exactly am I supposed to do with that? The best I could do is conclude that social media has the huge bias that we all fear it has.  That it's basically a qualitative method to be valued for its depth and richness rather than its accuracy.  That even though we can count things it's not quant research in the traditional sense and we should be really careful with any conclusions we might draw.

But there is a larger issue here as well and it's the matter of fact presentation of an online survey as "survey research" and the gold standard against which the accuracy of social media should be measured.  They might have been better served by a comparison to a Gallup Poll  done by telephone during the same period showing support for healthcare reform never dipping below 48 percent. (And, yes, Gallup called landlines and cell phones.) At least it's closer to their 65 percent.  It makes you wonder whether we can find the truth any more or, worse yet, if anyone cares.

In Decembers past I often have left the audio store with some sleek new piece of hi-fi gear.  This year I left with a single record.  Yes, a record, not a CD.  I own plenty of both but I'm mostly a vinyl guy, which probably explains a lot.