The Survey Geek

ISO 20252 – Market, opinion and social research

In December we were audited for ISO compliance. After a week of staff interviews, project reviews and records inspections we were certified to the 20252 standard. Good for us!

Now I recognize that there are more than few people in the industry who think ISO certification is some kind of retro thing that does not reflect how much the industry is changing. More on that in a bit. But first let me say that when it's all said and done ISO 20252 comes down to four pretty basic things:

1. Making sure that people know how to do their jobs and can succeed in the assignments we give them.
2. Ensuring that everyone on a project team understands their individual roles, the roles of their colleagues and how to work as a team to produce great work for their client.
3. Being completely transparent with clients about what was done and how it was done so that they can make whatever business decisions they need to make with confidence.
4. Minimizing the mistakes that generate rework and disappoint clients while maximizing the chances of learning from those mistakes.

Now back to those folks who say that ISO is for the dinosaurs and out of place in the fast-moving world of contemporary market research. Or that all that process ISO insists on works against the innovation that is the supposed life blood of boutiques and startups. I think that's nonsense. A well-defined infrastructure of process and procedures need not be a barrier to change. In fact, it can make it easier. In Good to Great Jim Collins observes that in great companies people work within the framework of a highly developed system that fosters what he calls "a culture of discipline." When you have a culture of discipline you spend less time managing people and more time managing the system. And when change is called for you change the system, not the people. That's also what ISO brings you. I recommend it to you one and all.

A butterfly flaps its wings . . .

My head is still spinning from having just read Lenny Murphy's quick analysis of the SurveyMonkey acquisition of some of the assets for MarketTools. In his piece he describes MR as an industry under siege, in a state of chaos that ultimately may dump the industry into the "dustbin of history." Three things about this bother me.

First, I'm not biting on the rumor sourced from "a person with knowledge of the matter, who spoke on the condition of anonymity" that SurveyMonkey has a valuation of $1 billion and therefore needs to be taken seriously. Does anyone really believe that a SurveMonkey IPO would raise a billion? Or that anyone would consider writing a check of that size to acquire them? They may be the 800 pound gorilla in the DIY space but that hardly puts them in the upper echelon of research companies.

Second, the day has long since passed when a MR company could compete on the sort of simple and straightforward data collection that DIY makes possible. Any company in this industry that has not figured out a much more compelling value proposition is either already in that dustbin or about to fall in.

Finally, there's no question that the main impact of DIY has been and will continue to be a dramatic increase in the number of really bad surveys flowing into people's inboxes. However, that's not what ultimately will determine the impact of DIY. DIY will be judged like any other research approach. Clients will decide whether or not it's helping them make better business decisions, improve their business results. If they can get the same or better insights from DIY than they get from their MR suppliers then we deserve to be sent packing.

I'm not as worried as Lenny.

Eric Salama on Radio NewMR

This morning I listened to an interview of Eric Salama, CEO of Kantar (and now TNS as well), on Radio NewMR. I know Eric Salama by name only and was interested to hear his take on the future of the industry.

Overall his message seemed to be that our industry should focus on its strengths rather than becoming distracted by worries about big players from other industries coming in and eating our lunch. There are now many sources of insight that go beyond the world of surveys and focus groups. MR might exploit some of them but he sees no mandate to exploit them all. Four things in particular stood out in my mind:

• Surveys are not dead yet. There are some things you can only learn by asking people questions.
• There sometimes is a disconnect between what we hear about the explosion of new methods and client spend. His specific example was social media whose current share of client spend he characterized as "unimpressive."
• MR firms generally lack the technology to compete with major players like IBM when it comes to Big Data. Crunching massive databases of unstructured data is not what we are good at. We are good at analysis and that's where we should focus.
• Qualitative is a major opportunity, at least for a big player like Kantar. He sees the qual side of the industry as still very fragmented and a place where there has been a good deal of innovation still to be leveraged.

The interviewers—Ray Poynter and Lenny Murphy—did their best to get him to bite on more apocalyptic futures for MR but he would have none of it. Still, it was an interesting half hour. But don't take my word for it. You can hear a recording of this morning's show along with previous shows here.

Plus ça change – Part 1

Sometimes it feels like if you are around this industry long enough you begin to see things recycled, old ideas dressed up in new clothes and paraded out as the next big thing.  That’s how I’m beginning to feel about  big data.  I find myself increasingly reminded of the twin next big things of the 1990s: data mining and CRM.  Both of them had the power to scare the pants off market researchers by threatening two of the basic underpinnings of our industry: interviewing and privacy protection.  Both were believed to have the power to make what we do obsolete.  As far as I can tell, it didn’t happen.

The here-we-go-again aspect of big data was brought home to me while reading this article in the MIT Sloan Management Review  dealing with the obstacles to adoption of big data and analytics.  The impediments the study found are remarkably similar to those that bedeviled data miners and CRM specialists over a decade ago.  There is the same “failure” of managers to understand the power of leveraging data in different ways and the same “islands of data” problems on which enterprise-wide data management foundered.

I suppose the major difference on this go around is that technology is a lot better, the analytic tools more powerful and the data easier to come by.  At least that’s what I hear.   But those improvements don’t solve the core problem which would seem to be the culture of the enterprise.  That’s why I don’t think big data is going to put us all out of business just yet.  Do I think that over the long-term big data will be practiced on a broad scale?  Absolutely.  I’m just not going to lose any sleep over it this week.

Part 2 will be about rediscovering some of the basic principles of questionnaire design and discovering that there is more to it than eliminating grids.

Boys and toys

The title of this post is an expression a friend uses to describe the apparent male affinity for gadgets of all kinds—computers, cars, mobile phones, power tools and consumer electronics in generl. I'm not saying that MR or even specifically the NewMR is a male dominated business, but nothing excites us more than cool new toys.

This came to mind when I read Jeffrey Henning's latest post over on the Affinnova blog about still another presentation about neuroscience. Now I am completely on board with the idea that emotions play a much larger role in consumer decision making than our methods generally have acknowledged, and that we need to find ways to evolve those methods to take that into account. But, as I hinted in a previous post, I have concerns about whether neuroscience, or at least our imperfect understanding of it, is the right solution. Neuroscience is difficult stuff, even for people who do it all the time and it's not at all clear that we really know how to accurately interpret the reactions of different parts of the brain to different stimuli.

So as I was reading Jeffrey's post I kept wondering whether wiring up focus group participants like Alex in A Clockwork Orange offered insights above and beyond what we have been doing the past 25 years with dial testing. In dial sessions people may not be able to clearly express why they feel more positive or negative about an ad as it plays out but the squiggly line probably will tell you when they started to feel it and by how much. With a little probing they can probably tell you what in the ad triggered that feeling.

What's missing from the presentation that Jeffrey describes (or at least in his telling of it) is the controlled experiment that demonstrates why what we learn from one technique is better than what we learn from another. IMHO, too much of the NewMR is built on assertions that this is better than that but with too little data to make the case.