The Survey Geek

Innovate or ?

I spent most of this week in Atlanta at the Insight Innovation Exchange. All I can say is, “Whew!” There must have been 100 presentations over the three days, most a merciful 20 minutes with a few meatier 40 minute sessions mixed in. Coming on the heels of MRMW in Chicago just two weeks ago I am in a serious state of information overload.

To state the obvious, the whole point of this event was to showcase innovation in MR. It impressed me as a sort of MR version of the Consumers Electronics Show, and I mean that as a compliment. Several years ago (I can’t remember exactly when) I was interviewed by ESOMAR’s Research World on the topic of, “Is market research innovative enough?” (I said I thought it was.) Then again, several of the conferences I’ve attended over the last couple of years had speakers worrying about us not innovating enough. I am hard pressed to imagine anyone coming away from Atlanta believing that there is not enough innovation in market research.

But there is a catch and a couple of panel sessions brought that home: the take up rate with clients is nowhere near what some would expect given the explosion of new techniques. The innovation funnel is full if not overflowing, but other than a few home runs like mobile and communities, not much is filtering down to the mainstream (forgive the mixed metaphor).

One reason for that, I believe, is that many of these innovations are being driven by entrepreneurs from outside of MR whose main focus is the technologies themselves, rather than a clear understanding of clients’ unmet needs. Clients will always listen to faster and cheaper, but so far they don’t seem to have found the rest of the value proposition offered by most of these innovations all that compelling.

 I heard Innovation guru Clayton Christensen mentioned more than once but no one offered up my favorite quote from The Innovator’s Dilemma. “Customers control what companies can and cannot do.” There was a lot of navel gazing in Atlanta, and I did my share, but getting to know our clients better might move us farther faster.

Whither mobile?

I spent two days last week in Chicago at Market Research in the Mobile World (MRMW). I now have been to all four of the North American versions of this event, all the way back to Atlanta in 2011. Last week’s iteration caused me to go back and look at my post about the Atlanta event as a way to gauge how much things may or may not have changed.

One obvious change was the sorting out of marketing applications from genuine research. In Atlanta we heard from too many companies whose business was collecting personal data for direct marketing, and not always transparently. As far as I could tell, the presenters in Chicago were primarily focused on research. A second change was the emphasis on case studies. Atlanta was mostly about the potential for mobile—what could be—while Chicago was mostly actual studies completed.

Atlanta also was heavy on hype and there was plenty of that over the two days in Chicago, including a fair amount on the potential of wearbles. But it was hard to get too worked up about any of it given the sobering start.

• The first presentation was a paen to mobile as “the most important marketing channel, ever” that included the claim that 55% of people really like targeted ads, a figure undermined by virtually every credible survey on online privacy.
• That was followed by an update on last year’s eye-popping announcement by General Mills that they hoped to be doing 80% of their research via mobile in 2014. No exact figure was given but it was clear that it turned out to be harder than they thought. They have dismantled their mobile team,  but they keep on plugging.
• The segment was closed out with a panel of industry heavyweights on the topic, “Investments in (mobile) MR—where are they going and why?” The answer: they are not. The money is all going to big data analytics.

That was followed by about 30 presentations with an overwhelming emphasis on pure mobile applications: in-the-moment, geolocation, and mobile ethnography (as opposed to the unintended mobiles that comprise the bulk of mobile MR right now). Some were genuinely interesting and others seemed like sales pitches. The highlights and lowlights in no particular order:

• There were a couple of nice papers on the topic of integrating mobile with other platforms as a way to understand the context in which people view different ads.
• I heard no discussion of the sampling challenges beyond a preliminary report from TNS aimed at allaying concerns about bias in online surveys that include respondents using mobile devices. One might have gotten the impression from most presentations that there is 100% smartphone penetration and people are willing to use them to do pretty much anything researchers ask them to do.
• The potential power of mobile ethnography was nicely demonstrated by several presentations.
• There was a somewhat bizarre though seemingly heartfelt epilogue to one presentation pleading with us not to be swayed by the media into giving up on Google Glass.
• There was an equally bizarre presentation on mobile’s ability to reduce social desirability bias and satisficing that opened with the presenter acknowledging that all he knew about the phenomena was what he learned from their Wikipedia entries. It seemed to be just another study on recall.
• There continues to be some sloppiness around proper privacy protections, especially in mobile ethnography. There was a panel on the topic (I was on it) but less than half of the attendees were in the room.
• A number of presentations took up the theme of respondent-driven design that emphasized better interfaces, choice of channel, and better-designed if not shorter surveys. On this latter topic, one presenter showed that a well-designed 15-minute survey is possible with no discernible data quality drop off.  No doubt music to the ears of clients (of which there were very few) and even researchers who cling to long surveys like an NRA member clings to his assault rifle.

My bottom line is that things have changed substantially since Atlanta, but nowhere near as much as mobile evangelists predicted. Mobile has become mostly an extension of online, and while there is no shortage of startups offering in-the-moment and other pure mobile solutions there also are no clear signs yet of research buyers flocking to them. Mobile has changed substantially how we interact with one another and the world around us. It must be a fundamental concern in every research design, but it has yet to truly transform MR in any meaningful way.

That old seat at the table

Attend an MR conference and chances are good that sooner or later a case of consulting envy will break out.  Symptoms include repeated use of phrases such as “seat at the table” and “C-level access.”  This is not a new phenomenon; it’s been a pretty steady part of the MR inferiority complex for the almost 20 years that I’ve been in the industry. The syndrome manifested itself at last week’s ESOMAR Congress in a session titled, “Think BIG: Imagine.”  It started from the premise that MR is twice as old and yet only one-tenth the size of management consulting (poor us) and then wondered how we might create “exponential growth” to close the gaps in revenue and prestige.  The answer, it turns out, is behavioral science.

To be clear, I agree that behavioral science is a megatrend with the potential to transform our industry.  But will its smart application make us rich and at long last bestow upon us that mythical seat at the table?  I doubt it. 

Management consultants and market researchers work at fundamentally different levels.  The consultant’s playing field includes corporate strategy and the full range of company operations, the whole enchilada.  Market researchers primarily provide tactical advice on how to take products to market. The two value propositions are fundamentally different and they result in fundamentally different ways of interacting with clients.

I am tempted to say that we should just get over it, enjoy the work that we do and take pride in the value we deliver. Then again, as tiresome as it can be maybe our fundamental insecurity is a good thing.  Maybe the constant fear of our lunch being eaten by management consultants and predictive analytics keeps us on our toes, pushes us to innovate, and makes us better if not bigger.  Andrew Grove probably was right; only the paranoid survive. 

I think we check that box.

Thoughts on the 2013 AMSRS National Conference

    I spent most of last week at the 2013 AMSRS Conference in Sydney.  Lovely city and a great conference. There were over 500 attendees and around 40 presentations along with some interesting panels, two of which I had the opportunity to sit on.  The content was all that you would expect in an MR conference these days, a good reminder that we are a global industry in which pretty much everyone is working to solve the same set of problems.  The presentations I heard were all well done, better than what I sometimes see at other conferences.  Unfortunately, I don’t have any breakthroughs to report but it was a nice opportunity to think about the state of play in MR.

One nice feature of this conference was the significant representation from the public sector, some clients but mostly companies that do what we in the US might call social policy or government research.  By and large, MR and government researchers go their own separate ways here but in Australia and to a lesser extent in Europe that’s not always the case.  I have always felt that both MR and government research in the US would benefit from closer collaboration, but I don’t expect to see that happen any time soon.

There also was a significant contingent of client-side researchers, more than one usually sees in the US save for the artificial environment created by TMRE.  And, as conference co-chair Ray Poynter wryly observed, they weren’t hiding their name badges, suggesting that they felt more like colleagues than targets of opportunity.

There were five plenary sessions, mostly featuring the usual suspects (like me).  Our job was to be thought provoking and I think we managed.  I’ll leave it at that.

To accommodate that many papers there had to be some parallel sessions, and I attended three of them.  Some observations:

·       I’m not sure that MR has yet tuned into big data in a meaningful way.  It’s more than loyalty card data or social media data.  It seems to me that it’s more about combining different kinds of data from lots of different sources, and more about the techniques of data mining and machine learning than it is about sampling, inferential statistics, and hypothesis testing.  We continue to view big data through the lens of our current paradigm, and I don’t think that’s where the big data movement is going.

·       Mobile seems frozen in place.  Everyone continues to talk about “in-the-moment” and geolocation but I don’t see mobile as being really there yet in a meaningful way.  The unintentional mobile respondent issue is where the action is right now and I’m surprised it’s not getting more attention.  Long surveys designed for computer administration are a serious problem and the industry ought to be running as fast as it can to solve those two problems now.  But I don’t get a sense of urgency.

·       Finally there is behavioral economics, neuroscience, neuromarketing, behavioral science – whatever name you put to it.  MR has been struggling to figure out what to do with it beyond wiring people up in qual settings or grabbing images of faces and making inferences that some might say go beyond the current science.  There is a lot of experimenting going on, which is a good thing, but I think we are still in the learning phase.

The conference closed with one of those personally inspiring talks from an author of what we used to call “self-help” books (maybe we still do) that have become standard fare at MR conferences these days.  Nigel Marsh has written Fat, Forty, and Fired and Overworked and Underlaid.  He was typically personable and funny.  As always happens to me in these situations I was nearly overcome with envy of his command of the stage.  But I never feel especially inspired to go out and take command of my life like I am supposed to do.  Maybe it’s an age thing.

 

 

If You Need to Know What’s Legal, You’re Already on the Losing Path

This is the second and final post from Michael Link on location.

Privacy and ethics concerns in research are not new, but they have taken on considerably higher visibility in our 24/7 news world as researchers test the bounds of new measurement approaches. At a recent symposium on Leveraging Location hosted by Nielsen, a panel of legal experts provided some thoughts in this area on issues researchers are encountering today. Their insights went well beyond location data, hitting on aspects that involve many of us working with data from the public. (Obligatory warning: I’m not a lawyer; these are simply my observations and interpretation of the discussion). Three broad lessons caught my attention:

First and foremost, start with the respondent/consumer, understanding and acting in accordance with their expectations. How data are collected and the insights generated should be readily apparent to the “average person.” If your starting point is “the law” or “what is legal”, you’ve already put yourself in a hole. Laws and regulations provide a base -- a bare minimum, what the public demands is often much more. As researchers we should operate within the reasonable expectations of the majority of the public, yet not necessarily feel constrained by folks on the extremes.

Second, to lead in innovation you cannot be afraid to have your name in the paper and receive negative comments. In essence, as one panelist put it “get comfortable not being comfortable.” Pushing the envelope involves a degree of calculated and real risk. If your organization likes to keep a low profile and acts with alarm at the first half-dozen negative emails received, then you might want to take “innovation leader” off of your long-term business strategy. Note that Rule #1 above is still in effect, so setting expectations accordingly and having a good grasp of the potential risks is imperative.

Third, time is an important and often under-appreciated dimension of attitudes towards privacy and ethics. These are not static features of our society, but rather evolving concepts. What may have been unthinkable a few years ago (using a smartphone as a “virtual wallet”) now seems commonplace. Likewise, certain aspects of data collection in this digital/organic data era that seem unreasonable to the public today (and hence, would be good to avoid) may become more readily accepted with time and incremental exposure. The trick to innovating, of course, is knowing when the time may be right. To this there is no clear right answer, but the public (and the press) will let you know if you have chosen unwisely.