A little theory might help
A bad survey or no survey at all?

Whither mobile?

I spent two days last week in Chicago at Market Research in the Mobile World (MRMW). I now have been to all four of the North American versions of this event, all the way back to Atlanta in 2011. Last week’s iteration caused me to go back and look at my post about the Atlanta event as a way to gauge how much things may or may not have changed.

One obvious change was the sorting out of marketing applications from genuine research. In Atlanta we heard from too many companies whose business was collecting personal data for direct marketing, and not always transparently. As far as I could tell, the presenters in Chicago were primarily focused on research. A second change was the emphasis on case studies. Atlanta was mostly about the potential for mobile—what could be—while Chicago was mostly actual studies completed.

Atlanta also was heavy on hype and there was plenty of that over the two days in Chicago, including a fair amount on the potential of wearbles. But it was hard to get too worked up about any of it given the sobering start.

  • The first presentation was a paen to mobile as “the most important marketing channel, ever” that included the claim that 55% of people really like targeted ads, a figure undermined by virtually every credible survey on online privacy.
  • That was followed by an update on last year’s eye-popping announcement by General Mills that they hoped to be doing 80% of their research via mobile in 2014. No exact figure was given but it was clear that it turned out to be harder than they thought. They have dismantled their mobile team,  but they keep on plugging.
  • The segment was closed out with a panel of industry heavyweights on the topic, “Investments in (mobile) MR—where are they going and why?” The answer: they are not. The money is all going to big data analytics.

That was followed by about 30 presentations with an overwhelming emphasis on pure mobile applications: in-the-moment, geolocation, and mobile ethnography (as opposed to the unintended mobiles that comprise the bulk of mobile MR right now). Some were genuinely interesting and others seemed like sales pitches. The highlights and lowlights in no particular order:

  • There were a couple of nice papers on the topic of integrating mobile with other platforms as a way to understand the context in which people view different ads.
  • I heard no discussion of the sampling challenges beyond a preliminary report from TNS aimed at allaying concerns about bias in online surveys that include respondents using mobile devices. One might have gotten the impression from most presentations that there is 100% smartphone penetration and people are willing to use them to do pretty much anything researchers ask them to do.
  • The potential power of mobile ethnography was nicely demonstrated by several presentations.
  • There was a somewhat bizarre though seemingly heartfelt epilogue to one presentation pleading with us not to be swayed by the media into giving up on Google Glass.
  • There was an equally bizarre presentation on mobile’s ability to reduce social desirability bias and satisficing that opened with the presenter acknowledging that all he knew about the phenomena was what he learned from their Wikipedia entries. It seemed to be just another study on recall.
  • There continues to be some sloppiness around proper privacy protections, especially in mobile ethnography. There was a panel on the topic (I was on it) but less than half of the attendees were in the room.
  • A number of presentations took up the theme of respondent-driven design that emphasized better interfaces, choice of channel, and better-designed if not shorter surveys. On this latter topic, one presenter showed that a well-designed 15-minute survey is possible with no discernible data quality drop off.  No doubt music to the ears of clients (of which there were very few) and even researchers who cling to long surveys like an NRA member clings to his assault rifle.

My bottom line is that things have changed substantially since Atlanta, but nowhere near as much as mobile evangelists predicted. Mobile has become mostly an extension of online, and while there is no shortage of startups offering in-the-moment and other pure mobile solutions there also are no clear signs yet of research buyers flocking to them. Mobile has changed substantially how we interact with one another and the world around us. It must be a fundamental concern in every research design, but it has yet to truly transform MR in any meaningful way.