The Survey Geek

Truth in texting

I admit that I've never been much of a fan of SMS surveys. Too many limitations to do anything meaningful. Once Nathan Eagle started showing up at conferences and talking about his experiences with SMS in developing countries, especially in Africa, I could see a niche. That is, if you can call accessing two billion people a niche.

Last week someone sent me this little news item about a study by two academics, Fred Conrad and Michael Schober (I've collaborated with these guys on some unrelated work), that suggests one can collect some pretty good quality data via text. While you can't collect a lot they found that respondents were generally honest (no sign of social desirability bias on sensitive questions), showed little evidence of satisficing and exhibited few signs of heaping, that is, giving round numbers (typically numbers divisible by 5) when asked to report specific counts such as the number of movies they saw in the last month.

Given the kind of research that dominates MR I still believe that mobile ultimately depends on the continued growth in smartphone use. But whether via smartphone or good old-fashioned feature phone, feedback by text message may not be dead yet.

Is mobile finally ready for takeoff?

Not that I claim to have read them all, but the one thing that most of the year end MR prognosticators seem to agree on is that 2012 will be the year in which mobile finally takes off. But will that takeoff be more reminiscent of the Wright brothers or the Saturn V?

Larry Gold shows us some interesting numbers on this topic in the January issue of Inside Research. For a number of years now Larry has been maintaining an index of online spend using a group of 25 Honomichl Top 50 companies. (Here I note that my company is one of them.) These numbers are probably the best we have for charting the growth of online over about the last 15 years. On the last round Larry asked participating companies to report revenues for mobile data collection. Looking at a three year trend he estimates for US only $9.4M in 2010, $20.1M in 2011 and a forecast of $32.9M for 2012. That's an increase of 114% in 2011 and 64% for 2012. Impressive growth for sure! And as he points out, he misses revenues from smaller companies who are mostly focused on mobile, although it's not clear how much those smaller companies are likely to increase the overall number.

A couple of comments. The first is that in terms of spend mobile is now about where online was in the late 1990s (without adjusting for inflation) when revenues were doubling and tripling every year. It took until 2004 to top $1B and it's now forecast at $2.5B for 2012. So despite its recent growth, mobile is still in its infancy. The second is that mobile may face stronger headwinds than online. Rightly or wrongly, a lot of the conversion to online from other methods was pretty simple: just administer the same questionnaire using a different sample source. Online also came with a killer app in concept and product testing. Moving to mobile is a good deal more challenging because most of those questionnaires are going to need massive reworking before they are mobile ready. There are some interesting things being tested in terms of breaking up questionnaires into smaller chunks but that's still mostly in the testing stages. And mobile's likely killer app, "in-the-moment research," may not be enough to make it as dominant as online has become.

I don't question that mobile is poised for significant growth. But how soon, how fast and how big it will get are still open questions.

MROC and mobile revenues still unimpressive

I have long advocated that the real test for any new market research method is how well it resonates with clients. Or, in somewhat more crass terms, how much clients are spending on it.

Inside Research maintains a number of indices that provide estimates of US revenues by method and the August issue reports on two of the hottest new methods: MROCs and mobile.

IR estimates 2011 revenues for MROCs at $152 million. If we assume total revenues for US MR to be around $8 billion (some argue it's much higher) then MROCs are accounting for just about 2% of the total. The companies making up the index are pretty much a who's who of providers in the MR space. Granted, there is some DIY and other uncounted revenue from other providers but that's probably true of the $8 billion base as well. You also could argue that as a percent of total qualitative it's much more impressive and I would agree with you. Where I would disagree is when you try to argue that it's a significant threat to online quant. Not any time soon.

 

Mobile revenues are lower still. While mobile revenues are expected to almost double in 2011 over 2010 they are still only projected to be just over $16 million. The caveat here is that these numbers are derived from reports for the overall online index and a good deal of what's happening in mobile is happening in small, focused agencies whose revenues will not be included in the $16M. So go ahead and double it.Still, the best you can say at this stage is dramatic growth from a very small base.

MROCs and mobile

Earlier in the week someone sent me what I take to be a marketing piece from Communispace called "Connecting with Connected Consumers." Its purpose seems to be to describe how mobile might be used in the MROC context to add a new dimension of insight. Two things struck me about the piece.

First, they report that when it comes to mobile advertising their MROC members report ". . . a strong appetite for contact from brands, provided that contact is both solicited and relevant, and that they are able to retain some control over the cadence and nature of the interaction." It occurred to me as a read the words a second time that we probably could say something very similar about what it takes to conduct successful mobile research or, for that matter, any kind of research. It implies subtlety, creativity and treating people as individuals rather than a commodity. Well said, but I'm not sure brands or researchers have yet figured out how to do just that.

Second, in the requisite in-the-moment section of the mobile pitch they talk about "deputizing" their MROC members "to become participant observers—not just of the lives of others, but of their own lives." The words that then immediately popped into my head were "mystery shop." I confess that those are just words to me, that I've never been within spitting distance of a mystery shop project and what little I know has never impressed me.  However, in the context of an MROC it somehow seems more interesting. I'm not a mobile evangelist who believes that mobile will soon become a dominant methodology, but I have always felt that there are some really cool niche applications where mobile can add a lot of value. The MROC context seems to provide just that kind of opportunity.

Mobile apps or HTML5?

One of the inside baseball debates about mobile is whether it will be driven by apps or web browsing and HTML5. At first blush it would seem that the cost of developing, certifying, and maintaining apps across mobile platforms would give a clear edge to HTML5 with its enhanced offline browsing and (forgive me) flash-like functionality. But of course, the pace of full implementation of HTML5 and the final feature set (even the full specification is two to three years away) could change all that. This article suggests some likely disappointments with HTML5, some of which may prove problematic for the more complex mobile apps currently being talked about. We probably should not write off apps just yet.