The Survey Geek

Waiting for mobile

There is a nice little piece in the June issue of Research that tries to put interviewing on mobile phones in perspective. I'm not talking about something as old fashioned as calling people, but rather administering surveys on these little buggers by SMS, by Web browser, or by a special app you download that then can run survey questionnaires. There is a lot of hype around mobile right now, most of it by companies who have products they are promoting as the next big thing. But Diana Mitkov from Harris Interactive has done a nice job of running through the pros and cons, and it turns out that mobile currently is mostly cons.

There is a lot of good stuff in what is a very short article and I recommend it to you if you are interested in the topic. But like most other observers she comes down on what I think are the three key points. First, surveys by SMS are just too limited and the future of mobile interviewing, if there is one, is on the Web. Second, recognize that attention spans are short, screens are small, and download speeds are slow, all of which mean only short, simple surveys make sense. Finally, ignore all the hype about Web-enabled mobile phone penetration and pay attention to how few people who have the capability actually use it. In the UK, for example, while 38 percent of the population could browse the Web from their mobile phones just 16 percent actually do. People either find it too expensive, too slow, or just don't see the point. The most recent numbers that I have seen for the US are not terribly different.

The title of the piece says it all, "Still on hold."

Who really is worrying about online data quality?

The indomitable Simon Chadwick has a a little piece in the June issue of Research World in which he reports on the results of Cambiar's annual survey of the research industry--clients and suppliers. (You can get a topline of  the report from the Peanut Labs site here.)  One of the ongoing themes of this survey since Simon launched it way back in 2005 has been concern about the quality of online sample, and that concern is front and center again in 2009.  The concern has changed some over the years from one of worry about recruitment sources and procedures to those of respondent engagement, representivity, and questionnaire design.  No surprise there.  But what is something of a surprise to me at least is his finding that "Clients . . . appear to be rather less concerned about data and sample quality than research companies are."   Ummm.  It seems to me that one underlying premise of the industry frenzy on this issue has been that clients are worried about online quality and we'd better get some good answers and get them fast.  But Simon's data suggests that may  not be the case.  Further, he points out that despite concern about online quality clients use of online and even expansion into new online methods continues unabated.  But what does it mean?  Do clients not require the level of precision and certainty that we think they do when they make business decisions?  Or are the issues so arcane that they just don't understand them or care about them?

We say that MR is important as a means to help clients understand their markets and their customers.  Findings like this make we wonder how well we as researchers  understand ours.

Day 2 PM: CASRO International Conference

Back from lunch. CASRO conferences always seem to have great lunches!

There is just one session this afternoon that the co-chair promises will get us out of future pie-in-the-sky stuff and down to the basic blocking and tackling we need to know to do well in today's world.

The first presenter is Kate Southwood from Kantar Operations. Her presentation is titled, "Conducting Multi-National Research Operations on an International Scale." She wants to talk to the fundamental issues that have been with us a long time and they start with culture. Then there are the logistical challenges around time zones, holidays and seasonality and, of courses, translation and localization issues. Client structure—headquarters office versus in country local affiliates—can seriously complicate operations and introduce political issues the research company needs to learn to stay clear of.

But the world is changing and now there is a whole new set of challenges. Understanding and leveraging new technologies is one of them. Privacy regulation is another. In addition, clients want a single point of contact and a cohesive network to do work or even may want to dictate which vendors you use and manage, some of whom may have back-channel communications with the client that bring those politics into play again. Increasingly companies are turning to "offshore hubs" to take advantage of lower cost countries (like India) and managing them can be difficult.

Some keys:

1. Know your client, their structure, expectations and experience with global research.
2. Use experienced project managers with good organizational skills and knowledge of the pitfalls of global research.
3. Understand and manage the risks.
4. Communicate! Communicate! Communicate! Never assume anything.
5. Use common platforms, where possible.
6. Be flexible and expect the unexpected. One size does not always fit all.
7. Be bold. Collaborate with in-country experts and don't always bow to "what the client wants." This is often more about the client service people than the client.

Good solid presentation.

Next up is Tara Hutton from a smaller company—Consumer Electronics Association-- to talk about the pitfalls of international research. She has started by stressing how critical global research is to consumer electronics companies. Now she's moving to the pitfalls and, to be honest, not adding a whole lot to what the previous speaker presented.

Finally, we have Holly Jarrell from GfK and Allyson Clarke from MS&L Worldwide. Their topic is, "Taking Data to New Places: How Syndicated Insights can Shape Global Communications Strategy." They are going to talk about syndicated rather than custom. Their argument is that you have to do all of the same things you have to do with custom, but this is harder because you are serving multiple masters and analysis is more difficult. But some of what she is talking about—like difficulty of changes—is really about tracking rather than syndicated per se. And I must say, that the more I listen the more I question the basic premise that syndicated is harder. Seems to me, once you get the vendors lined up and the relationships in place it's actually a little easier from an operational perspective.

Now, unfortunately, I have a plane to catch. This has been a pretty good conference as these things go. Some good nuts and bolts stuff by people with the experience to speak to the issues. This is about the fourth one of these I've been to and this one was probably the best. I'm glad I came. But at the end of the day the real secret to doing good global work well is in the experience of doing lots of it. It's like anything else in that regard. The more you do the better you get. There are no magic bullets.

Day 2 AM: CASRO International Conference

Back for Day 2. Mike Cooke is kicking us off again. He is arguing that we have come to rely too much on quant and that the whole social network thing is a reinvention of the older qual methods. He sees quant as "the industrialization of research" and believes that in the process we have lost something in terms of our ability to elicit truth. I suppose that at one level this about precision and the level that we need to make decisions. What level of certainty do clients need? Tough question.

The first speaker is Dan Shapero from LinkedIn who is going to talk about social networks. He has started with the stats which are very impressive. They are up to 40 million members (half international) and getting another two million every month. All in all, impressive reach. Now he is into telling us what it does and how cool it is. Not very interesting until he starts describing a joint venture that they are planning with Twitter that will give LinkedIn users some sort of access to "relevant" Twitter users. The ability to look only at certain kinds of LinkedIn users and thereby get to Twitter subgroups could be very interesting. Finally we are getting to LinkedIn as a B2B sample source. His starting point is that this is much better sample than what we are likely to get elsewhere. People are who they say they are. It's tough to over represent yourself in this kind of system. They did some verification work and were able to validate 98 percent of their sample. My take on all of this is that they have the basis for a terrific product here but they are still figuring out how to leverage it effectively. Pete Cape, Global Knowledge Director at SSI, once observed that "we have allowed MR to be taken over by venture capitalists and IT geeks." That certainly applies here, but they have the potential to provide something very useful although the sense is that they are having trouble figuring out how to do so.

Now we are going to hear from Larry Ponemon, who does a lot of government relations type stuff for CASRO, on the difficult-to-make-interesting topic of global privacy. He is going to report on a study in 19 countries looking at perceptions of privacy by consumers and business people. The top line finding is that privacy is most important in the EU and Canada and less important in the BRICs. The US falls in the middle. He has a lot of interesting data supporting this top line.

This presentation causes me to reflect back on a conversation at breakfast this morning in which Mike Cooke was saying that the EU is on the verge of outlawing targeted marketing and that Canada already has outlawed "digital fingerprinting" as a way to identify duplicate respondents on Web surveys. Point being that the US has not yet been hit by the kind of privacy legislation that is increasingly common in Europe and in Canada, but that may not last. Worries about privacy legislation have sort of ebbed in the US over the last couple of years but I expect the issue will surface again before too long.

Now we will hear from a couple of guys from Greenfield Online (Hugh Davis and Keith Smart). Their topic is emerging technologies for global research. They are starting out by talking about their global reach and their underlying technology. They are into the whole "world is flat" thing, but is it really? Differences in culture, in regulation, in use of technology are just a few things that vary dramatically from country to country although they don't seem to see it that way. But I digress, and they are now into technology trends. They are showing us the Microsoft 2019 video. They are admitting that there are a variety of things that "impact the use of technology" in global research but only at a very high level. There are five things that they see as important in conducting global surveys:

1. Using the cleaning technologies developed for panels in the US on a global scale.
2. Using the right tool and technology platforms.
3. Understand the cultural differences and how they may impact technology use.
4. Recognize how people from across the globe take surveys.
5. Use technology on the back end.

My problem with this presentation is that the presenters make it all seem so easy when, in fact, it is very very difficult. Some people seem to think that we can just project the US panel paradigm to the rest of the world, but even these guys note that in some developing countries we will need to do mobile and in others learn to harvest blogs and social networks, things we are not yet very good at. So all in all this is a technologist's view of global research rather than a researcher's.

Day 1: CASRO International Conference

I am here in Washington, DC, at the CASRO International Conference. Over the next two days I will do my best to keep up with the presenters.

Mike Cooke, one of the conference chairs, is introducing the speakers and making the key point that international research is ultimately about culture. We can focus on process and rationality, but ultimately the ability to generate insights rests on understanding the cultures of the countries in which we do research. He is arguing that historically culture has been more or less imposed by communications media (books, newspapers, radio, and TV) but all that has now changed. Now it's all about user generated content rather than media generated content. Example: Iran where the media has been largely squeezed out and the news we are getting is coming out of various social media outlets like Twitter. That's meant as a transition to the first speaker.

That first speaker is Nick Nyhan, Chief Digital Officer at Kantar. He's talking about the Obama campaign from the point of view of a volunteer. It's interesting in its own way but I'm not sure what it has to do international research and he never really gets to Mike's point.

The next speaker is Ari Popper from BrainJuicer. This is cool stuff. All about using faces to measure emotions. I've posted on this before and I think it's a very interesting line of research. Some of it comes out of behavioral economics, some from fMRI, and some of it is just common sense. The central point is that we all respond instinctively to things. The problem with survey is that we try to translate these instinctive reactions into words and it just doesn't work. This is especially true in international research where culture and language interact in ways that it is difficult for us to understand. So the trick is to find a way to record people's emotions without asking them to translate that into words. Just react instinctively to things. The basic theoretical underpinning comes from Ekman's seven basic universal emotions and the use of faces to represent them. MR is all about the rational when real decision making (including purchase decisions) is the irrational—emotions and instinct. They ran an experiment across 11 countries in which they tested 19 ads in five different categories. The findings across countries correlated very highly, while more traditional measurement had all of the inconsistencies that we often see in cross cultural research. Good and interesting stuff.

Now we are getting into the real meat. First we have Ndirangu wa Maina from Consumer Insights. He is talking about research in Africa. He is recommending a book called Africa Rising by Vijay Mahajan. Africa is much like the BRICs—huge markets but a difficult place in which to do research and market products. He's not pulling any punches, pointing out that Africa is often synonymous with disease, instability, corruption, and violence. But he is quick to remind us that it is a huge market with huge potential—53 countries and 900 million consumers who want a lot of the same things other consumer societies want. Economic growth is strong –about five percent per year—and it's rapidly urbanizing. But there are issues with the technology infrastructure. Mobile phone penetration is only about 15 percent and Internet only five percent. There are other challenges. Seventy percent of markets are informal and unregistered and including them in research is very challenging. Roughly half the population is at subsistence levels, but another 40 percent are more middling and good targets for the usual consumer goods. And, of course, there is the top 10 percent with money to spend. The total value of MR is pretty modest (around 500 million) compared to other parts of the world, but growing quick (at around 15 percent a year). The research is almost all face-to-face, tends to be ad hoc rather than continuous, and quantitative rather than qualitative. This is changing very slowly. But, of course, given the low wage rates face-to-face is very cheap. And cooperation is high. But there also are challenges: shortage of qualified staff and over 1000 languages. Sometimes the corruption gets in the way as does violence. And the weather can also delay survey work. All in all, a nice informative talk.

In the QA someone asked him to name four countries where it would be worthwhile to do research:

• South Africa because it has a well developed economy
• Nigeria because it's a large market
• Kenya because the economy is developing rapidly
• Congo because it's large and has/had a well developed economy, although that is changing and it's just not safe

When the questionnaire pushed about Egypt and Libya he described them as culturally distinct from the rest of Africa and better considered with the Middle East.

Now we move to China and presentation by Linda Liu, an independent marketing consultant (formerly of Pfizer). She has started by over viewing the tremendous changes in the Chinese market over the last 20-30 years. Consumer aspirations have moved from basics like bikes, radios, and sewing machines to electronics, cars and high end consumer/luxury goods. Of course, most of that is in the urban areas. The myth of 1.3 billion consumers really breaks down when you look at where the money and the people are which is mostly on the coast. She argues that the real market is more like 500 million. She also sees an ageing population, health problems like obesity and smoking (western influence), and higher stress levels. And so it's a potentially lucrative market for health care. But some real issues:

• Multiple government agencies setting policy
• Under developed health insurance system
• 80% of hospitals are government owned
• 80% of drugs are dispensed from hospitals
• The drug distribution system is highly fragmented and chaotic
• Public awareness of health issues—especially in urban areas—is rising rapidly

The government is aware of these problems and has an ongoing health care initiative designed to address the key issues.

In terms of MR, China ranks eight globally in terms of spend and increasing fast. Like Africa, cooperation is high and people tend to be very engaged in research. Another excellent presentation.