The Survey Geek

Today is Privacy Day!

Almost 20 years ago some colleagues and I edited a book with the inviting title, Computer Assisted Information Collection. Experts on a wide variety of computer-assisted methods contributed the chapters and I was tasked with writing the last chapter, a look into the future of technology and survey research. At the risk of tooting my own horn, I did a decent job of imagining the data rich world that we still believe is our future and worried what it would mean for surveys. My colleagues, all good friends, argued that while all those personal data might exist out there in cyberspace, public privacy concerns and supporting legislation would restrict our access to them.

Over the last five years I have had roughly that same argument on and off with numerous colleagues in MR, mostly taking the side of my friends back in the 1990s. I think it is now clear to all of us that Facebook did not redefine privacy, and ongoing public concerns about the use of personal data collected online and evolving regulatory frameworks will have a major impact on what we can and cannot do in market, opinion, and social research.

I am reminded of this because today is Privacy Day, at least in Europe. Its purpose is to alert consumers to the risks they face and to educate them about how to control the disclosure and use of their personal data. I think this is a good thing for us as researchers and here is why.

Cooperation is the elephant in the room every time we talk about what might be possible as technology becomes increasingly intertwined with the activities of our daily lives. Research requests are one-on-one conversations with potential participants and a key element of those conversations must include assurances about how their personal data will be used and protected. That conversation is easier when both parties share a common understanding about the risks and the steps that must be taken to mitigate them. The more the public understands about responsible data protection practices, the better off we all are.

Of course, this also depends on researchers understanding and meeting their responsibilities, both in terms of longstanding ethical principles and current legislative frameworks. In this researchers also require some education.

Today ESOMAR has done its part by releasing a Data Protection Checklist that sets out the key principles that form the foundation of data protection laws worldwide. It is a very practical document developed by a small group of experts from around the world. It is a ‘must read’ for all of us. Doing the right thing is more important now than ever before.

 

 

Aligning practice and principle in the NewMR

Regular readers of this blog (assuming they exist) may have noticed that since retiring almost three years ago posts here are few and far between. That does not mean lack of interest. Much to my surprise I have discovered that my fascination with market, opinion, and social research remains strong, even when I no longer face the day-to-day challenges of actually doing research that clients find useful and actionable. Retirement has allowed me to become more “methodology agnostic,” to use a phrase coined by Joan Lewis, as I no longer have to pick winners and losers, decide which methodologies are going to be adopted on a broad scale and which are niches or flashes in the pan. In short, I have become more of a spectator at the circus that is contemporary MR than a participant in it.

Of course, the challenges that we face go beyond methodology. The public seems less willing than ever before to cooperate with research requests, partly because they don't see the value of what we do but also because they simply don’t trust us to do the right thing with information they share. I like to think that this is not a problem we created but a spillover from less scrupulous behavior on the part of direct marketers and data brokers. Regardless, complying with evolving regulatory requirements and engaging with the public in ways that respect their privacy and protect their confidentiality is more important than ever before.

My buddy Lenny Murphy over at GreenBook worries about whether our businesses will survive in the face of new methods, technologies, and players. I worry less about research as a business and more about research as a profession.

I spend the bulk of my time these days on two major assignments, one as Executive Director of the Marketing Research Institute International, the non-profit that develops content for the online Principles courses offered by the University of Georgia, and the other as a consultant to the ESOMAR Professional Standards Committee. These organizations share the goal of promoting the longstanding and still relevant values of the research profession in the context of rapidly evolving and emerging research methods. This is interesting and worthwhile work.

In the abstract, what we do as researchers and our responsibilities to both clients and people—research participants and the public at large—have not changed. What is changing is how we do it. Put another way, there is a tension between principles and practice. The future of our profession rests on how well we reconcile the two.

That's what I think about these days. I hope you do, too.

Game-changers and same-changers

I am a political junkie. I wish I could write “recovering political junkie” but I know I’m not there yet. One current challenge is to not buy Double Down: Game Change 2012, the new book on the 2012 US presidential election by the same guys who gave us Game Change: Obama and the Clintons, McCain and Palin, and the Race of a Lifetime on the 2008 election. I know in my heart that books like this are no different than websites like Politico that I visit several times a day and all of those TV pundits I listen to (despite Nate Silver’s sage advice to the contrary). They all make their money by taking everyday events of political life and make each one seem like the beginning of a turning point in history. Nonetheless, I struggle to stay away.

Fortunately, I stumbled on this review of Double Down by Ezra Klein, himself an emergent TV pundit, and it strengthened my resolve to not buy Double Down. The review actually is of two books: Double Down and The Gamble.  The latter is a different retelling of the 2012 election based on continuous polling from start to finish. The Gamble thesis is simple. While the political pundits pronounced 68 events over the election cycle to be game-changers, almost none really were. The underlying dynamics never changed, and the polls bear that out.

There is, of course, a parallel in contemporary MR. By my count we are about five years into, what shall we call it? MRX? It might be fun to go back and count the game-changers and see how many of them have really changed in any fundamental way how clients spend their money on research.  More likely, we have had a lot of “same-changers.”

MR has its pundits, too. Takes one to know one.

Is research on research the real deal?

David Carr had a piece in last Sunday's New York Times about the difficulty of distinguishing journalism from activism. His first sentence sums up the issue pretty succinctly, "In a refracted media world where information comes from everywhere, the line between two 'isms' — journalism and activism — is becoming difficult to discern." His case in point is Glenn Greenwald, the Guardian reporter who has been breaking all the Snowden stories. Greenwald seems to be a guy with a very strong suspicion of government, and it shows in what he covers and how he writes about it. Cable news provides still better examples, where people who describe themselves as journalists routinely put a political agenda ahead of the objectivity that some of us expect (hope) from "the news."

We have a similar problem in market research when it comes to distinguishing between good methodological research and what we like to call research on research (RoR). Good methodological research is based in honest and objective scientific inquiry. Hypotheses are formed, the relevant literature reviewed, experiments designed and executed, data analyzed, hypotheses accepted or rejected, conclusions reached, and potential weaknesses in the research fully disclosed.  The best of these studies end up in peer-reviewed journals where they help us to build and refine research methods, brick by brick.

Much of RoR, on the other hand, has become something much different. It often features a point of view rather than a hypothesis, and the exploration of the data is a search for proof points rather an objective analysis aimed at uncovering what the data can tell us. The end product typically is a white paper, designed to sell rather than inform. We might attribute some of the poor quality of RoR to a lack of training and skill, but I expect most of it comes back to the simple fact that MR is a business. Academics achieve success by doing good solid research that earns the respect of their peers. MR companies succeed by selling more of their stuff.

All of which is not to say that there is not some good RoR being done, studies that are based in the fundamentals of objective scientific inquiry. It's just that it's getting harder and harder to tell the difference. And given the methodological disruption that has come to characterize our industry over the last decade, that's a real problem for all of us.

Sir Martin on crowdsourcing of ads

The March issue of The Harvard Business Review has in interview with Martin Sorrell.  At one point the interviewer asks whether crowdsourced ads and algorithms are the future model of advertising.  Sir Martin’s response is interesting:

You are tapping into the knowledge and the information of people all over the world.  That’s fantastic. And the power of the web is that it opens up and plumbs people’s minds and mines all their knowledge.  But somebody has to assess it, and you can’t do that with an algorithm. There’s a judgment call here.  That’s the problem I see: Ultimately, somebody still has to decide what is the best piece of knowledge or the big idea that you got from the crowd. If you don’t get that right, it fails.