The Survey Geek

Is Insights a Profession?

Yesterday, like 100 million or so other Americans, I tuned into the Super Bowl. I was not there to see whether the Flying Elvis would beat a team that was only there because of one of the most glaring officiating errors in the history of American football. Rather, I tuned in to hear Romostradamus, the twitter-endowed name for Tony Romo, former quarterback turned color commentator who has become a phenomenon based on his ability to analyze a football game, explain strategy to the audience, and predict what play will be run with uncanny accuracy.

Writing in the New Yorker, Zach Helfand describes how Romo learned to do it: “In the course of his career, he watched hundreds of plays from the bench, lived through thousands more on the field, and then relived them many times over in the film room.” Frank Bruni writing in the Times echoes this theme of relentless study and preparation even to this day. In short, Romo approaches his job as a professional.

The Oxford Dictionary defines a profession as “a paid occupation, especially one that involves prolonged training and a formal qualification.” How many of us who fancy ourselves as “insight professionals” meet that test?

As I have said in other contexts, market researchers seem to take an almost perverse pride in describing how they “fell into” MR with little or no relevant education or training.  Over the last decade in particular we have celebrated the inclusion of specialists in other fields—computer science, anthropology, social psychology and journalism, to name a few. As one well known practitioner (whose name I withhold to protect his innocence) once observed, “We have allowed this industry to be taken over by venture capitalists and technology geeks.”

All this would be fine were there a concerted effort to teach these new entrants the foundational principles that underpin good research and actionable insights. What we have some learning by doing, some basic orientation in how a single company does what it does.

In December I ended a four-year stint as Executive Director of the Market Research Institute International (MRII), a non-profit that develops online courses in market research in partnership with the University of Georgia. When I first joined I was astonished by the widespread disinterest in education and training by individual practitioners and their employers, on both the client and supplier sides. And it’s not just me. This unfortunate reality is documented in a recent NewMR survey in which 39% of respondents reported receiving less than six hours of training per year. This in an industry that fancies itself as undergoing rapid and dramatic change, but it’s doing precious little to address it in a meaningful way.

Tony Romo can do what he does because of relentless and ongoing study that continues to this day. That might be more than those of us who aspire to be insight professionals can or want to commit to, but continuing to do what we have been doing is not going to produce a future where MR can play the essential role it seeks to play.

Online Sampling Again

Last week two posts on the GreenBook Blog, one by Scott Weinberg and a response by Ron Sellers, bemoaned the quality of online research and especially its sampling. And who can blame them? All of us, including me, have been known to go a little Howard Beale on this issue from time to time. We all know the familiar villains—evil suppliers, dumb buyers, margin-obsessed managers, tight-fisted clients, and so on. I was reminded of a quote from an ancient text (circa 1958) that a friend sent me a few months back:

Samples are like medicine. They can be harmful when they are taken carelessly or without adequate knowledge of their effects. We may use their results with confidence if the applications are made with due restraint. It is foolish to avoid or discard them because someone else has misused them and suffered the predictable consequences of his folly. Every good sample should have a proper label with instructions about its use.

Every trained researcher has an idealized notion of what constitutes a good quality sample. Every experienced researcher understands that the real world imposes constraints, that any one who says you can have it all—fast, cheap, and high quality—is selling snake oil. So we make tradeoffs and that’s ok, as long as we have “adequate knowledge of their effects.”

It helps to be an informed buyer, and ESOMAR has for about ten years now offered some version of their 28 Questions to Help Buyers of Online Samples to help. It’s an excellent resource, often overlooked. More recently, ESOMAR has teamed up with the Global Research Business Network to develop the soon-to-be-released ESOMAR/GRBN Guideline on Online Sample Quality (here I disclose that I was part of the project team). In the meantime, there is an early draft here. Its most prominent feature, IMO, is its insistence on transparency so that sample buyers are at least informed about exactly what they are getting and in as much detail as they can stand.

Granted, that’s not the same as “a proper label with instructions about its use.” That still is a job left to the researcher. If you can’t or won’t do that, well, shame on you. Getting depressed about it is not an option. Or, as another of the ancients has said, “You're either part of the problem or part of the solution.”

Today is Privacy Day!

Almost 20 years ago some colleagues and I edited a book with the inviting title, Computer Assisted Information Collection. Experts on a wide variety of computer-assisted methods contributed the chapters and I was tasked with writing the last chapter, a look into the future of technology and survey research. At the risk of tooting my own horn, I did a decent job of imagining the data rich world that we still believe is our future and worried what it would mean for surveys. My colleagues, all good friends, argued that while all those personal data might exist out there in cyberspace, public privacy concerns and supporting legislation would restrict our access to them.

Over the last five years I have had roughly that same argument on and off with numerous colleagues in MR, mostly taking the side of my friends back in the 1990s. I think it is now clear to all of us that Facebook did not redefine privacy, and ongoing public concerns about the use of personal data collected online and evolving regulatory frameworks will have a major impact on what we can and cannot do in market, opinion, and social research.

I am reminded of this because today is Privacy Day, at least in Europe. Its purpose is to alert consumers to the risks they face and to educate them about how to control the disclosure and use of their personal data. I think this is a good thing for us as researchers and here is why.

Cooperation is the elephant in the room every time we talk about what might be possible as technology becomes increasingly intertwined with the activities of our daily lives. Research requests are one-on-one conversations with potential participants and a key element of those conversations must include assurances about how their personal data will be used and protected. That conversation is easier when both parties share a common understanding about the risks and the steps that must be taken to mitigate them. The more the public understands about responsible data protection practices, the better off we all are.

Of course, this also depends on researchers understanding and meeting their responsibilities, both in terms of longstanding ethical principles and current legislative frameworks. In this researchers also require some education.

Today ESOMAR has done its part by releasing a Data Protection Checklist that sets out the key principles that form the foundation of data protection laws worldwide. It is a very practical document developed by a small group of experts from around the world. It is a ‘must read’ for all of us. Doing the right thing is more important now than ever before.

 

 

Aligning practice and principle in the NewMR

Regular readers of this blog (assuming they exist) may have noticed that since retiring almost three years ago posts here are few and far between. That does not mean lack of interest. Much to my surprise I have discovered that my fascination with market, opinion, and social research remains strong, even when I no longer face the day-to-day challenges of actually doing research that clients find useful and actionable. Retirement has allowed me to become more “methodology agnostic,” to use a phrase coined by Joan Lewis, as I no longer have to pick winners and losers, decide which methodologies are going to be adopted on a broad scale and which are niches or flashes in the pan. In short, I have become more of a spectator at the circus that is contemporary MR than a participant in it.

Of course, the challenges that we face go beyond methodology. The public seems less willing than ever before to cooperate with research requests, partly because they don't see the value of what we do but also because they simply don’t trust us to do the right thing with information they share. I like to think that this is not a problem we created but a spillover from less scrupulous behavior on the part of direct marketers and data brokers. Regardless, complying with evolving regulatory requirements and engaging with the public in ways that respect their privacy and protect their confidentiality is more important than ever before.

My buddy Lenny Murphy over at GreenBook worries about whether our businesses will survive in the face of new methods, technologies, and players. I worry less about research as a business and more about research as a profession.

I spend the bulk of my time these days on two major assignments, one as Executive Director of the Marketing Research Institute International, the non-profit that develops content for the online Principles courses offered by the University of Georgia, and the other as a consultant to the ESOMAR Professional Standards Committee. These organizations share the goal of promoting the longstanding and still relevant values of the research profession in the context of rapidly evolving and emerging research methods. This is interesting and worthwhile work.

In the abstract, what we do as researchers and our responsibilities to both clients and people—research participants and the public at large—have not changed. What is changing is how we do it. Put another way, there is a tension between principles and practice. The future of our profession rests on how well we reconcile the two.

That's what I think about these days. I hope you do, too.

Game-changers and same-changers

I am a political junkie. I wish I could write “recovering political junkie” but I know I’m not there yet. One current challenge is to not buy Double Down: Game Change 2012, the new book on the 2012 US presidential election by the same guys who gave us Game Change: Obama and the Clintons, McCain and Palin, and the Race of a Lifetime on the 2008 election. I know in my heart that books like this are no different than websites like Politico that I visit several times a day and all of those TV pundits I listen to (despite Nate Silver’s sage advice to the contrary). They all make their money by taking everyday events of political life and make each one seem like the beginning of a turning point in history. Nonetheless, I struggle to stay away.

Fortunately, I stumbled on this review of Double Down by Ezra Klein, himself an emergent TV pundit, and it strengthened my resolve to not buy Double Down. The review actually is of two books: Double Down and The Gamble.  The latter is a different retelling of the 2012 election based on continuous polling from start to finish. The Gamble thesis is simple. While the political pundits pronounced 68 events over the election cycle to be game-changers, almost none really were. The underlying dynamics never changed, and the polls bear that out.

There is, of course, a parallel in contemporary MR. By my count we are about five years into, what shall we call it? MRX? It might be fun to go back and count the game-changers and see how many of them have really changed in any fundamental way how clients spend their money on research.  More likely, we have had a lot of “same-changers.”

MR has its pundits, too. Takes one to know one.