Back to the Future

I have been in a sort of accidental exile from social media since just before I left for Amsterdam. It was a busy week with the Congress, ISO Technical Committee meetings and WAPOR all in the same week with no time left for blogging or even monitoring Twitter. And then an all-too-brief family vacation enjoying some lovely Belgian cities (Bruges, Antwerp, Ghent) with the help of the best beers in the world followed by the standard punishment our day jobs visit upon us when we are out of the office "too long."

All of that behind me I have begun to catch up with what's been going on in MR's social media echo chamber and well, it's like they say about American TV soap operas: it's no big deal if you miss a bunch of episodes because when you finally tune in again the story line hasn't advanced all that much. One thing that has struck me is the plethora of reports, surveys, presentations and general prognostication about the future of our industry. Maybe it's because it's the conference season. At the highest level they all say pretty much the same thing and only disagree about how fast all of this will happen. But when you look closely at most of the surveys in particular and evaluate the methodology from a survey research perspective you can't help but be amazed that these are designed, executed and reported on by research professionals. Convenience samples and overly-hyped findings are the norm. There is story telling for sure, but it sometimes borders on fiction. As I said, at some very high level they all come to more or less the same conclusions but if I want to use any one of them to decide where to place my bets and how I need to evolve my organization to continue to be competitive in a changed industry I'm not going to find the confidence I need in any one of these studies.

In that sense this is a sort of parable for our industry and its future. We've come to recognize that all of our sources, all of our methods have important weaknesses and our story telling needs to take that into account. The smartest commentators among us see that the real future lies in our ability to consider a broad range of relevant data, understand their strengths and weaknesses as related to business problem at hand, and synthesize disparate findings into a compelling story that helps our clients make very specific business decisions. I'd love to see someone do that with all of the research being presented about the future of our industry. I wonder if the story would be any different.

Comments

[Reg]

I think the heart of the problem is getting a good sample. A professional or trade association can survey its members and should have enough information about those members to characterize the results as representative of their particular segment of the industry. But for others it's tricky. Interpreting the results starts with understanding the bias, either due to coverage error or nonresponse. In the particular case of GRIT, I'd like to see more effort to understand the bias so results can be properly characterized.

[Leonard Murphy]

Fair points as always Reg and Kathryn; how would you like to join the "coalition of the willing" for the next round of GRIT to make sure your concerns are addressed? You can find me in the echo chamber... :)

[Kathryn Korostoff]

So many great points here! My facvoratites:
"There is story telling for sure, but it sometimes borders on fiction. "
"But when you look closely at most of the surveys in particular and evaluate the methodology from a survey research perspective you can't help but be amazed that these are designed, executed and reported on by research professionals. "
and the brutally honest and oh-so-true "MR's social media echo chamber"

I for one have cut back in recent months. Too much content that is redundant, too many "experts" who make obvious errors, and just too many darn groups/sites. Still, I have met some amazing researchers via SM, some of whom have become clients, partners or suppliers. But I am frequently reminded that SM is an unfiltered firehose--it comes out fast and you never know if the content will be good. Note: I found out about this article via Twitter ;-)