TRC

A recent discussionon Linkedin pondered whether MR is having its own global warming crisis in the form of an ever dwindling respondent pool. As always, this brought on arguments that response rates need to be improved, quality enforced and of course talk about how much we have slipped as an industry since the good old days. Some blame clients for this (they demand speed and lower cost without concern for quality!) and some blame researchers for not holding clients’ feet to the fire.   It struck me that this is yet another case of researchers not viewing things from a client perspective.

Tags: Market Research Methodologies, Market Research, Market Research Innovation

Over the past year I’ve blogged about the things that I think will drive the future of Market Research and I’m pleased to announce that for our Frontiers of Research annual conference (May 8th, in NYC, view full agenda or register) we have assembled speakers who will drive that conversation forward. The conference will cover the full spectrum of buzz-worthy topics (Behavioral Economics, Neuroscience, Gamification, Predictive Analytics). And the focus, as always, will be on ideas presented in an easy to understand way (no math!). With speakers from four Ivy League schools, and presentations that range from poker to motion picture box office, this should be an informative and enjoyable day.

Leonard Murphy will set the table by calling on his extensive knowledge of the industry to illuminate how academia can and is driving us forward. Anyone who follows his blog knows that he is not only one of the most knowledgeable industry leaders around, but that he has a provocative view of where we are heading.

Tags: Conferences, Behavioral Economics, Neuroscience, New Product Development

Gamification as a means to understand consumer choice is a relatively new idea for research (and controversial in many circles), but it is not new everywhere. For example, one sociologist, Dmitri Williams, has been studying economic behavior using gamfication for four years. His experiments were based on the online fantasy game EverQuest II, which involves thousands of players selling millions of virtual items every month. In essence it is a fantasy economy that works like a real economy.  

Professor Williams theorized that this provided an opportunity to observe the choices players made without fear of the Hawthorne effect (some people give different answers when they know they are being watched).   It also allowed him to set up test and control groups and observe what happens when, to take a simple example, prices go up (if you guessed “people buy less” you win) and to look at gender roles. He saw applications in many fields, not the least of which being testing the impact of various government intervention options before implementing them in the real world.

Tags: Engaging Research, Gamification, Market Research

The Economist reviewed a study by Dr. Neil Brewer about effective police lineups which I think had implications for Market Research. Like researchers, police typically like to encourage witnesses to take their time to ensure they are making the correct choice. This makes logical sense, more time, means more thinking which naturally should lead to better results. Sadly, Dr. Brewer found otherwise.

He had volunteers view short films which detailed mundane scenes of everyday life and a crime (shoplifting, car theft, etc). Later (some minutes later, some a week), they were asked to identify the criminal from a group of 12 pictures of “suspects”. Half were given 3 seconds to evaluate each picture and asked how confident they were of their choice. The other half were given as much time as they wanted. The results showed that the group that had the limited time was correct 67% of the time. The group with more time was only correct 49% of the time.  

Tags: Gamification, Behavioral Economics, Market Research Innovation

For the past few years MR blog posts have been dominated by posts questioning the future of Market Research or talking about just how tough it is to be a researcher in the new millennium. A recent discussion on Linkedin about the threat from DIY is a good example. If you read my blog frequently you know that I see the industry evolving, not going extinct. In any case, at TRC we do a great deal of research about Health Insurance and so I know that as challenging as research is, it is nothing compared to what the health insurance industry is going through.

First off, I'll ignore issues that have been with the industry for decades. More often than not they don't sell to the folks who use their products (most insurance comes through employers) and they often don't sell to the folks who pay the bills (a majority of insurance is sold through independent brokers). While some research clients don't expose us to their internal clients, we are nowhere near as separated from the folks who use our work as health insurance firms are.

Tags: Market Research, Brand

As researchers it is critical that we ensure our data accurately reflect the thinking of the market....in other words, getting to the truth. This is complicated by several factors including limitations of a questionnaire, respondent's lack of attention and the fact that people don't always know what they really want or need. While careful design and methodology can help to minimize these issues (at TRC we believe in using choice questions and shorter surveys) and the use of other data (which can establish the facts), it is impossible to eliminate them.

Technology such as eye tracking, bio metrics and facial recognition software can be applied to neuroscience to help us understand more about what respondents are thinking. The trouble is they are often expensive (sometimes getting the whole truth isn't worth the price) and slow down the research process (sometimes a faster less complete answer is better than a slow one). The limited data available also make it difficult to draw good conclusions. An outstanding presentation at the ARF's 75^th Annual Conference showed this quite well. Tags: Conferences, Market Research Innovation

Cybercrime is a fear for just about everyone, from individuals fearing identity theft to large corporation guarding sensitive data. The question is, how valid is this fear?   It is a question that was raised recently in an Economist article and it makes it clear that politicians are not the only ones who misuse and abuse numbers.

Claims have been made that cybercrime is bigger than the drug trade and that it costs a trillion dollars annually. Most of these figures come from firms who specialize in preventing cybercrime...in other words the same folks who will benefit if people feel the need to protect themselves from cybercrime. These figures are generally not questioned, either out of numerical ignorance or the belief (probably correct) that big numbers scare people and help to sell newspapers (or in today's world web hits).  

Tags: data security, misleading statistics

This month here in the States we will be celebrating our biggest secular holiday, Thanksgiving.   Traditionally, the holiday is thought to have started when early settlers to the "new" world, the Pilgrims, sat down to have a meal to celebrate the harvest with the Native American's who had befriended them. As we begin to close out 2011 in an industry facing an uncertain future, I was struck by the similarities between those early settlers and market researchers today.

On the surface the story of adventurers seeking a better life is a bit different than the story of boring market researchers seeking to survive, but I disagree.

Tags: Social Media, Market Research

In my last blog I talked about a simple chart on Morning Joe, which was presented by Steven Rattner. I submitted that when we see data presented in the media or especially by politicians, we should judge it in terms of how a researcher would have presented the same data (because of course researchers are free of bias...well let's leave that for another blog). I gave Mr. Rattner a pass last time, but his presentation of a chart on infrastructure was misleading and would only have pleased a client who wanted misleading data to prove a point.

In this case he presented a chart showing infrastructure spending as a percentage of GDP . It showed a massive drop from the high in the 1950's to the low of today. The chart had a y axis that went from 0% to 1.5% which made the drop easier to see. Nothing wrong with that (assuming those viewing the chart understood that it was not based on 0-100%).

Tags: misleading statistics, visualization, Reporting

A  few blogs back I talked about how the political season would bring on a rash of misuse and abuse of numbers. I've had my ears open for examples and a couple that came up recently got me to realize that a more nuanced view is necessary here. The real rule should be that pundits and politicians should be held to the same standards as we are by our clients. Namely, the numbers should help in the decision making process...not mislead or confuse the facts.

In the next two blogs I'll use some charts presented by Steven Rattner on the Morning Joe television program. For those of you who don't know, Mr. Rattner was the President's Car Czar. While this probably means he comes with his own bias, I have generally found that when he presents data he does so in a pretty fair way.

Tags: misleading statistics

Last time I talked about how we as an industry worry about response rates and respondent engagement either too much or for the wrong reasons. This time, I'd like to expand on that point by picking up on a comment made by Joan M. Lewis of Procter &Gamble.

The second day of the ESOMAR CONGRESS conference featured a panel of big research buying clients. They talked about the things they wanted and were not getting. Two big areas were boiling data down to as few charts as possible and to help them drive innovation and change. Both are related. In essence, don't give me a 100 page report or a chart with 100 numbers on it. Boil it all down and tell me what to do!

Tags: Reporting

Two other topics that came up a lot at ESOMAR were respondent engagement and representativeness.   Personally, I think discussions of the former are often misguided and discussions of the latter are a waste of time.   Not that I oppose engaging respondents or high response rates, just that I'm practical enough to recognize that neither will happen without a good business reason for them to happen.

With regards to response rate, the boat has clearly sailed. Surely this is clear now that huge research buyers like P&G suggest moving beyond focus on response rate. I suspect they, like me, would love higher response rates, but they have come to realize that it isn't going to happen. The massive increase in the number of surveys being done (I get one every time I take my car in, and I was just handed on here on my plane trip back from ESOMAR) has caused the public to tire of doing them. Add in that improving response rates involves greater costs (more attempts, mixed modes, higher incentives) and greater time. 

Tags: Untagged

I'm on the plane heading back from ESOMAR. I found the diversity of opinions and ideas shared there to be both interesting and thought provoking. Over the next couple blogs I'll share my thoughts on what I got from the event.

First off, gaming; no subject divides researchers more. Several presentations showed tests that used game elements to engage the respondents. One effort by MSI created a sort of fantasy backdrop in which players answered questions to get things they would need on their game quest. The idea was to engage respondents and with that get better data. Sadly, the results didn't back that up at all. Results did not vary much (specifics are available on the ESOMAR site), but respondents who did it were more engaged. At the same time, response rates were lower (loading time put some people off and some had no interest in the game). Easy enough to theorize that the mistake here was that the game was a sort of reward for doing the survey, but not related to it. As such, it does little to engage the respondent.

Tags: Untagged

I was watching the final round of the Bridgestone Invitational and my 14 year old son came in to the room.  I told him the established narrative. After a difficult two years Tiger Woods had returned to golf, but not before firing his long time and very loyal caddie.   Most saw this as just plain nasty on Tiger's part.  

I then told him how another golfer, Adam Scott, hired the caddie and was now on the verge of winning the tournament. I summed it up by saying that justice had prevailed.

He didn't even miss a beat before asking me, "Did Adam Scott fire his caddie so that he could hire the caddie Tiger fired?"

I don't follow competitive golf closely enough to know the answer. Worse, I had not even considered that the narrative "Tiger mean/Adam good" might be a bit off.  

A good lesson for any analyst to learn.

Tags: Market Research, misleading statistics, Insights

Sometimes as researchers we get too hung up on knowing everything.   We get frustrated by interesting findings that can't be explained with the available data and this can cause us to miss important insights. I suspect that the proliferation of available data will do little to help fill in the blanks...in fact, it might make the problem worse. A simple exercise in text analytics highlights this point.

There are now an array of tools available to help quantify and understand massive amounts of text.  For example,  at one of our conferences last year, Oded Netzer of Columbia University presented an amazing tool that analyses message boards and other online forums to learn about specific markets (slides can be found at:  http://www.trchome.com/research-knowledge/conferences/437). Tools like these provide a rich and valuable source of data, but insight can also be gleaned from far more simple approaches.

Tags: Text Mining, Statistics

The recent New MR Virtual Festival on presenting data had a number of really useful and interesting presentations. Mike Sherman’s presentation, “Less is More: Getting Value (Not Just Reams of Data) From Your Research” led to an interesting exchange that I think highlights the change in thinking that Market Research must make.

Mike reiterated the point that many have been making…we need to focus our reporting on the key things we learned and not waste executives’ time with a lot of superfluous information. In addition, the report should not just summarize the data, but rather it should synthesize it. He gave an example of a data set with these facts:

 

• · Jim broke his knee

• · A burglar broke Jim’s car window

• · Jim got a speeding ticket.

A summary of these data might be “Jim’s knee and car window were damaged and he got a speeding ticket”.

A synthesis of that data would be “Jim has been living dangerously”.

Tags: Market Research, misleading statistics, Science, Insights

The 2012 Presidential Election season is upon us. I don't know about you, but other than the barrage of commercials, the thing I like least about political campaigns is the terrible abuse of numbers. Combined with the current debate on the debt limit and we have the makings of a tsunami of misleading or outright incorrect statistics.

A  few weeks ago, Megan Holstine started a discussion about a Senator using a totally made up statistic. Sadly for him, he quoted a number that was far from accurate, but also one that was easily verified. His defense was that he didn't intend the statistic to be taken "literally".

Makes me wonder if perhaps we've got it wrong.  Think of the possibilities for us if we stopped taking numbers literally!

Tags: Statistics, Market Research, correlation, causation, misleading statistics, politics

A new book attempts to make behavioral economics interesting and approachable by couching it in the world of sports. Personally I try to avoid books on economics, but I did find a review quite interesting. Not only did it help to explain why the Philadelphia Flyers lost the 1980 Stanley Cup, but it also helps to illustrate the limitations of crowd sourcing and the reality of Asymmetry in key driver analysis.

Behavioral economics studies the role of emotion in economic decision making (something marketers need to master). In application it can help to explain the illogical decision making of shoppers. A classic example of this is when someone spends $1000 on a product they don't need thanks to a price cut of say $200. They will often focus on what they saved ("I saved $200!!!) and not on what they spent or the actual need.

Tags: Market Research, Behavioral Economics

Don't panic. CASRO's government affairs committee isn't warning this will happen and I don't have any evidence that it will. The point of the question is along the lines of "necessity is the mother of invention".

For example, over the past 15 years we have seen a move away from phone data collection and toward the web. Initially the focus was on cutting costs and ensuring the quality of the data were the same. As the industry embraced web, however, we began to use all kinds of innovative techniques that we simply could not do on the phone (or by mail for that matter). So, as we face a future with more and more access to data, I thought it would be interesting to think about what we would do if our traditional tools were simply taken away and we had to go cold turkey.

A good place to focus is Satisfaction research, which is already showing signs of decline. According to Inside Research (February, 2011), spending as a percentage of all MR has dropped in Europe (from 18% in '06 to 13% last year) and is stagnant at best in the states (11% last year which is in line with 12% in '09 and 10% in '08). I suspect this decline is not an indication that firms no longer care about satisfaction. More likely it reflects cheaper data collection methods and a realization that it need not be measured as intensively as in the past.

So in a world with no traditional MR, how will firms measure and impact satisfaction?

 

Tags: Untagged

I come to you, as is the tradition, with glad tidings for the New Year.

I do this in the midst of a lot of doom and gloom talk about the industry and our future. At CASRO's annual meeting, Simon Chadwick talked about "Do it Yourself" (DIY) research continuing to grow with no end in sight. A recent LinkedIn thread asked if there was a better word for 'survey' that wouldn't carry the negative connotations. The MR Heretic calls their site "Market Research Deathwatch" with constant warnings about engaging respondents better or destroying our industry. Add in the worst couple years the industry has ever faced and purchasing departments increasingly viewing us as commodities and you have the makings of glad tidings indeed!

 

Tags: Untagged