Welcome visitor you can log in or create an account

800.275.2827

Consumer Insights. Market Innovation.

blog-page
Rich Raquet

Rich Raquet

President, TRC


Rich brings a passion for quantitative data and the use of choice to understand consumer behavior to his blog entries. His unique perspective has allowed him to muse on subjects as far afield as Dinosaurs and advanced technology with insight into what each can teach us about doing better research.  

Really enjoyed the IIeX Greenbook conference. I generally concurred with the opinions expressed and many of the presentations gave me ideas on how we might better serve our clients. Thought I might share some of my reflections here.

In general terms this was a conference that likely scared more than one researcher to jump. For example, Charles Vila the head of Campbell Soup’s Consumer and Customer Insights for North America said that within five years he doesn’t expect to use any survey data.   Personally, I tend to disagree with such sweeping statements (hopefully this won’t prevent me from working with Campbell’s moving forward), but perhaps they are necessary to shake our often complacent industry into thinking differently.

In that regard, Campbell’s is a good example. Their flagship product is soup, a product that has been around forever and sold by them for 100 years. This doesn’t stop them from innovating not just with new products, but in the way they engage the customer. Their staff is immersed in the latest gadgets that consumers are using so they can better understand how they can be employed in Campbell’s marketing efforts.

So, I’d encourage researchers to do the same. Ultimately it doesn’t matter if surveys go away or simply cease to be the primary form of data collection. If we allow ourselves to be defined by how we acquire data then we deserve to go the way of the proverbial buggy whip manufactures did at the turn of the last century.

The great news is that many of the new technologies being shown off are not really competing with us. Most seek to provide new tools for traditional research companies to use.   Some might replace surveys and others augment them. Some are really just surveys in another form (such as Google’s) and there are new ways to design and implement surveys to better get at the truth (my partner Rajan Sambandam’s presentation on “Behavioral Conjoint” being one self-serving example). The possibility of improving our ability to guide product development, pricing research and marketing is one we should embrace.

...

Big news today as Ron Johnson, CEO of J.C. Penney “resigned”. He did so following a series of decisions designed to make the staid Penney’s brand more hip. Sadly, thus far these changes have chased away existing customers without attracting enough new ones to turn around the store chain’s long decline.

Johnson’s decisions, like those of his mentor Steve Jobs, were made from the gut…no need for any market research. Jobs of course had an almost magic touch. Carefully choosing the markets to enter, when to do so and producing products that were seen as cutting edge. Often his decisions were seen as counter intuitive (such as the opening of retail stores in the Internet age), but time and time again he was proven to be right. So, why didn’t it work for Mr. Johnson?

First, Mr. Jobs was producing products for Apple, not J.C. Penney. Apple was known as a producer of fine computers that were easy to use (intuitive is a word often used). Some of the luster came off that reputation when Jobs left the company, but when he returned there was little doubt what Apple stood for and the types of products to expect. From the moment he returned he looked for places where that reputation (intuitive electronics) might find a market. Mr. Johnson, by contrast saw the J.C. Penney reputation as a problem and looked to change it…a far tougher task.

Second, Jobs often had success by leaping into relatively new markets and then using the power or Apple design and engineering to dominate it. He didn’t create the first digital music player, but he created one that was intuitive and he backed it up with a legal way to buy digital music. He could do this without giving up the existing Apple business (computers). Johnson needed to focus resources on shoring up the flailing store chain…perhaps if he’d had the luxury of creating small J.C.Penney Boutiques it would have worked.

Third, I am reminded of something that I once heard the legendary Warren Mitofsky say with regards to flawed sampling, “Results will be right until they aren’t”. Mr. Jobs was not always right. He left Apple the first time a failure (one could argue that others were at least as much to blame), started a new company that was largely a failure and then started his run, first at Pixar and then his triumphant return to Apple. He was clearly brilliant and had incredible vision, but he was not always right. Of course, it goes without saying that not everyone has the same skills (me included).

...

Market Research in the Toilet

Posted by on in New Research Methods

market research in toiletI read an astounding fact this week, “More Indians have used a mobile phone than a toilet”. It seemed absurd to me that a relatively new technology would outpace an old (and very useful) one. I came to realize that the absurdity was mainly due to the fact that I couldn’t imagine a world without either device and it struck me that this is an example of what ails the market research world.

The fact is that Indians have not chosen the cell phone over indoor plumbing. The former is widely available (because cell phone infrastructure is relatively easy to build) and the latter is not. So it wasn’t a choice of toilets over telcom, it was a choice of having a cell phone or not having one. Those who got the phones have begun to find uses for it that go far beyond the obvious. For example, fishermen call in while at sea to find out which port is offering the best price for their catch, thus maximizing their profits.

In Market Research we are often blinded by our experience. Instead of viewing new market research technology for its potential, we view it through the lens of what we know. When web data collection arrived, many didn’t see the opportunities it offered and instead defensively dismissed it as being inferior to existing methods and only offered the benefits of being “cheap and fast”. After more than a decade, it amazes me how many still hold this belief.

Recent Comments - Show all comments
  • Ed Olesky
    Ed Olesky says #
    @Dave: thanks for the comments. I recently saw a post on Facebook that said something like this: "If someone came 30 years from
  • Ed Olesky
    Ed Olesky says #
    Well done Rich... I am in the middle of writing a paper for ESOMAR Congress this september in Istanbul on the subject of toilets a
In my last blog I talked about the value of market research even if all it does is validate what you thought you already knew. A further question might be, "Should we encourage our clients to hypothesize?". My answer would be a definitive "YES!".

My answer is likely biased by the fact that we work with Hierarchical Bayesian (HB) Analytics so frequently (mainly using choice data such as that created by conjoint). After all, HB requires a starting hypothesis. But the reality is that even if we don't use HB, a hypothesis is a useful thing.

First, understanding what our clients EXPECT to find is a great way to understand what they NEED to find. They need to validate or reject their prior thinking so the more we understand their thought processes the more we know where to focus. In addition, this understanding often leads to insight into their firm's business decision making. This helps us to present results that tell a story that resonates with them. This is true even if the findings contradict their thinking.

Second, by presenting results in this way we help our clients to do more than meet the objectives of the current study, but to walk away with a better understanding of what to expect in the future. Flaws in logic will help them to avoid those flaws when similar issues come up.

Of course purists will point to the risk that starting with a hypothesis may bias our results. We might be inclined to design our research and reporting to match the narrative we expected to find. We might also be tempted to avoid the "kill the messenger" problem by sugar coating the truth.

These are fair points and well worth guarding against. They do not, however, undercut the premise that having a starting hypothesis makes for better market research and likely better use of results.

Hits: 5566 0 Comments
higgs bosonI read an article about the discovery of the Higgs Boson at CERN. This is the so called "god particle" which explains why matter has mass. While the science generally is beyond me, I was intrigued by something one of the physicists said:

"Scientists always want to be wrong in their theories. They always want to be surprised."

He went on to explain that surprise is what leads to new discoveries whereas simply confirming a theory does not. I can certainly understand the sentiment, but it is not unusual for Market Research to confirm what a client already guessed at. Should the client be disappointed in such results?

I think not for several reasons.

First, certainty allows for bolder action. Sure there are examples of confident business people going all out with their gut and succeeding spectacularly, but I suspect there are far more examples of people failing to take bold action due to lingering uncertainty. I also suspect that far too often overconfident entrepreneurs make rash decisions that lead to failure.

Second, while we might confirm the big question (for example in product development pricing research we might confirm the price that will drive success) we always gather other data that help us understand the issue in a more nuanced way. For example, we might find that the expected price point is driven by a different feature than we thought (in research speak, that one feature in the discrete choice conjoint had a much higher utility score than the one we thought was most critical).

...

Market Research Data, A Love Story

Posted by on in Market Research

If you have read my blog, you know that I love digging through data to find new insights and I’m a believer that choice questions (such as those used in Discrete Choice Conjoint or MaxDiff) are the best way to engage respondents and unlock what they are thinking. Given that, a book called “Data, a Love Story” should be a natural fit for me because it is about the ultimate choice…choosing the right person to marry. Ultimately I decided against buying the book (wasn’t sure my wife would see it as purely a curiosity). At the same time, the review I read made me realize that some of the issues the author faced, are the same as those we face as researchers.

The premise is that dating websites can be gamed to find the right mate. Having never used one (my marriage pre-dates them), I assumed that these sites use complex algorithms to match compatible people. The trouble is that while this is true, these algorithms can break down.

First off, many people are not honest in their profile. They might be looking for someone to sit around with and watch television but admitting that is tantamount to saying “I’m really lazy” so they fudge a bit. Some go beyond this and tell whoppers like “I’m not married”. Obviously any bad data will lead to bad matches.

Second, aligning profiles is only a first step…it determines which profiles an individual sees. At that point the individuals are free to contact each other or not. Thus, how that profile reads is more important than the questions that determine the “match”.  

The author, Amy Webb, decided to gather her own data. After crunching the numbers she was able to both better attract invitations from the right men AND figure out which of them she should be talking to.  

...

Asymmetry and the Lottery

Posted by on in Market Research

If the lottery can accurately be called a “tax on the stupid”, does my playing it make me stupid? To understand (or perhaps rationalize) the answer, you need to understand the principles of Asymmetry

As usually happens when the jackpot on PowerBall goes into the stratosphere (in this case it reached nearly $600 Million), someone here at TRC started a collection to play as a group. A pretty high percentage of our staff decided to play, even those with the most advanced degrees in statistics. So given the chances of winning are something like 1:175 million per ticket, why did we do it?

It certainly wasn’t that by buying so many tickets (nearly 50), the odds became anything near a slam dunk. In fact, they were easy enough to calculate (1:3,650,489.79) so there was no doubt in my mind that I wouldn’t win when I played and yet I still did.

The reason was simple. I had to choose to play or not to play and consider the likely outcome if we won or didn’t win:

  • I play and lose (A small $6 loss and an outcome that my brain expected all along)
  • I play and win (A massive win with my share being $10Million…despite expecting to lose, my brain is now elated)
  • I don’t play and they lose (I have some very minor bragging rights, but ultimately I missed out on the fun and only saved $6)
  • I don’t play and they win (Even as I console myself that the odds were with me, I feel like a complete idiot)

In other words, playing offered only upside and not playing only downside. That is exactly why we consider Asymmetric effects whenever we do analysis.   Otherwise we may miss what really drives consumer decision making.

Hits: 5904 0 Comments

A friend of mine posted on Facebook that she’d taken a web quiz to tell her which presidential candidate best lined up with her stand on the issues. She was outraged that the web site thought she would vote the way it did. I’m not surprised (by the outrage, not her choice)…it is a case of a badly applied choice technique.

Basically the quiz worked by asking a series of questions to see where she stood on the issues. It then aligns her choices against the stand taken by the candidate (if you want to try one, here is one from the GOP Primaries this year). In essence it is a Configurator. Instead of building the perfect product for you (as you would with a Configurator) you build the perfect candidate. There are a couple of problems with this application.

First, Configurators allow you to build the ideal but generally don’t give a clear idea of what choices you might make if that ideal were not available (our proprietary Texo™ helps overcome that issue). In politics it is not unusual for voting decisions to hinge on a single issue and unlike products you can’t decide to add or subtract an important feature.  

When we dropped my daughter off for her first year of college a few weeks back my parting words were “Be true to yourself”. I thought this reflected both my accepting that my influence on her was now very limited and my hope that whatever good I’ve done should be put into practice. It strikes me that researchers too should heed the advice.

Our industry has changed and continues to change. Many of the old rules either no longer work or can’t be easily applied to the new tools at our disposal. So how can we apply what we know? A philosophy like “be true to yourself” allows us to do just that.

Personally it has allowed me to accept that representative sampling is no longer the most critical rule (it can’t be in a world where truly representative sampling is too slow and costly). It doesn’t mean I take any respondents I can get…care in trying to get as representative a sample as we can remains important. It just isn’t a stone cold requirement of quantitative research.  

Recent comment in this post - Show all comments
  • Ed Olesky
    Ed Olesky says #
    Nice article, thanks for the information.

The Olympics of Statistics

Posted by on in A Day in a (MR) Life

Watching sports provides a lot of great entertainment. The thrill of victory, agony of defeat and all that. It also provides many great opportunities for never ending arguments about just how great various sports achievements are. Often these arguments are bolstered by the misuse of statistics. One such example was the constant references to Michael Phelps as “the Greatest Olympian Ever” which was based on the fact that he’d won more medals than any other athlete in history.  

To be clear, I’m sure an argument can be made that he is the greatest ever, but the use of one number, medal count, to determine that really bothers me. As often happens in the media, the number is looked at in only one context (compared to the number of medals other athletes have won) rather than considering a great number of other factors:

imaginelehrerOn vacation I read a number of books (love my Kindle) including Why Nations Fail by Daron Acenoglu and James Robinson and Imagine by Jonah Lehrer. While clearly quite different, one on what has allowed some nations to grow and endure while others fail and the other one about unlocking the creative processes of the brain; I took away lessons for my work from both.

“How Nations Fail” isn’t a business book. It is more of a history book than anything, but I saw parallels with what we are facing. The book details a long string of historical examples of nations that either failed outright or that saw some success but then reversed course. The central core is that nations that succeed over time always feature the same factors which feature truly inclusive systems. Meaning, everyone has a chance to succeed on an equal footing. 

mra market research conference 2012Spent a good bit of last week at the MRA conference in San Diego. The weather was overcast and cloudy for the first couple days, a perfect metaphor for the general mood of the industry and uncertain outlook the future holds for us. But as always, I saw a lot to be optimistic about. In particular the first and second to last presentation I watched featured experience researchers who are enthusiastically embracing the opportunities that exist today.

Hal Bloom of Sage Software talked about their satisfaction research using a standard likelihood to recommend approach. They attempt to survey every customer every year and succeed in getting 20% of them to respond. This means tens of thousands of surveys with a multiple of that in terms of open ended responses. Sage makes extensive use of text recognition software to determine sentiment and help sort out who their most vocal promoters and detractors are. A great use of new technology, but what struck me even more was what they do next.

Like any research, market research has always recognized that to be certain results of research can be projected to an entire population; you need to eliminate any bias. We worried about things like:

  • Representativeness Effects – Needed to not only make sure we selected a random representative sample, but then do everything possible to maximize the percentage of people who completed the survey.
  • Interviewer Effects – Surveys needed to be done identically.   If one was done by mail, all should be with identical forms. If done by phone interviewers needed to be careful not to lead respondents and to keep pacing at consistent rate.
  • Framing Effects– If responses from one question are going to potentially bias a future response then the order should be changed to reflect it. In cases where changing the order merely changes which question biases which, use rotation or split samples so that bias effects can be measured and softened.

I know this is a simplified view of things, but the above three do get at the major forms of bias that we seek to eliminate in market research. In this blog, I'll focus on representativeness and at some point in the future I'll cover the other two.

low response rateA recent discussionon Linkedin pondered whether MR is having its own global warming crisis in the form of an ever dwindling respondent pool. As always, this brought on arguments that response rates need to be improved, quality enforced and of course talk about how much we have slipped as an industry since the good old days. Some blame clients for this (they demand speed and lower cost without concern for quality!) and some blame researchers for not holding clients’ feet to the fire.   It struck me that this is yet another case of researchers not viewing things from a client perspective.

market research conferenceOver the past year I’ve blogged about the things that I think will drive the future of Market Research and I’m pleased to announce that for our Frontiers of Research annual conference (May 8th, in NYC, view full agenda or register) we have assembled speakers who will drive that conversation forward. The conference will cover the full spectrum of buzz-worthy topics (Behavioral Economics, Neuroscience, Gamification, Predictive Analytics). And the focus, as always, will be on ideas presented in an easy to understand way (no math!). With speakers from four Ivy League schools, and presentations that range from poker to motion picture box office, this should be an informative and enjoyable day.

Leonard Murphy will set the table by calling on his extensive knowledge of the industry to illuminate how academia can and is driving us forward. Anyone who follows his blog knows that he is not only one of the most knowledgeable industry leaders around, but that he has a provocative view of where we are heading.

Even Economists Are Gamifying

Posted by on in New Product Research

Gamification as a means to understand consumer choice is a relatively new idea for research (and controversial in many circles), but it is not new everywhere. For example, one sociologist, Dmitri Williams, has been studying economic behavior using gamfication for four years. His experiments were based on the online fantasy game EverQuest II, which involves thousands of players selling millions of virtual items every month. In essence it is a fantasy economy that works like a real economy.  

Professor Williams theorized that this provided an opportunity to observe the choices players made without fear of the Hawthorne effect (some people give different answers when they know they are being watched).   It also allowed him to set up test and control groups and observe what happens when, to take a simple example, prices go up (if you guessed “people buy less” you win) and to look at gender roles. He saw applications in many fields, not the least of which being testing the impact of various government intervention options before implementing them in the real world.

Discrete Choice in a Police Lineup

Posted by on in New Research Methods

police lineup discrete choiceThe Economist reviewed a study by Dr. Neil Brewer about effective police lineups which I think had implications for Market Research. Like researchers, police typically like to encourage witnesses to take their time to ensure they are making the correct choice. This makes logical sense, more time, means more thinking which naturally should lead to better results. Sadly, Dr. Brewer found otherwise.

He had volunteers view short films which detailed mundane scenes of everyday life and a crime (shoplifting, car theft, etc). Later (some minutes later, some a week), they were asked to identify the criminal from a group of 12 pictures of “suspects”. Half were given 3 seconds to evaluate each picture and asked how confident they were of their choice. The other half were given as much time as they wanted. The results showed that the group that had the limited time was correct 67% of the time. The group with more time was only correct 49% of the time.  

You Think Researchers Have It Tough?

Posted by on in Healthcare

For the past few years MR blog posts have been dominated by posts questioning the future of Market Research or talking about just how tough it is to be a researcher in the new millennium. A recent discussion on Linkedin about the threat from DIY is a good example. If you read my blog frequently you know that I see the industry evolving, not going extinct. In any case, at TRC we do a great deal of research about Health Insurance and so I know that as challenging as research is, it is nothing compared to what the health insurance industry is going through.

First off, I'll ignore issues that have been with the industry for decades. More often than not they don't sell to the folks who use their products (most insurance comes through employers) and they often don't sell to the folks who pay the bills (a majority of insurance is sold through independent brokers). While some research clients don't expose us to their internal clients, we are nowhere near as separated from the folks who use our work as health insurance firms are.

Tagged in: Brand Market Research

webcamAs researchers it is critical that we ensure our data accurately reflect the thinking of the market....in other words, getting to the truth. This is complicated by several factors including limitations of a questionnaire, respondent's lack of attention and the fact that people don't always know what they really want or need. While careful design and methodology can help to minimize these issues (at TRC we believe in using choice questions and shorter surveys) and the use of other data (which can establish the facts), it is impossible to eliminate them.

Technology such as eye tracking, bio metrics and facial recognition software can be applied to neuroscience to help us understand more about what respondents are thinking. The trouble is they are often expensive (sometimes getting the whole truth isn't worth the price) and slow down the research process (sometimes a faster less complete answer is better than a slow one). The limited data available also make it difficult to draw good conclusions. An outstanding presentation at the ARF's 75th Annual Conference showed this quite well.

Is Cybercrime a Huge Problem?

Posted by on in A Day in a (MR) Life

cybercrimeCybercrime is a fear for just about everyone, from individuals fearing identity theft to large corporation guarding sensitive data. The question is, how valid is this fear?   It is a question that was raised recently in an Economist article and it makes it clear that politicians are not the only ones who misuse and abuse numbers.

Claims have been made that cybercrime is bigger than the drug trade and that it costs a trillion dollars annually. Most of these figures come from firms who specialize in preventing cybercrime...in other words the same folks who will benefit if people feel the need to protect themselves from cybercrime. These figures are generally not questioned, either out of numerical ignorance or the belief (probably correct) that big numbers scare people and help to sell newspapers (or in today's world web hits).  

Want to know more?

Give us a few details so we can discuss possible solutions.

Please provide your Name.
Please provide a valid Email.
Please provide your Phone.
Please provide your Comments.
Enter code below : Enter code below :
Please Enter Correct Captcha code
Our Phone Number is 1-800-275-2827
 Find TRC on facebook  Follow us on twitter  Find TRC on LinkedIn

Our Clients