We are now officially two months into 2017, which means it’s time to keep up with those New Year’s resolution goals. Resolutions can be difficult to attain in both personal and professional life settings. Recently, I stumbled upon an article by Crawford Hollingworth, an interesting read about behavioral science and its effect on New Year’s resolution goal attainment. As I was reading the article, I realized the suggestions for preparing resolution goals provided in the article also relate to the process of preparing a market research study. The four steps for developing a New Year’s resolution recommended in the article are: Make a plan, Substitute old behavior for new behavior, Make it easy, and Make only one New Year’s resolution. My view on how these strategies relate to market research is as follows:
The first step of the market research journey is to make an action plan. Figure out what the objective of your research is going to be – what do you want to know and from who do you want insight? Next, consider the methods through which you will obtain the most meaningful and useful results for your research objective. Finally, put together a schedule that includes every aspect of the research, including questionnaire design, fielding the survey, data delivery and reporting the research findings.
In the grand scheme of market research methodologies, there are plenty of approaches to choose from that will provide the results needed to make powerful decisions about your product or service. Of course, it is normal human behavior to have the desire to stick to what you know, and market research isn’t much different. However, methodologies are continuing to evolve and can provide findings in various ways. For example, TRC has developed methodologies such as Message Test Express™, Idea Mill™ and Bracket™, along with other solutions that are increasingly popular among the research we conduct. This is an opportunity to be creative and try methodologies that have been tested and offer proven results, which will allow you to view research findings from an alternative perspective.
In order to get reliable results from your research, it is best to start with consideration of the questionnaire design. Plan the design with the end in mind first, then work your way to the front; if you consider what you want to know first, the questions themselves will come together easily. This will allow you to easily interpret and analyze data during the final reporting stages. On the other hand, in terms of the actual survey, you want to avoid developing questions that are overly complicated or time consuming for respondents. Make sure the questions asked make sense and the instructions are clear and concise so that respondents can quickly grasp the idea of what you are asking of them.
A colleague of mine, Rajan Sambandam, provided insight during a recent meeting about the scope of market research studies being “Broad and Shallow” versus “Narrow and Deep” that I found to be interesting. A take-away from his statement is that you should either have a broad and shallow scope through which you will have less informative findings about a larger group of topics, or a narrow and deep scope through which you will have an abundance of detailed findings about one topic. Instead of striving to accomplish both “broad and shallow” and “narrow and deep” research in one initiative, focusing on one or the other will provide the most meaningful and useful information to be applied to your product or service....
Over the years our clients have increasingly looked to us to condense results. Their internal stakeholders often only read the executive summary and even then they might only focus on headlines and bold print. Where in the past they might have had time to review hundreds of splits of Max-Diff data or simulations in a conjoint, they now want us to focus our market research reporting on their business question and to answer it as concisely as possible. All of that makes perfect sense. For example, wouldn’t you rather read a headline like “the Eight Richest People in the World Have More Wealth than Half the World’s Population” than endless data tables that lay out all the ways that wealth is unfairly distributed? I know I would…if it were true.
The Economist Magazine did an analysis of the analysis that went into that headline-grabbing statement from Oxfam (a charity). The results indicate a number of flaws that are well worth understanding.
• They included negative wealth. Some 400 million people have negative wealth (they owe more than they own). So it requires lots of people with very low positive net worth to match the negative wealth of these 400 million people…thus making the overall group much larger than it might have been.
• For example, there are 21 million Americans with a net worth of over $350 Billion. Most of them would not be people you might associate with being very poor…rather they have borrowed money to make their lives better now with the plan to pay it off later.
• They were looking at only material wealth…meaning hard assets like property and cash. Even ignoring wealth like that of George Baily (“The richest man in town!”), each of us possesses wealth in terms of future earning potential. Bill Gates will still have more wealth than a farmer in sub-Saharan Africa, but collectively half the world’s population has a lot of earnings potential....
I always dread the inevitable "What do you do?" question. When you tell someone you are in market research you can typically expect a blank stare or a polite nod; so you must be prepared to offer further explanation. Oh, to be a doctor, lawyer or auto mechanic – no explanation necessary!
Of course, as researchers, we grapple with this issue daily, but it is not often we get to hear it played out on major news networks. After one of the debates, I heard Wolf Blitzer on CNN arguing (yes arguing) with one of the campaign strategists about why the online polls being quoted were not "real" scientific polls. Wolf's point was that because the Internet polls being referenced were from a self-selected sample their results were not representative of the population in question (likely voters). Of course, Wolf was correct, and it made me smile to hear this debated on national TV.
A week or so later I heard an even more, in-depth consideration of the same issue. The story was about how the race was breaking down in key swing states. The poll representative went through the results for key states one-by-one. When she discussed Nevada she raised a red flag as to interpreting the poll (which has one candidate ahead by 2 - % points). She further explained it is difficult to obtain a representative sample in Nevada due to a number of factors (odd work hours, transient population, large Spanish speaking population). Her point was that they try to mitigate these issues, but any results must be viewed with a caveat.
Aside from my personal delight that my day-to-day market research concerns are newsworthy, what is the take-away here? For me, it reinforces how important it is to do everything in our power to ensure that for each study our sample is representative. The advent of online data collection, the proliferation of cell phone use and do-it-yourself survey tools may have made the task more difficult, but no less important. When doing sophisticated conjoint, segmentation or max-diff studies, we need to keep in mind that they are only as good as the sample that feeds them.