TRC

By Rajan Sambandam

When conducting segmentation analysis there are several issues to consider, not the least of which are the questions used for analysis. More than almost any other factor, the questions asked will influence the quality of the segmentation results. The two main issues with regard to questions are:

• Question content
• Question type

Question content refers to the subject areas covered by the questions. They could be attitudinal, behavioral, demographic and such. While there are plenty of issues to discuss on this topic, that is not the focus of this article. We will confine ourselves to a discussion of question type.

Question type refers to how questions are asked. Various types of scales are often used to collect data for segmentation analysis. The primary purpose of all of these scales is to be able to get sufficient discrimination between respondents. If a scale is unable to discriminate between respondents, then it is not contributing anything useful to a segmentation analysis. Using such a scale would be no different from using a constant in the analysis.

Consider that one of the most popular types of questions asked in segmentation studies is the importance question. This could take various forms such as importance of product features, brands, decision criteria and so on. The traditional way of asking the question is to use an importance scale where each of these items is rated (perhaps on a 1-10 scale). The problem with this approach is that the respondent considers each item in isolation and further, has no incentive to say that anything is unimportant. As a result one often sees data where many items are rated as important. More damagingly from a segmentation perspective, the questions don’t sufficiently discriminate between respondents. This greatly reduces the usefulness of these questions in the analysis. So, what is the alternative?

Trade-Off

One could use an approach where respondents to a survey are asked to make trade-offs, rather than rate each item in isolation. For example, a pair-wise comparison task could be used where the respondent indicates the item in each pair that is more important. While this has the capacity to provide better discrimination between respondents, it is also more tedious as the number of pairs to be evaluated quickly balloons. Designs can be used to pare down the number of pairs, but the fundamental problem is that we are not making use of the respondents’ full cognitive capacity. Comparing two items and choosing the more important one is often a very easy task. Respondents have the capacity to choose from more than two items at a time, and it is precisely this ability that Maximum Difference Scaling (Max-Diff) exploits to give us better results.

Max-Diff is a recent development in statistical analysis. It is a comparative method where respondents are shown sets of items and asked to pick the best and worst, or most and least important item, in each set. The number of items shown per set usually varies from 3 to 5. The manner in which the items are grouped together and the order in which they appear are carefully selected through an algorithm. Data are then analyzed using hierarchical Bayes estimation to provide importance scores for all of the items used in the design. Scores appear like percentages and add up to 100. Since respondents have to make comparisons and choices, the problems mentioned with traditional importance scales are largely absent.

It therefore stands to reason that using Max-Diff to collect data for segmentation is likely to be more fruitful than using importance scales. In order to demonstrate this, a test was conducted and the results are reported in this article.

To read the rest of this article and have access to the entire TRC white paper library, please click here.