TRC

Beyond Crosstabs:
The Usefulness of Covariate Analysis

Consider a very common occurrence in marketing research: the need to tell if two groups are different on a given measure. Examples could be differences between men and women in an awareness study, or current and former customers in a satisfaction study. The usual approach in such situations is to use cross-tabulation where the groups in question are positioned as banner points and the measure is set up as a stub and the objective is to see if there are statistically significant differences between the two groups on the given measure. If there is, the groups are then deemed to be different from each other on that measure. This is a commonly used procedure and, in fact, many cross tabulations are run after research studies to identify if differences exist between specified groups. But could there be a problem with the conclusion that the two groups are different? Are there situations where the cross tabulations could show a difference that did not exist in reality Yes, and in this article we will look at one major issue and the solution.

Imagine that a company is conducting a satisfaction study and is interested in understanding whether satisfaction varies by ethnicity. Let's say it is a financial services company that has branches in different parts of the country, some of which are more likely to serve customers of certain ethnicities. Hence, understanding the relationship between satisfaction and ethnicity is important to the company. Let's also say that the company is particularly concerned about Hispanic customers because it sees them as a rapidly growing market and one that is potentially vulnerable to low satisfaction scores. The common approach here would be to run cross tabulations to identify if differences exist between Hispanic and non-Hispanic customers on satisfaction. Let's say the company did run this analysis and it showed a significant difference between the satisfaction scores of these two groups. If the analysis stopped there the recommendation to the marketing team may have been to explore further the reasons for this discrepancy and perhaps to spend considerable resources trying to address the problem.

Bivariate and Multivariate Analysis

But was there really a difference between the two groups? This is where multivariate analysis comes into play. To understand it, let's first take a step back and consider an old but reliable technique: multiple regression analysis. This technique is normally used in situations where one needs to identify the key drivers of a given target variable, say, overall satisfaction. The general approach would be to run a regression model with overall satisfaction as the outcome (or dependent) variable and a group of attribute satisfaction questions as predictor (or independent) variables. While most or all of those attributes may have a positive and significant correlation with overall satisfaction, regression works in such a way that it identifies the individual, adjusted impact of each predictor variable. Let's consider this in a bit more detail.

Let's say there are two variables A and B that have large, positive and significant correlations with overall satisfaction (see Figure 1). Let's also say that A and B are largely independent of each other, that is, they have very low correlation with each other. The regression works in such a way that, more often than not, both A and B will show up as key drivers of overall satisfaction, . That is because they are independently contributing to changes in overall satisfaction. To put it another way, a change in A could influence overall satisfaction but not affect B, while a change in B could influence overall satisfaction without affecting A. So it is reasonable that they should have independent impacts on overall satisfaction with neither being overly affected by the other.

Figure 1: Uncorrelated Independent Variables

Now consider two more variables C and D that also have large, positive and significant correlation with overall satisfaction (see figure 2). Let's also say that C and D have a strong positive correlation with each other, that is, they are not independent. More likely than not, one of these variables will show up as a strong driver of overall satisfaction, while the other may not show up as a driver at all. Again this is reasonable since having one variable is sufficient to explain changes in overall satisfaction and having both would be over-counting the impact of the construct that the two variables jointly represent.

To read the rest of this article and have access to the entire TRC white paper library, please click here.