Over the years our clients have increasingly looked to us to condense results. Their internal stakeholders often only read the executive summary and even then they might only focus on headlines and bold print. Where in the past they might have had time to review hundreds of splits of Max-Diff data or simulations in a conjoint, they now want us to focus our market research reporting on their business question and to answer it as concisely as possible. All of that makes perfect sense. For example, wouldn’t you rather read a headline like “the Eight Richest People in the World Have More Wealth than Half the World’s Population” than endless data tables that lay out all the ways that wealth is unfairly distributed? I know I would…if it were true.
The Economist Magazine did an analysis of the analysis that went into that headline-grabbing statement from Oxfam (a charity). The results indicate a number of flaws that are well worth understanding.
• They included negative wealth. Some 400 million people have negative wealth (they owe more than they own). So it requires lots of people with very low positive net worth to match the negative wealth of these 400 million people…thus making the overall group much larger than it might have been.
• For example, there are 21 million Americans with a net worth of over $350 Billion. Most of them would not be people you might associate with being very poor…rather they have borrowed money to make their lives better now with the plan to pay it off later.
• They were looking at only material wealth…meaning hard assets like property and cash. Even ignoring wealth like that of George Baily (“The richest man in town!”), each of us possesses wealth in terms of future earning potential. Bill Gates will still have more wealth than a farmer in sub-Saharan Africa, but collectively half the world’s population has a lot of earnings potential.
• Most egregious, the article claims that Oxfam did the math wrong. They calculated the poor’s wealth at $409 Billion when in fact it should be $384 Billion. The difference is enough that they could have said “the seven wealthiest…”, since Warren Buffet (#8) is worth less than $25 Billion (how does he make do?)
• One final point, they were measuring wealth and not income. While that should be obvious to the reader, many pundits reported it differently.
While our work rarely involves numbers in the Billions (except for market projections perhaps), there is plenty of room for error. Whether we are talking about new product development, pricing, segmentation or any other type of research, I strongly support being concise. For it to work though, we must be not only thorough and accurate in our analysis but careful in how we word our recommendations to ensure they are in no way misleading.
Consider the case of Oxfam. In the end, their point was accurate…wealth is very unevenly distributed. However, by ignoring the points above they made their result subject to question. Thus it allows opponents to pick on the bad analysis rather than the spirit of the message. The same tactic will be used by stakeholders within our clients’ organizations if they disagree with our recommendations.
Rich brings a passion for quantitative data and the use of choice to understand consumer behavior to his blog entries. His unique perspective has allowed him to muse on subjects as far afield as Dinosaurs and advanced technology with insight into what each can teach us about doing better research.