Welcome visitor you can log in or create an account

800.275.2827

Consumer Insights. Market Innovation.

blog-page
Posted by on in Researchers are communicators
  • Font size: Larger Smaller
  • Hits: 359199
  • 0 Comments
  • Print

Should You Need a License to Quote Statistics?

The 2012 Presidential Election season is upon us. I don't know about you, but other than the barrage of commercials, the thing I like least about political campaigns is the terrible abuse of numbers. Combined with the current debate on the debt limit and we have the makings of a tsunami of misleading or outright incorrect statistics.

A  few weeks ago, Megan Holstine started a discussion about a Senator using a totally made up statistic. Sadly for him, he quoted a number that was far from accurate, but also one that was easily verified. His defense was that he didn't intend the statistic to be taken "literally".

Makes me wonder if perhaps we've got it wrong.  Think of the possibilities for us if we stopped taking numbers literally!

A short list of taking numbers literally:

  • Politicians and interest groups quote statistics that are either totally made up or gathered using very questionable methods. To this day people will quote that domestic violence skyrockets on Super Bowl Sunday even though the truth is the complete opposite. Of course domestic violence is a terrible thing and a problem (for real stats go to: http://www.ncjrs.gov/pdffiles1/nij/183781.pdf) , but statistics like this are the equivalent of counting any customer who gave a rating higher than "1" on a ten point scale as "satisfied".
  • Another great trick is quoting an accurate number that is meaningless to the question at hand. The current budget debate is full of claims like this...on both sides. I'll stay clear of that mess though and go back to one I'll never forget that then Vice President George H. W. Bush used to back up how great the jobs the Reagan administration had created were. He said something like "A majority of these jobs have an average salary of $XYZ", can't recall the figure. My point isn't that the jobs created were not good...just that this statistic tells us nothing. Most people hearing the quote didn't consider that he was only talking about the highest half of wage earners or that using average, or mean, in situations where there are likely to be some extreme outliers (Warren Buffet for example) is far less useful than using median. The reality is that we often quote statistics on sub samples (the top 1/5th of your customers spend $XYZ on average), but not without making it clear and not without considering outliers (if the top 1/1000th of customers spend 100 times as much as the rest of the top quintile then we'd likely use median and point out this lucrative group at the top.)
  • Finally there is the great misuse of a statistical term. Correlation is probably the one used most often. We will often hear about irrefutable proof that x causes y. Basically someone might figure out that cancer rates among heavy cell phone users is higher than among light cell phone users (of course they will choose the definition of "heavy" and "light" that best suit their premise) and thus cell phones cause cancer. Thanks to stumbleupon.com I became aware of a cool way to show how correlation and causation are not the same thing. Basically they generate pairs of random numbers, plot them on an x/y axis, then overlay the dots on a random map. If you assume the dots mean something bad, you can easily start to come up with explanations of what is causing them by looking at where they cluster on the map...since it is all made up information it highlights the point quite well. Or, for fun, got to google's new and really cool correlation tool and type in the name of a client or one of their products. Without much effort you'll have a list of totally unrelated search terms that correlate fairly well...allowing all kinds of silly conclusions (note that google is smart enough to warn users in their comic book user manual to remember that correlation is not causation.

In some states you need a license to be a florist or interior designer (among other things).  The justification is that these jobs pose a danger to the public. I'd love to see a license to quote statistics, after all, we could get our clients in a lot of trouble if we misused numbers as noted above so clearly quoting statistics is far more dangerous than a dozen roses ever could be.

Sadly, I don't see our elected officials giving up their best trick that easily. So we'll have to settle for being active citizens spreading our understanding of numbers. Whether that is by alerting the media, blogging, commenting or whatever...every little bit helps.

President, TRC


Rich brings a passion for quantitative data and the use of choice to understand consumer behavior to his blog entries. His unique perspective has allowed him to muse on subjects as far afield as Dinosaurs and advanced technology with insight into what each can teach us about doing better research.  

Comments

  • No comments made yet. Be the first to submit a comment

Leave your comment

Guest
Guest Thursday, 19 September 2019

Want to know more?

Give us a few details so we can discuss possible solutions.

Please provide your Name.
Please provide a valid Email.
Please provide your Phone.
Please provide your Comments.
Enter code below : Enter code below :
Please Enter Correct Captcha code
Our Phone Number is 1-800-275-2827
 Find TRC on facebook  Follow us on twitter  Find TRC on LinkedIn

Our Clients