Numbers Don’t Lie… Or Do They?

As a risk manager, you rely on “numbers” for many aspects of your job. You assume that numbers tell the story. You might think that numbers don’t lie, but actually they can.

I have found over the years that numbers generated by trustworthy people and institutions don’t really lie by themselves, but that their interpretation can often be misleading. Let’s look at a couple of examples.

One of my favorite running backs had what looked like a great game recently. He averaged 7.6 yards per carry, while both the rest of his team and the opposing team averaged about 3.5. Way to go! Well, only if your measure of “average” is the mean (the arithmetic average). Had we looked at an alternative measure of central tendency such as the median (the middle-most score), we would have concluded the opposite: his median performance was only 1.6 yards per carry, but he did have one (lucky?) 70 yard play? Awesome day or terrible day?

The mean - median argument impacts on almost everything we look at in disability. Historically, we almost always look at the mean, but keep in mind when looking at, for example, disability durations or costs, that a few individuals at the extreme end of the distribution will have an outsized impact on the mean but not on the median (with medians, every “person” is a data point, whereas with means every disability day or every dollar is a data point). So, “average” disability durations and costs are invariably much higher when reporting means than medians. So, which is right? Both and neither, depending on your interest or your customer’s interest. I personally always like to look at both to get a fuller picture.

One other example. Do you ever use Uber or Lyft? If so, after each ride you are asked to rate your driver on a 5-point scale, and the driver also rates you. Have you ever seen any driver with a rating below 4.8? Drivers will practically beg you to give them a 5, crying that they would lose the job if they average (mean) less than around 4.7. So, what do we do? Unless they end up in an accident, most people will just give a 5 and hope that the driver rates them likewise, and everybody is happy. But what about the 5-point rating scale? It’s a fake, it’s actually a 2-point scale (acceptable / unacceptable), but using a 5-point scale allows the use of a number manipulation (calculating the mean) that gives an outsized impression of quality (my driver averages 4.95, he / she must be awesome!). The scale used to present numbers is important, and it can be misleading.

Sometimes the underlying scale changes over time– a bit like moving the goal posts, which makes comparisons between numbers questionable. Numbers can be very useful, but they can also be manipulated to mislead you. Always be skeptical, and make sure they tell you the real story. One useful strategy is to ask yourself: Is somebody trying to sell me something with these numbers? If so, is there some other way I could look at the underlying data?