How to Lie with Statistics – Part 4: Much Ado About Practically Nothing

Continuing with my previous post, here I summarize Chapter 4 of the book How to Lie with Statistics. This chapter is titled, Much Ado About Practically Nothing.

This brief chapter explains the critical importance of understanding how representative is a sample metric of the population as a whole. We can actually calculate that answer. First, we must understand that there are two figures that can be used to give us a sense of that representation: the probable error and the standard error. My summary will focus on the Standard Error because it is the most commonly used measure these days.

The Standard Error is used to construct confidence intervals, whereby if the calculation is sound then the true value would be contained within those intervals. This means that there is some probability that the true value will in fact be outside those intervals (range of values). For example, you are 95% confidence that the true value is within some confidence interval. Said another way, there is a 5% chance that the true value is outside the range depicted by the confidence interval.

Also of importance is to recognize that the wider the intervals, the less confidence we have as relates to the estimated value. For instance, saying some value is within a range of 5 to 10 will carry greater weight if you are told some value is within 1 to 100. The latter may be more accurate, but precision is way off. This would be a polite way of saying, “take our results with a ‘grain of salt’”; or putting it another way, the results are close to meaningless.   

As you can imagine, rarely you see reported estimates this way. That is why it is important to go to the source documents, not just uncritically receive the headline media reports. Do not be fooled.