There’s a goodly amount of good advice in the book What the Numbers Say: A Field Guide to Mastering Our Numerical World by Derrick Niederman and David Boyum (2003, Broadway Books). The quantity of this advice is sizable, considerable, hefty, copious, substantial, and voluminous. It’s a lot! But I’m more impressed qualitatively. Their message is germane to the points I have tried to make in this blog.
The first chapter is entitled “Ten Habits of Highly Effective Quantitative Thinkers.” Don’t hold the title against the authors, though. Their book is written primarily for the business field where an upbeat approach is expected protocol. Their choice of this title was (I hope) subterfuge since their goal is to convince people to prefer substance over form.
Here are the first three of their recommended practices for using quantitative information effectively:
1. Only Trust Numbers… If you want to be a good quantitative thinker, you must learn to make decisions on the basis of [evidence, which typically takes the form of] numerical information, even when that information conflicts with your instincts and perceptions… Try to raise your level of trust in careful quantitative analysis, and reduce your confidence in hunches, theories, and casual observations.” p. 6-7
2. Never Trust Numbers… Before we reconcile our apparently inconsistent advice, first let us explain why numbers are not worthy of your trust: It’s because numbers can be wrong, are frequently misleading, and all too often have an agenda… Even when accurate, numbers can easily mislead. Quantitative data are seductive; they invite us to engage in the risky behavior of reading more into data than is warranted.” p. 9-10
Tell yourself this: Every number I see is generated and presented by people who have an interest in how that number is used or interpreted. You have our permission to suspend this presumption if you’re looking at the periodic table [of elements]…but any other suspensions are taken at your own risk… You might think distrusting numbers is easy, but it’s not. Distrusting numbers is not the same as disregarding them.”
Shortly before a scheduled trip to the Netherlands, Clinton administration drug czar Gen. Barry McCaffrey called Dutch drug policy ‘an unmitigated disaster’ and backed up that characterization by asserting that ‘the murder rate in Holland is double that in the United States.’ In fact, the Dutch homicide rate is less than a quarter of the American rate… Evidentally, [McCaffrey’s] staff relied on cross-national crime statistics published by Interpol, data in which the Dutch homicide rate included attempted murders… A McCaffrey spokesman tried to pass the buck by saying ‘We have said if we are wrong, speak to Interpol—it’s [sic] not our statistics, it’s (their) reporting.’ Never mind that the relevant Interpol publication prefaces the stats with the following caveat:
- The data gathered in these sets of statistics is not intended to be used as a basis for comparisons between different countries since the statistics cannot take account of differences which exist between definitions of punishable acts in different national laws, or the diversity of statistical methods…” p. 11-12
3. Play Jeopardy… In case you’re unfamiliar with the TV game show Jeopardy, here’s the key rule: All answers must be phrased as questions.” The authors then quote Mark Kleiman, professor of policy studies at UCLA: ” ‘A number only gets to be useful when considered as the answer to a question. To be a good consumer of numbers, the reader must constantly ask himself:
To what question is the number (supposed to be) the answer?
Is it the correct answer to that question?
Is that the question to which I need an answer?’ ” p. 13-14
Questioning quantitative data, reports, and presentations can seem like nay-saying. But nay-saying is getting a bad rap these days due to the distasteful arena of politics. The type of nay-saying Niederman and Boyum recommend is different. The purpose is to see whether a quantitative argument can stand up to counter-arguments concerning its accuracy and relevance. If it holds up well to these, the argument is strengthened. If it cannot, then the argument needs re-worked.