I noticed that yet another library value calculator has appeared on the scene. This one is offered by the National Network of Libraries of Medicine (NNLM) with the very best of intentions, I am sure. But, let me say that I am convinced that these calculators are a bad idea. Their underlying assumptions are weak and their designs are not well thought out. Eventually, library funders and stakeholders are going to realize that the calculations are superficial and…well…sloppy.
For one thing, sound cost-benefit analysis requires an examination of the full extent of relevant costs and benefits of a given project, program, or service. These quick-and-easy library calculators, however, use average retail prices as proxies for benefits. This oversimplification ignores important sources of library value like contributions to student and life-long learning, scientific and academic research, and public discourse, as well as roles libraries play in imparting cultural and humanitarian values and traditions, promoting literary appreciation and aesthetic values, facilitating community cohesion, and so forth.
But say that, for practical purposes, we accept the idea that value-boils-down-to-price as reasonable. Even so, the retail pricing approach these calculators use has definite problems. The calculators view retail prices as estimates of costs that patrons would incur if the library’s items and services were—hypothetically—unavailable to the community or institution. The library comes up with a retail price for each type of material and service it offers, and then these prices are translated directly into the value patrons receive from utilizing these materials or services.
In many cases, however, the alternative to obtaining an item or service from the library is not an outright purchase at retail prices. A student might purchase a textbook for $125 and then later re-sell it on Amazon.com for $50. Or perhaps she buys the item at a used price or borrows it from a friend for free. Clearly, a variety of alternative patron scenarios are possible, meaning that there is a range of alternative costs (approximate values) associated with each item or service use. The average of these ranges will typically be less than an item’s retail price. Besides, an item borrowed from a library does not include the breadth of rights and conveniences that item ownership does. So, it is a stretch to say that a patron always enjoys the same benefit from a borrowed item as from a purchased one.
Other problems with the calculators make their output suspect. For example, each time a patron renews an item or re-uses it in-house or online, the item’s retail price gets credited—again—to the library’s value totals. (Cha-ching!) On the other hand, when our Amazon.com shopper purchases a book at $75, that book’s value does not increase to $150, then to $225 and beyond each time the owner opens the book, or with each 3-week library loan period that passes.
Because the calculators tally only certain types of transactions, they end up painting a rather rosy picture of library performance. Consider the case of a patron who needs an item or service that is (really) not available from the library, and whose information need ultimately goes unmet. And the case of a service delivered that fails to meet a patron’s need, such as an unproductive reference consultation. The first case won’t be tallied at all by these calculators, and the second case will be tallied but will be significantly over-valued. (It will be considered a complete success.) Yet, the actual value of both of these patron transactions is negative and should be entered into these calculators this way. Unfortunately, the calculators’ designs do not accommodate this.
Given these problems and oversights, it is fairly obvious that these calculators produce exaggerated estimates of the benefits which libraries provide. Perhaps this exaggeration is only moderate or perhaps it is substantial—we cannot really tell for sure.
The calculators also underestimate the cost side of the equations, causing their benefit/cost ratios to be even more over-stated. They ignore several key costs incurred in delivering library materials and services, including expenses for information technology, equipment, building maintenance, utilities, and administrative overhead. These calculators also disregard the incidental costs that patrons may bear, like travel and parking costs, time lost due to item unavailability or poor service, usability difficulties encountered, and so on. In fact, NNLM’s calculator errors in the opposite direction: Assuming that libraries are always convenient, the calculator builds a patron time-savings factor into its formula. (I suppose you could enter in negative numbers to register patron lost time and inconvenience.)
When the calculators do recognize costs, they end up settling for data that are the grossest of estimates. For instance, users can enter estimated percent of total library staff time spent supporting access to materials or services. Creators of the calculators seem unaware that accurate benefit/cost ratios require meticulous collection of operational data, not just convenient guess-timates.
You will be hard-pressed to learn about these shortcomings from the materials that accompany library value calculators. Mostly, libraries receive general guidelines for entering data and encouragement to use the calculators without reservation. The library just keys in its data and—voilà!—receives an exact return-on-investment percentage or benefit/cost ratio right on the spot! Given the casual assumptions the calculations entail and the inexactness of the library’s input data, you’d think the final answer would at least include some type of margin-of-error disclaimer. Maybe something like this:
Your library’s benefit-to-cost ratio = $8.20 per $1.00 cost*
* Based on our calculations, we are 95% confident that your library’s benefit/cost ratio is between $4.50 and $12.50 (per $1.00 cost). If your data are especially inaccurate, this range will be larger. Note that our single $8.20 estimate may be high due to assumptions our model uses.
Needless to say, this kind of small print doesn’t appear in the instructions that come with library value calculators. As they are, the calculators generate figures that are precise to the penny, with no other explanations to speak of. Libraries confidently report the figures to stakeholders as accurate, authoritative, and nearly approaching Scientific Truth. Of course, the figures are nothing of the sort.
Clones of these library calculators have sprouted up on dozens of library websites, where patrons are invited to enter their custom data to receive their own monthly “value of library services.” Costs are typically not mentioned, so that final value calculations are simple multiplications of counts times arbitrary and often fanciful retail price estimates. Of course, the nifty and optimistic totals will delight library patrons. The totals might even please the population of nonusers who are happy to subsidize library use by others as an overall benefit to the community or institution.
On a few public library websites the calculations are made even more tantalizing by informing patrons about their “individual return-on-investment”—how much value they gain for every tax dollar they contribute. (Don’t you just love democracy!) Unfortunately, this approach casts the wrong light on the public value of libraries. First, the figures are further exaggerations because they use per capita revenue data. Not every public library user pays taxes, a fact that makes the individually quoted return rates artificially high. (Instead of library tax revenue per capita, the calculations should use tax revenue per tax-paying household.)
Second, these seemingly benign “value” calculations actually hide information. The websites fail to provide overall return-on-investment rates for all tax-paying households or for all tuition-paying students. As I have already alluded, an individual patron’s rate of return is being subsidized by nonusers of library services. For every patron elated with his own personally-calculated rate, there will be several households or students whose return rates are less than $0, meaning they lose money on their library tax or tuition “investments.” (This mix of returns rates also applies to using the vanilla versions of the calculators that don’t bother to factor costs in.) Omitting this larger picture from these presentations is slanted and misleading—something that libraries should not be involved in.
From a public or institutional value perspective, these Library 2.0-inspired patron calculators completely sidestep the rightful purpose of library evaluation. This purpose is to assess the extent to which the library provides value to the institution or community as a whole, not how each individual fares. This assessment must also confirm that products and services are equitably distributed, that is, equally available and accessible to all who wish or need to use them (see Creating Public Value by Mark H. Moore).
In actuality, economic valuation is not so simple as it appears. It involves complicated (and frustrating) concepts like exchange value, use value, contingent value, and others. Even business corporations have misgivings about standard return-on-investment analysis because of how difficult it is to obtain reliable data to input into the formulas.
If we want to use purely monetary estimates of the value of our services, we need more rigorous methods than these makeshift library calculators. This exact advice was offered to us a couple of years ago by Donald Elliot and Glen and Leslie Holt in their book Measuring Your Library’s Value. Their work provides important guidance that we should be heeding. Like the fact that benefit/cost valuations are unique to the communities and institutions from whence they come. The figures are really not comparable across communities or for different libraries. This is something that most of us would not have thought about. The central message from their book, though, should already be obvious to us: We can’t just make these benefit/cost numbers up, the way these calculators do. There have to be sound theoretical and empirical bases for our findings.
Sure, quick-and-dirty estimations might be helpful in certain situations, as long as they are recognized for what they are. But the numbers gushing from these library calculators are nonsensical and disingenuous, in many cases. The whole idea has become an impediment to the real work of assessing library value. When the batteries in these little pocket library calculators wear out, I recommend that we just not replace them.