Lib(rary) Performance » Measurement

E Pluribus Unum

raylyons — Wed, 25 Aug 2010 02:36:06 +0000

This simple statement is one of several “myths” appearing on GeekTheLibrary:

The busier the library, the more money it receives.

GeekTheLibrary is concerned that the general public mistakenly believes libraries are funded based on how much they are used by patrons. Perhaps their concern is well founded, I don’t know. But the statement happens also to be a great jumping off point for discussing ways to look at library data.

As I described in my prior post, scatterplots are graphical tools for exploring relationships between two characteristics of single things, such as heights and weights of children or educational levels and reading habits of library non-users. Inspired by the GeekTheLibrary statement, we can use these tools to help answer the question, “What is the relationship between U.S. public library visits and expenditures?” [Read more....]

Just Make Up The Numbers

raylyons — Wed, 12 May 2010 16:14:24 +0000

My prior post about the library research report, Opportunity for All, reminded me that as simple as percentages are, they sure can lead into bizarre territory. Suppose the report authors had determined that in 1990 an estimated 200 Internet terminals had been installed in U.S. public libraries nationwide. By 2007, then, the cumulative percentage growth in terminal installations would have been 103,900%! Though this percentage would be perfectly factual, it is practically meaningless. Fortunately, the authors didn’t extrapolate back to 1990.

Percentages require us to pay attention to what baselines are used and how reasonable these are. Derrick Niederman and David Boyum emphasize this in their book, What the Numbers Say: A Field Guide to Mastering Our Numerical World, in the chapter, ‘Playing the Percentages.’ [Read more...]

Skyrocket Science

raylyons — Mon, 19 Apr 2010 15:28:54 +0000

I am still on my excellence-in-graphical-data-presentation kick. Insufferably so, I am afraid. As my Feb. 16 post mentioned, the principles of high quality graphical data presentation have been articulated by William Cleveland, Edward Tufte, Howard Wainer and others. Good graphing practice is based on these three rules:

Be clear. (Strive for clarity. – William Cleveland)
Be fair and accurate. (Tell the truth about the data. – Edward Tufte)
Be thorough. (‘You can see a lot just by looking.’ – Howard Wainer quoting Yogi Berra)

When I got a copy of the new study conducted by the University of Washington, Opportunity for All: How the Public Benefits from Internet Access at U.S. Libraries,* I went straight to the pictures, of course! A chart from Chapter 2 (shown further on in this entry) seemed like a candidate for fine-tuning using the three rules. The chart is intended to illustrate a central theme of the study: In U.S. public libraries over the past decade (a) the delivery of public Internet access services has grown immensely, (b) utilization of public library services in general has grown healthily, though not as dramatically, and (c) overall library operating resources have increased only modestly.

But, first, I have an assignment for you… [Read more...]

———————————

* The study was funded jointly by IMLS and the Bill & Melinda Gates Foundation.

A-One-and-a-Two-and-a-Three

raylyons — Mon, 05 Apr 2010 13:21:49 +0000

There’s a goodly amount of good advice in the book What the Numbers Say: A Field Guide to Mastering Our Numerical World by Derrick Niederman and David Boyum (2003, Broadway Books). The quantity of this advice is sizable, considerable, hefty, copius, substantial, and voluminous. It’s a lot! But I’m more impressed qualitatively. Their message is germane to the points I have tried to make in this blog.

The first chapter is entitled “Ten Habits of Highly Effective Quantitative Thinkers.” Don’t hold the title against the authors, though. Their book is written primarily for the business field where an upbeat approach is expected protocol. Their choice of this title was (I hope) subterfuge since their goal is to convince people to prefer substance over form. [Read more...]

Navigating with Fragmentary Information

raylyons — Wed, 17 Feb 2010 00:23:04 +0000

I have implied this in other entries in this blog, but I might as well say it outright: The library and information science profession needs to come to terms with the issue of standards for (i.e., rules of) evidence for performance, statistical, and advocacy research data. There, now I’ve said it.

I recently read the short and enjoyable book Graphic Discovery: A Trout in the Milk and Other Visual Adventures by statistician Howard Wainer (Princeton, NJ: Princeton University Press, 2005). The subtitle of the book comes from something Henry David Thoreau wrote. During a dairy strike in 1850 in New England people began to suspect that dairy owners were watering down the milk supply. This led Thoreau to write in his journal, “Sometimes circumstantial evidence can be quite convincing; like when you find a trout in the milk” (quoted in Wainer, p. 81).

Wainer’s main point, one certainly made also by others like William Cleveland and Edward Tufte, is that well designed graphical representations are invaluable for exploring and understanding data. Graphical representation of data can lead to revelations about data and the underlying phenomena they describe that would otherwise be missed. [Read more...]

Sawing with a Dull Saw

raylyons — Mon, 25 Jan 2010 18:35:42 +0000

In spite of their evolution over the last few decades, accelerated most recently due to the Googlization of information, public libraries have been amazingly impervious to change in the arena of performance measurement. I found the following observations about library measures in the early history of American libraries:

There is no branch of library economy more important, or so little understood by a librarian as helps to himself, as the daily statistics which he can preserve of the growth, loss, and use (both in extent and character) of the collection under his care. The librarian who watches these things closely, and records them, always understands what he is about, and what he accomplishes or fails to accomplish. The patrons to whom he present these statistics will comprehend better the machinery of the library, and be more indulgent toward its defects. Public Libraries in the United States of America, Warren, S.R. And Clark, S. N., Eds., Washington, DC: U.S. Bureau of Education, 1876, p. 714.

Interesting that use of library statistics for advocacy purposes was recognized in 1876. [Read more...]

The Telephone Game

raylyons — Tue, 05 Jan 2010 13:34:47 +0000

Readers of the, say, older persuasion may recall a time when children actually enjoyed games that required no peripheral devices, infrared sensors, or satellite tracking. There was one party game, simply called (I think) “Telephone,” where one player whispered a message to the next, and that player to the next, until the message was passed all the way around the circle of players. The fun came when everyone heard the amusing distortions that ended up in the final message.

In library advocacy research, though, message distortion is not amusing. I noticed a serious instance of this in a recent IMLS Research Brief which cites an American Library Association (ALA) report finding that patron use of library computers for job-seeking purposes has “greatly increased.” [Read more...]

Thoroughly Modern Museums and Libraries

raylyons — Mon, 31 Aug 2009 22:18:54 +0000

I think I get it now. I had thought the term assessment meant a systematic and appropriately rigorous measurement of a construct or phenomenon of interest, like program outcomes, community needs, service quality, and so on. Only now have I come to understand that a self-assessment is a different animal altogether. Who would have thought that the purpose of a self-assessment is not really to assess anything? The purpose, I now realize, is to inform and educate. All this time I have been applying research methodology standards to tools that are intended to advocate and indoctrinate. No wonder my observations have been so off-base!

When I critiqued WebJunction’s online competencies assessment questionnaire (see my April 22, 2009 entry), the WebJunction staff explained to me that the true objective for their surveys was to increase awareness of these competencies. I immediately wondered, “Well, how then will WebJunction measure awareness?” But that is quite an irrelevant question when these ques- tionnaires are actually teaching tools, not measurement instruments. [Read more...]

Cha-Ching!

raylyons — Fri, 14 Aug 2009 16:34:22 +0000

I noticed that yet another library value calculator has appeared on the scene. This one is offered by the National Network of Libraries of Medicine (NNLM) with the very best of intentions, I am sure. But, let me say that I am convinced that these calculators are a bad idea. Their underlying assumptions are weak and their designs are not well thought out. Eventually, library funders and stakeholders are going to realize that the calculations are superficial and…well…sloppy.

For one thing, sound cost-benefit analysis requires an examination of the full extent of relevant costs and benefits of a given project, program, or service. These quick-and-easy library calculators, however, use average retail prices as proxies for benefits. This oversimplification ignores important sources of library value like… [Read more...]

Shorter

raylyons — Mon, 20 Jul 2009 05:56:24 +0000

You may not want to spend time reading this blog post. It’s rather long and drawn out and is likely to be dull. And it gets kind of complicated. Besides, the graphics are sparse and uninteresting. Plus there’s no video.

Instead, you might appreciate some other informational exper- ience better, one that happens also to be thoroughly cool and engaging. Like Facebook walls or those omnitemporal slice-of-life Twitter tweets.

This post definitely is not slice-of-life. Hardly. It is conceptual, meaning that it is mostly tedious and definitely time-consuming. It entails plod- ding through the text to see if any of the ideas make any sense. And even if they do, you have to figure out whether they are at all relevant. Worse, the topic could be one of those god-awfully amorphous ones that have no clear, calculable bottom lines—like conundrums or Zen Buddhist koans. [Read more...]