Lib(rary) Performance » Advocacy

The Path of Most Resistance

raylyons — Tue, 31 Jan 2012 16:04:21 +0000

The campaign to assess public library outcomes got a tremendous boost by Library Journal’s Director Summit held last month in Columbus, Ohio. It’s heartening to see library leaders getting serious about making outcome assessment integral to the management of U.S. public libraries! The excitement and determination are necessary for making progress on this front. And it sounds like the summit was designed to let folks absorb relevant ideas in ways that make them their own.

The onset of this newfound energy is the perfect time to commit ourselves to gaining a firm grasp on the core concepts and methods of outcome assessment. Although measurement of outcomes is a new undertaking for libraries, it has been around for a long time in other contexts. In fact, outcome evaluation approaches have been studied, debated, refined, and chronicled over the past forty-five years… [Read more]

The U Word

raylyons — Sun, 10 Apr 2011 00:50:54 +0000

This week Chase Bank sent an email to its customers saying that one of their vendor’s computer systems were hacked. The bank stated that they:

…are confident that the information that was retrieved [i.e., stolen] included some Chase customer e-mail addresses, but did not include any customer account or financial information. Based on everything we know, your accounts and financial information remain secure.

Confidence based on whatever they happen to know, eh? Because Chase could easily be mistaken, customers would be foolish to put their full trust in the bank’s assurances. I definitely plan to keep an eye on my Chase account for the next several months.

This same caution also applies to the most recent OCLC membership report, Perceptions of Libraries, 2010: Context and Community. The report’s energetic graphics and narrative make the information seem to be true. But, as my prior posts1 explain, surveys are always incomplete and imperfect. Findings from a single survey like OCLC’s are just not weighty enough to deserve our unconditional trust… [Read more...]

—————————

1 See Discussing Accuracy, Checking It Twice, Stranger Than Fiction, and Objects In Mirror Are Closer Than They Appear.

Stubborn Facts

raylyons — Sat, 12 Mar 2011 03:55:02 +0000

In the book John Adams author David McCullough writes about Adams’ legal defense of British soldiers on trial for murder in 1770. In his argument to the Massachusettes jury Adams said:

Facts are stubborn things. And whatever our wishes, our inclinations, or the dictums of our passions, they cannot alter the state of facts and evidence.1

Indisputable facts are difficult to ignore, indeed. Yet, facts are not always clear and unambiguous. Getting to the plain facts and drawing valid conclusions from them can be stubborn matters in their own right. To quote science teacher and YouTube lecturer, wonderingmind42, “Interpreting evidence well requires skill, training, and experience.”2 [Read more...]

—————————

1 In McCullough, D., 2001, John Adams, Simon & Schuster, p. 68. Red emphasis added.
2 Quote appears in the video at the 5:18 time mark. Also watch the segment from 2:40 to 4:20 about facts versus the interpretation of facts.

Non-Exponential Potential

raylyons — Tue, 15 Feb 2011 18:25:27 +0000

A new OCLC membership report, Perceptions of Libraries, 2010: Context and Community, is hot off the…er…PDF-Maker! The report is formatted more like a magazine than a study, with key findings summarized in a myriad of graphical illustrations. So, I must confess that I have rather neglected the narrative so far. But from browsing mostly through the pictures, I have come up with a few suggestions that might enhance the report’s message, quantitatively speaking.

First, it would be better if the OCLC market researchers avoided citing large percentages, like the 1,544% growth in e-Book sales (p. 11) and 1,050% growth in smart phone ownership (p. 15). As Derrick Niederman and David Boyum explain in their book, percentages like these tend to be overstatements due to the baseline figures used. And the percentages just aren’t that informative … [Read more...]

Discussing Accuracy

raylyons — Tue, 11 Jan 2011 22:07:29 +0000

Now that I am no longer distracted by the subject of last week’s entry I can get to the intended topic for my first 2011 blog entry. I should say, though, that I won’t be turning over any kind of new leaf for the new year. For now I’m sticking with the theme I’ve dwelt on already, a theme statistician Howard Wainer expressed concisely:

Whenever we discuss information we must also discuss its accuracy.1

Applied to the library world the message is: Formal library research and advocacy studies should always explain the strengths and limitations of their data. The best reason for abiding by this principle, in my opinion, is to allow readers to decide how much credence they want to give to conclusions drawn in the studies.

With new library advocacy studies on the horizon, I thought I better wrap up any unfinished topics from 2010. Howard Wainer’s advice made me think of the study, Opportunity for All: How the American Public Benefits from Internet Access at U.S. Libraries. Findings from this report have been broadcast widely with barely a mention of how approximate the figures are… [Read more]

—————————

1 Wainer, H., 2009, Picturing the Uncertain World: How to Understand, Communicate, and Control Uncertainty through Graphical Display, Princeton, NJ: Princeton University Press, p. 121.

Experience Keeps a Dear School

raylyons — Wed, 29 Sep 2010 03:09:19 +0000

Last June the final report from an IMLS-funded study of public library summer reading programs, The Dominican Study: Public Library Summer Programs Close the Reading Gap, was published. The “reading gap” refers to the cumulative loss in proficiency that has been observed in students who struggle with reading. The gap is cumulative because the “summer setbacks” that some students have add up, making them lag further and further behind good readers each year.

Researchers for the study, Susan Roman, Deborah Carran, and Carole Fiore, say that their main research question was, “Do public library summer reading programs impact student achievement?” The answer they delivered was, basically, “Yes.” Except the study does not actually demonstrate that the summer programs affect reading achievement at all… [Read more]

I Geek Information Accuracy

raylyons — Wed, 14 Jul 2010 23:04:20 +0000

The second phase of OCLC’s national library advocacy project debuted last year as the “GeekTheLibrary” campaign. The campaign is cool, chic, hip, flashy, geeky, and so on. Pretty ambitious to try to coin new slang! For sure I’m not a good judge of communication campaigns, but I wish them success on this one.

When it comes to data, though, these advocacy campaigns can be clueless. They really need to kick it up a notch, clean up their acts, circle back, and cut out the BS (bogus stuff). Which is to say they need to think more critically about the claims they make.* The results will be worth the effort. As it is, the GeekTheLibrary site repeats the same ol’ same ol’ I’ve talked about already. Is it really so painful to just tell it like it is? Like, tell the whole data story… [Read more]

Sugar High

raylyons — Tue, 08 Jun 2010 04:41:07 +0000

It’s very simple. Government agencies that issue distorted information in a time of crisis lose credibility and end up appearing incompetent. Even federal bureaucracies eventually learn this. Take the Gulf of Mexico oil disaster for example. Last week CBS News reported that the U.S. Department of the Interior misrepresented findings from the independent scientific assessments of the oil leakage rate. The Department’s press release said the scientists estimated leakage to fall within a range of 12,000 to 19,000 barrels of oil per day. Yet the scientists had submitted this range as an estimate only of the best case scenario (the lower bound of their overall estimate). They had yet to come up with the worst case estimates (the upper bound), an important fact omitted from the original press release. This led CBS News to accuse the government of “sugar coating” the estimates.

In the library world, however, this lesson has yet to be fully appreciated… [Read more]

Navigating with Fragmentary Information

raylyons — Wed, 17 Feb 2010 00:23:04 +0000

I have implied this in other entries in this blog, but I might as well say it outright: The library and information science profession needs to come to terms with the issue of standards for (i.e., rules of) evidence for performance, statistical, and advocacy research data. There, now I’ve said it.

I recently read the short and enjoyable book Graphic Discovery: A Trout in the Milk and Other Visual Adventures by statistician Howard Wainer (Princeton, NJ: Princeton University Press, 2005). The subtitle of the book comes from something Henry David Thoreau wrote. During a dairy strike in 1850 in New England people began to suspect that dairy owners were watering down the milk supply. This led Thoreau to write in his journal, “Sometimes circumstantial evidence can be quite convincing; like when you find a trout in the milk.” [Read more...]

Sawing with a Dull Saw

raylyons — Mon, 25 Jan 2010 18:35:42 +0000

In spite of their evolution over the last few decades, accelerated most recently due to the Googlization of information, public libraries have been amazingly impervious to change in the arena of performance measurement. I found the following observations about library measures in the early history of American libraries:

There is no branch of library economy more important, or so little understood by a librarian as helps to himself, as the daily statistics which he can preserve of the growth, loss, and use (both in extent and character) of the collection under his care. The librarian who watches these things closely, and records them, always understands what he is about, and what he accomplishes or fails to accomplish. The patrons to whom he present these statistics will comprehend better the machinery of the library, and be more indulgent toward its defects.1

Interesting that use of library statistics for advocacy purposes was recognized in 1876. [Read more...]

—————————

1 Warren, S. R. & Clark, S. N., Eds., 1876, Public Libraries in the United States of America, Washington, DC: U.S. Bureau of Education, p. 714.