I want to communicate what I believe is the single most useful message about library assessment. This is not an announcement of a new data analytic technique or some all-purpose library value calculator. Nor is it advice on the importance of aligning work and measurement with vision and strategy, recognizing the political pitfalls of evaluation, or solidifying an annual assessment plan.
All of these are secondary to one fundamental step. But this step is a giant one: Libraries must become “self-evaluating organizations.” The importance of this dawned on me (again) when I heard a librarian describing how her library used customer surveys to rethink their service approach. I realized it was not their survey questionnaire nor the planned service changes that mattered. It was their whole mindset that made the difference. They had a willingness to be inquisitive and exploratory, to be logical and systematic, to question comfortable assumptions, to look for unexpected answers, and to act on what they learned.
The term “self-evaluating organization'” comes from a classic article by the late political scientist and pioneer in the field of public administration, Aaron Wildavsky.1 There are other more current and stylish terms for ideas in this same vein. Libraries must develop a culture of assessment, become learning organizations, pursue performance excellence and practice evidence-based decision-making.
But, these are all variations on a single basic theme: To do effective evaluation, libraries have to want to improve. They must seek out unbiased information about what needs done, how best to do it, how they are doing it, and what actual results their efforts produce. They must objectively examine their operations and accomplishments as well as any unintended consequences that might result from these. Libraries need to view their successes and failures—including fortunate circumstances and missed opportunities—impartially and non-defensively. Most of all, they must be willing to confront the errors of their ways and be prepared to change their operations based on evaluation results. (Sorry for the dramatic phrasing, but that is the crux of a learning organization. Detecting shortcomings and fixing them!)
To improve their performance libraries need fair and balanced assessments of that performance. Wildavsky took this even further:
The ideal organization…would continuously monitor its own activities so as to determine whether it was meeting its goals or even whether these goals should continue to prevail. [It] would have no vested interest in continuation of current activities.” 2
He understood that commitment to cherished beliefs is an impediment to evaluation. A thoughtful attendee at a recent PLA conference helped me understand this idea more clearly. She explained to me that librarians feel passionate about their ideas and pursuits. Pounding her fist on her chest, she said, “They speak from the heart out of commitment.” To take a dispassionate view of things would be out of character for this profession. Librarians want to be in the middle of the fray, getting things done and making a difference. (This explains some of the field’s exuberance for new technologies. It is oh-so-easy for a person with a hammer to see everything as a nail.)
How ironic it is that dedication might actually be an obstacle to effectiveness! Yet, it is true that subjectivity can make us victims of tunnel vision and hide our misconceptions from us. Stephen Jay Gould expressed this in his book, Full House:
The most erroneous stories are those we think we know best—and therefore never scrutinize or question.3
This is why Amos Lakos and Shelley Phipps preach that effective assessment requires significant change in a library’s organizational culture.4 Libraries need to be willing to critique everything they do. Without a zeal for self-evaluation we are not ready for assessment. Better to not waste time dabbling in something that we are not able to take seriously.
1 Wildavsky, A. (1972). The self-evaluating organization. Public Administration Review, 32(5), 509-520.
2 Wildavsky, A. p. 509. Red emphasis added.
3 Gould, S. J. (1996). Full house: The spread of excellence from Plato to Darwin, New York: Harmony Books, p. 56.
4 Lakos, A. and Phipps, S. (2004). Creating a culture of assessment: A catalyst for organizational change. portal: Libraries and the Academy, 4(3), 345-361.