Library Data: Empowering Practice and Persuasion

Library Data

Empowering Practice and Persuasion

Darby Orcutt, Editor

LIBRARIES UNLIMITED

An Imprint of ABC-CLIO, LLC

Copyright 2010 by Libraries Unlimited

All rights reserved. No part of this publication may be reproduced, stored in a

retrieval system, or transmitted, in any form or by any means, electronic, mechanical,

photocopying, recording, or otherwise, except for the inclusion of brief quotations in a

review, without prior permission in writing from the publisher.

Library of Congress Cataloging-in-Publication Data

Library data : empowering practice and persuasion / Darby Orcutt, editor.

p. cm.

Includes bibliographical references and index.

ISBN 978–1–59158–826–9 (acid-free paper) — ISBN 978–1–59158–827–6 (ebook)

1. Library statistics. 2. Library statistics—United States. 3. Academic libraries—Statistics. I. Orcutt, Darby.

Z669.8L54 2010

025.007´23—dc22 2009039781

14 13 12 11 10 1 2 3 4 5

This book is also available on the World Wide Web as an eBook.

Visit www.abc-clio.com for details.

ABC-CLIO, LLC

130 Cremona Drive, P.O. Box 1911

Santa Barbara, California 93116-1911

This book is printed on acid-free paper

Manufactured in the United States of America

For my son, Cameron,

my love for whom cannot be quantified.

Contents

Introduction ix

Part I: Approaching Data 1

Chapter 1: Yielding to Persuasion: Library Data’s Hazardous Surfaces

Jamene Brooks-Kieffer

3

Chapter 2: Using Educational Data to Inform and Persuade

Kate Zoellner

17

Chapter 3: Telling Your Library’s Story: How to Make the Most of Your Data in a Presentation

Anne C. Elguindi and Bill Mayer

25

Part II: Evaluation of Monographic Collections 35

Chapter 4: Collection Evaluation: Selecting the Right Tools and Methods for Your Library

Lucy Eleonore Lyons

37

Chapter 5: Comparing Approval and Librarian-Selected Monographs: An Analysis of Use

Erin L. Ellis, Nikhat J. Ghouse, Monica Claassen-Wilson, John M. Stratton,

and Susanne K. Clement

53

Part III: Serials and E-Resources Management 69

Chapter 6: E-Journal Usage Statistics in Collection Management Decisions: A Literature Review

Andre´e J. Rathemacher

71

Chapter 7: Perspectives on Using E-Journal Usage Statistics in a Serials Cancellation Project

Andre´e J. Rathemacher and Michael C. Vocino

91

Chapter 8: Using ‘‘Meta-Analysis’’ in Electronic Resource Decision-Making

Tracie J. Ballock, Carmel Yurochko, and David A. Nolfi

103

Chapter 9: Usage Statistics: Resources and Tools

Margaret Hogarth

129

Part IV: Reference and Instruction 149

Chapter 10: Moving Beyond the Hash Mark: Capturing the Whole Reference Transaction for

Effective Decision-Making

Danielle Theiss-White, Jason Coleman, and Kristin Whitehair

151

Chapter 11: Maximizing the Value of Reference Data: A Case Study

Erika Bennett, Sommer Berg, and Erin Brothen

183

Chapter 12: Instruction by the Numbers: Using Data to Improve Teaching and Learning

Wendy Holliday, Erin Davis, and Pamela Martin

197

Part V: Specific Methods and Issues 215

Chapter 13: If the Library Were a Business, Would It Be Profitable? How Businesses Go

Beyond Numbers

Michael A. Crumpton

217

Chapter 14: The Use of Grounded Theory in Interlibrary Loan Research: Compliance

Always Occurs

David E. Woolwine and Joseph A. Williams

227

Chapter 15: Investing in Electronic Resources Using Capital Budgeting

Timothy M. McGeary

237

Part VI: Emerging Contexts 261

Chapter 16: Data for Repositories: Making the Case and Integrating into Practice

Hilary Davis

263

Chapter 17: How Library Homepage Vocabulary Influences Database Usage: Use of

Vendor-Provided Usage Data for Homepage Design

Melissa Johnson

271

Chapter 18: NUC Accreditation Data and Evidence-Based Library Management in Nigerian

University Libraries

Samuel C. Utulu

281

Index 297

viii Contents

Introduction

DARBY ORCUTT

If you’re actually reading this introduction, not only does it mean that you’ve opened this book on

libraries and data, but it also means that you’re interested in context. You may be curious what

prompted the creation of this book, what drove the selection and organization of its contents, or how

it all fits together. Strong data analysis begins with similar questions. In fact, data without context

ceases to be data. Until units of information are brought together in such a way that they can be manip￾ulated, interpreted, and used, they are not truly data.

The germ of this book—and the need for it—was made clear while I participated on the Association

of College & Research Libraries’s Educational & Behavioral Sciences Section’s 2007 Program Plan￾ning Committee, chaired by Penny Beile, and I wish to thank her and the members of that committee

for their encouragement and support. The resulting 2007 panel presentation stands as just one of a

great many data-focused programs ubiquitously present at virtually all library conferences in recent

years. In addition, I have met and corresponded with a growing number of professionals having new

and major data responsibilities if not profiles, such as my own former position of Collection Manager

for Data Analysis which I originated at The North Carolina State University Libraries (the second

librarian in that position, my colleague Hilary Davis, is a contributor to this volume). Despite over￾whelming interest and need, no publication thus far has sought to broadly and thoroughly address

the obvious gap in the professional literature.

Especially in the digital age, libraries have grown more and more capable at generating data. Con￾currently, the library profession has increasingly valued data as a—or even the—key component in

decision-making. Whether we librarians will prove so adept at understanding and using data is still

an open question, a positive future answer to which this collection will hopefully influence.

Given also the recent economic crisis of historic proportion, this book suddenly seems of even more

immediate interest and value than its contributors may have realized at the time of writing. The strat￾egies herein should prove useful in times when every decision within, and every persuasive argument

about, the library could have especially lasting and defining consequences. The subtitle of this volume

might appropriately be ‘‘Hard Data for Hard Times.’’

This book draws together research, theory, pragmatic reflections, honest assessments, and new

ideas from a range of librarians: mostly academic and mostly American, but addressing issues as well

as sharing ideas and practices that could be valuably applied within a wide variety of contexts. I have

purposefully solicited contributions not only from seasoned researchers and senior administrators, but

also from new and upcoming voices in the field—including many authors for whom this is their first

publication.

‘‘Data’’ is frequently equated in library circles with ‘‘quantitative information,’’ and I have asked

contributors to hold to this terminological convention wherever possible. Yet, in many instances it is

necessary to challenge this potentially dangerous terministic screen, and I am pleased to open this vol￾ume with Jamene Brooks-Kieffer’s excellent essay that does just that, problematizing this simplistic

correlation. Comprising the remainder of Part I, ‘‘Approaching Data,’’ Kate Zoellner’s survey of sour￾ces for educational data will prove a handy reference and Anne C. Elguindi and Bill Mayer’s tips for

persuasive presentation offer practical guidance for all librarians.

Part II, ‘‘Evaluation of Monographic Collections,’’ offers Lucy Eleonore Lyons’s useful new matrix

for planning evaluation in any context, as well as the results of a detailed use-study of approval versus

librarian-selected titles at K-State, coauthored by Erin L. Ellis, Nikhat J. Ghouse, Monica Claassen￾Wilson, John M. Stratton, and Susanne K. Clement. Turning to serials and e-resources, Part III opens

with a thorough review of the literature on usage statistics in e-journal decision-making by Andre´e J.

Rathemacher, who next teams with colleague Michael C. Vocino to discuss the application of theory

to a real-life serials cancellation project. Tracie J. Ballock, Carmel Yurochko, and David A. Nolfi

articulate a theoretical yet practical guide to electronic resource decision-making, and Margaret

Hogarth outlines sources, perspectives, and problems in collecting and understanding usage data.

Part IV casts a light on ‘‘Reference and Instruction.’’ Danielle Theiss-White, Jason Coleman, and

Kristin Whitehair detail an elegant system for capturing and interpreting transaction data from multi￾ple reference points, while Erika Bennett, Sommer Berg, and Erin Brothen’s similar approach within

an online-only library complements and reinforces its usefulness. Wendy Holliday, Erin Davis, and

Pamela Martin’s system for outcomes-based assessment of instruction likewise offers a full and prac￾ticable way of measuring success in this aspect of library service.

Part V focuses on ‘‘New Methods’’ with Michael A. Crumpton’s business-style perspective on data

in libraries, David E. Woolwine’s and Joseph A. Williams’s fruitful application of sociology’s

‘‘grounded theory’’ to interlibrary loan practices, and Tim M. McGeary’s capital budgeting model

for electronic resources. Lastly, Part VI, ‘‘Emerging Contexts’’ takes us into new library directions.

Hilary Davis explores the use of data in developing and managing an institutional repository, Melissa

Johnson outlines a project of using vendor-provided usage data to drive library home page design, and

Samuel C. Utulu describes the increasingly important uses of data within the context of Nigerian uni￾versity libraries.

My hope is that this volume will not be the last of its kind but rather will provoke further ideas and

discussion of tools, methods, perspectives, and results of data analyses in library contexts. Data is the

future of libraries; let’s work together to shape that future.

x Introduction

PART I

Approaching Data

CHAPTER 1

Yielding to Persuasion: Library Data’s

Hazardous Surfaces

JAMENE BROOKS-KIEFFER

If I was wrong in yielding to persuasion once, remember that it was to persuasion exerted on the side

of safety, not of risk.

—Jane Austen, Persuasion

INTRODUCTION

Anne Elliot, the sober and sedate heroine of Jane Austen’s Persuasion, speaks at last to her longtime

love Captain Wentworth as a woman whose reactions to acts of persuasion throughout her life have

not left her entirely happy. Curiously, she defends the judgment of her persuasive friends rather than

her own. She argues that Lady Russell’s persuasive powers were intended to keep Anne safe at a time

when Anne’s own immature convictions led her into a risky engagement with the then younger and

poorer Wentworth. Seven years later, Captain Wentworth’s second offer of marriage, fortune, and con￾sequence seems sensible rather than uncertain. Anne’s years of loneliness have taught her that not all

risk is dangerous, just as not all safety is comfortable. In the context of Austen’s other novels this is a

relatively radical conclusion. Dashing, risky suitors reward Austen’s heroines with happiness and

stability much less often than do ordinary, familiar, even offensive men. But in the unrelated context

of libraries and their data, what can Anne Elliot teach us?

From Persuasion, we librarians may glean something from Anne’s point of view. The blame or credit

for an unhappy or a fortunate outcome does not lie with the persuader. The object of persuasion controls

her own fate because she chooses to be convinced, or not, by the persuader’s argument. Neither risk nor

safety guarantees a happy ending. We librarians are often the objects of persuasion, exhorted by users,

vendors, administrators, stakeholders, and others to do this, buy that, and produce thus-and-such. When

we attempt to take on the role of persuader ourselves, we often restrict our persuasive powers by using

inferior tools. The traditional ways in which librarians gather and process data often stop short of the

analysis, processing, or mining techniques that would be considered a necessity in any other profession

as data-rich as ours. Such techniques are not easy to employ but they produce remarkably informative,

if at times uncomfortable, results. Instead, we prefer to deal with the surface, safe meaning of our data,

relying on a predominance of quantitative variables and the simple, arithmetic conclusions we can draw

from them. These conclusions are safe because they seldom yield unexpected results. For example, safe

conclusions often occur during the serials renewal process. A subject librarian totals a long list of

numeric usage data, assumes that a large total equals frequent use, and justifies a physics journal’s

renewal. Even if a thorough data analysis supports the opposite decision, cancelling the journal could

jeopardize the librarian’s relationship with the physics department.

ARGUMENT

This chapter is concerned with the data that libraries produce and analyze internally. A library’s

internal data does not reside in a vacuum; it is made relevant by the library’s activities of evaluation

and assessment. I do not intend to thoroughly investigate evaluation and assessment activities but to

show that these pursuits succeed when they study carefully analyzed internal data. Many in-depth

examinations of libraries’ evaluation practices, assessment techniques, and cross-institutional data

may be found in the current literature. While cross-institutional data is an important area of inquiry,

I will only examine how it influences the internal collection of data.

Internal data gathered by libraries has long offered a wealth of information about interactions

between a library and its users. Librarians who lay claim to this data attempt to inform their work of

acquisition, deaccession, collection management, accessibility, and a host of other duties. Libraries’

internal data often includes browse and checkout numbers (from the catalog); types and numbers of

transactions at service desks; funds spent or available (from the integrated library system or ILS);

requests and clickthroughs (from a link resolver or federated search tool); search and access logs

and full-text downloads (from e-resource vendors); analysis of Web page activity; and logs of proxy

server transactions. The acquisition and management of serial and electronic resource collections

has contributed to the recent explosion in available data. Since many libraries spend a majority of their

acquisitions budgets on these items, it comes as no surprise that data about serials and electronic

resources is plentiful to the point of excess.

This plentiful data is often discussed at conferences and in the literature as a problem. Its very abun￾dance creates compatibility and time management difficulties. Its quantity hinders a healthy question￾and-answer relationship between libraries and their data. For example, librarians can often access

usage data on a given journal title from at least three different sources: the ILS, the link resolver,

and one or more e-resource vendors. When making journal renewal decisions, a librarian might

choose to ask only one of these data sources, ‘‘Was this journal used?’’ rather than attempt to assemble

one valid data pool from three distinct sets. Among librarians, the core problem seems to be extracting

meaning from huge stores of internal records.

Many libraries’ questions cannot be answered reliably by data from a single source:

• What are the consequences of changing library hours or other services?

• What resources and services do users value most highly?

• How do our users really use the library?

Only multiple sources of data can supply trustworthy answers to these and other questions. Transform￾ing too-plentiful data from various sources into a truly valuable information source requires careful

manipulation and analysis.

Unfortunately, data analysis is an area where librarians and libraries often perform poorly. Several

pieces of current and historical literature support this assertion. Blake and Schleper claim that librari￾ans ‘‘do not always use the information that we gather, preferring instead to point to the numbers

themselves as evidence of our work’’ (2004, 460). Moroney (1956; cited in Allen 1985, 211) says,

‘‘there is something very sad in the disparity between our passion for figures and our ability to make

4 Library Data

use of them once they are in our hands.’’ Allen’s own lament (1985, 211), echoed by later authors

(Ambrozˇicˇ 2003; Hiller and Self 2004), is fairly damning: ‘‘Undoubtedly librarians are great compil￾ers of statistical data, but exhibit poor abilities in its interpretation, manipulation or use.’’ Each of

these authors implies a slightly different meaning when invoking the word use in the dual contexts

of librarians and data, but all are differentiating between a reliance on facts themselves and any scru￾tiny carried out on the collected information. These opinions span roughly 50 years of observing the

profession and contain the same elemental criticism. Librarians’ relationship with data is dysfunc￾tional at best and pathologic at worst. We have far too much experience with and affinity for the safe

activity of data gathering, and far too little experience with the risks of using it. In this chapter, using

data means analyzing multiple inputs and feeding the results into the library’s processes of decision￾making, management, evaluation, and assessment.

Why do we have such difficulty with data analysis? A cynic might answer by pointing out other

shaky foundations upon which libraries often base decisions. Some libraries stand firm in institutional

tradition and when their methods are questioned, say, ‘‘We’ve always done it this way.’’ Other libraries

are quick to buy in to the newest heavily marketed trend and, when a topic of concern arises, say

‘‘Everyone else is doing it.’’ Neither of these is a reason good enough for neglecting or foregoing data

analysis. A different critic might fault organizations that supply their employees with ample tools for

acquiring and amassing data but neglect to train them or allow them enough time to conduct data

analysis. My own opinion is that librarians are confounded by data analysis because they assume that

the common use of the word statistics is equivalent to the processes, skills, and outcomes associated

with the discipline of statistics. This particular misunderstanding creates a path paved with inaccura￾cies and assumptions about data. Librarians happily trod this path, believing we are on the way to

informed decision-making when we are really walking in circles.

In this chapter, I will examine libraries’ bias in favor of quantitative data and clarify the relation￾ships among quantitative and qualitative data, the popular definition of ‘‘statistics,’’ and the discipline

of statistics. I will also study the errors and assumptions made by librarians in their dealings with data.

I will consider three models of data-supported persuasion and explore the misnomer that is data-driven

decision-making. Finally, I will observe four risk-taking organizations that push various kinds of data

through analysis, recommendation, and action.

DATA GOES BEYOND NUMBERS

Our discussion of data encompasses both quantitative and qualitative measures. Sullivan (2007)

defines quantitative as giving ‘‘numerical measures’’ (6), explaining that arithmetic performed on

these measures will produce meaningful results. Even though a phone number is numeric data, arith￾metic performed on the numbers does not mean anything. On the other hand, qualitative ‘‘allows for

classification ... based on some attribute or characteristic’’ (6). The same phone number that is arith￾metically nonsensical can be a meaningful trait in a qualitative context (7). Although both measures

are useful, libraries seem to have a decided preference for quantitative data.

Libraries occasionally dip a toe in the waters of qualitative measurement by, for example, sending

out surveys, organizing focus groups, participating in LibQUAL+, and so forth. However, the most

commonly gathered data in a library is quantitative: gate counts; quantity of reference questions; num￾ber of circulated items; or amount of database usage. One reason for this biased practice is found by

examining the measurements gathered by library-related agencies. The Association of Research

Libraries (ARL) Statistics Interactive Edition lists 18 measures of expenditures, 15 of holdings, 6 of

activities, and 5 of staffing, among others. The annually gathered Association of College & Research

Libraries (ACRL) Statistics include 8 measures of print collections, 12 of expenditures, and 18 of

electronic resource collections and usage. The National Center for Education Statistics (NCES) data

Yielding to Persuasion 5

on libraries gathered between 1976 and 2004 include 5 measures of collections, 7 of staffing, 1 of

activities, and 26 of expenditures. Almost all of these measures are counts, totals, differences, or other

arithmetic results. These agencies’ emphasis on quantitative data sends an unsubtle message to libra￾ries that it is the only significant kind.

The other glaring reason for quantitative data’s takeover of the library is the sheer volume of

numeric data our systems are capable of producing. In an age dominated by manual data collection

methods, Allen wrote that decision-making was hampered by the slow pace of data extraction. He pre￾dicted that bringing computer systems into libraries would remove ‘‘physical obstacles to the

assembly and digestion of data based on unit operations’’ (1985, 213). In one way, he was correct.

Computerized, automated data collection has given nearly every library the ability to become a data

warehouse. We now have the opposite problem, one of tonnage. White and Kamal point out that

‘‘managers are often overwhelmed with statistical numbers because digital technologies can be so

efficient at capturing data’’ (2006, 14).

Gorman thinks about this onslaught of numeric data in the context of library stakeholders who have

little time and short attention spans. He points out that these people are a primary audience for library

data and a major source of library funding. Stakeholders, he says, prefer simple proof that X is bigger

(and therefore better) than Y. When such stakeholders control a library’s funding, the library may be

forgiven for focusing on this data to the exclusion of all other types. Gorman also hypothesizes that

the ease with which computing systems supply numeric data makes librarians more and more depen￾dent on that data as the only means of evaluation. He points out that stakeholders’ perception of com￾puterized, numeric data as simple and quick to gather makes them more demanding of numbers as

proof of value. Stakeholders assume that computerized methods of gathering and supplying quantita￾tive data are accurate and easy for librarians to supply. Librarians assume that such data is efficient

and effective at communicating progress to stakeholders. Such circumstances create a perfect storm

of library preference for quantitative data, making it our default choice for empirical information

(1999, under ‘‘The Problem with the Stakeholders ... ’’).

DATA IS NOT STATISTICS

Having established the dominance of quantitative data in libraries, we must now consider librari￾ans’ equivalent treatment of data, ‘‘statistics,’’ and statistics. This treatment first caught my attention

during a forum sponsored by the National Information Standards Organization (NISO). During his

talk, Bollen (2007) pointed out that usage data and usage statistics are not identical concepts. He char￾acterized usage data as raw usage events and usage statistics as reports about those events. Even

though Bollen defined these terms in the context of the MESUR project (Los Alamos National Labo￾ratory), his very specific descriptions apply to many of the ways librarians speak about and work with

data and statistics. Librarians’ work with these topics is not limited to usage, but also incorporates

collections, expenditures, and services—in short, any work of the library.

The discipline of statistics seems to take for granted the conceptual difference between the words

statistics and data. Tietjen introduces statistics to the novice as ‘‘the accepted method of summarizing

or describing data and then drawing inferences from the summary measures’’ (1986, 1). A recent

introductory statistics textbook defines the topic as ‘‘the science of collecting, organizing, summariz￾ing, and analyzing information to draw conclusions or answer questions’’ (Sullivan 2007, 3). This

same author determines that facts describing characteristics of an individual are data (3). Both authors

speak of statistics as a process that acts on all forms of data. This characterization supports Bollen’s

dividing line between data (raw events) and statistics (an analysis of those events). This professional

treatment ignores the popular definition of ‘‘statistics’’ as a collection of numbers (e.g., the ‘‘stats’’

on a favorite ballplayer). This common usage echoes the definition of quantitative data, and it is

6 Library Data