Digital Library Use: Social Practice in Design and Evaluation

Social Practice in Design and Evaluation

edited by

Ann Peterson Bishop,

Nancy A. Van House, and

Barbara P. Buttenfield

Digital Library Use

Digital Library Use

Social Practice in Design and Evaluation

Edited by Ann Peterson Bishop, Nancy A. Van House,

and Barbara P. Buttenfield

Foreword by Bruce Schatz

The contributors to this volume view digital libraries

(DLs) from a social as well as technological perspec￾tive. They see DLs as sociotechnical systems, networks

of technology, information artifacts, and people and

practices interacting with the larger world of work and

society. As Bruce Schatz observes in his foreword, for

a digital library to be useful, the users, the documents,

and the information system must be in harmony.

The contributors begin by asking how we evaluate

DLs—how we can understand them in order to build

better DLs—but they move beyond these basic con￾cerns to explore how DLs make a difference in people’s

lives and their social worlds, and what studying DLs

might tell us about information, knowledge, and social

and cognitive processes. The chapters, using both

empirical and analytical methods, examine the social

impact of DLs and also the web of social and material

relations in which DLs are embedded; these far￾ranging social worlds include such disparate groups

as community activists, environmental researchers,

middle-school children, and computer system

designers.

Topics considered include documents and society;

the real boundaries of a “library without walls”; the

ecologies of digital libraries; usability and evaluation;

information and institutional change; transparency as

a product of the convergence of social practices and

information artifacts; and collaborative knowledge

construction in digital libraries.

Ann Peterson Bishop is Associate Professor,

Graduate School of Library and Information Science,

University of Illinois, Urbana–Champaign. Nancy A.

Van House is Professor, School of Information

Management and Systems, University of California,

Berkeley. Barbara P. Buttenfield is Professor,

Department of Geography, University of Colorado,

Boulder.

Digital Libraries and Electronic Publishing series

Contributors

Philip E. Agre, Imani Bazzell, Ann Peterson Bishop, Christine L. Borgman, Geoffrey C.

Bowker, Barbara P. Buttenfield, Anita Komlodi, David M. Levy, Clifford Lynch, Gary

Marchionini, Catherine C. Marshall, Bharat Mehra, Bonnie A. Nardi, Laura J. Neumann,

Vicki L. O’Day, Catherine Plaisant, Cynthia Smith, Mark A. Spasser, Susan Leigh Star,

Nancy A. Van House.

Of related interest

Digital Libraries

William Y. Arms

The emergence of the Internet and the wide availability of affordable computing equip￾ment have created tremendous interest in digital libraries and electronic publishing. This

book provides an integrated overview of the field, including a historical perspective, the

state of the art, and current research. An underlying theme of the book is that no aspect

of digital libraries can be understood in isolation or without attention to the needs of the

people who create and use information.

From Gutenberg to the Global Information Infrastructure

Access to Information in the Networked World

Christine L. Borgman

Will the emerging global information infrastructure (GII) create a revolution in communi￾cation equivalent to that wrought by Gutenberg, or will the result be simply the evolu￾tionary adaptation of existing behavior and institutions to new media? Will the GII replace

libraries and publishers? What are the trade-offs between tailoring information systems

to user communities and standardizing them to interconnect with systems designed for

other communities, cultures, and languages? This book takes a close look at these and

other questions of technology, behavior, and policy surrounding the GII.

The MIT Press

Massachusetts Institute of Technology

Cambridge, Massachusetts 02142

http://mitpress.mit.edu

0-262-02544-2

Foreword by Bruce Schatz

Digital Library Use Bishop, Van House, and Buttenfield, editors

,!7IA2G2-acfeeh!:t;K;k;K;k

Digital Library Use

Digital Libraries and Electronic Publishing

William Y. Arms, series editor

Gateways to Knowledge: The Role of Academic Libraries in Teaching, Learning,

and Research

edited by Lawrence Dowler, 1997

Civic Space/Cyberspace: The American Public Library in the Information Age

Redmond Kathleen Molz and Phyllis Dain, 1999

Digital Libraries

William Y. Arms, 1999

From Gutenberg to the Global Information Infrastructure: Access to Information

in the Networked World

Christine L. Borgman, 2000

The Intellectual Foundation of Information

Elaine Svenonius, 2000

Digital Library Use: Social Practice in Design and Evaluation

Ann Peterson Bishop, Nancy A. Van House, and Barbara P. Buttenfield, 2003

Digital Library Use

Social Practice in Design and Evaluation

Edited by Ann Peterson Bishop, Nancy A. Van House,

and Barbara P. Buttenfield

The MIT Press

Cambridge, Massachusetts

London, England

( 2003 Massachusetts Institute of Technology

All rights reserved. No part of this book may be reproduced in any form by any electronic or

mechanical means (including photocopying, recording, or information storage and retrieval)

without permission in writing from the publisher.

This book was set in Sabon on 3B2 by Asco Typesetters, Hong Kong.

Printed and bound in the United States of America.

Library of Congress Cataloging-in-Publication Data

Digital library use : social practice in design and evaluation / edited by Ann Peterson

Bishop, Nancy A. Van House, and Barbara P. Buttenfield.

p. cm. — (Digital libraries and electronic publishing)

Includes bibliographical references and index.

ISBN 0-262-02544-2 (hc. : alk. paper)

1. Digital libraries—Social aspects. 2. Digital libraries—Planning. 3. Information

technology—Social aspects. I. Bishop, Ann P. II. Van House, Nancy A. III. Buttenfield,

Barbara Pfeil. IV. Series.

ZA4080.D546 2003

0250

.00285—dc21 2002045248

10 9 8 7 6 5 4 3 2 1

Contents

Foreword vii

Bruce Schatz

1 Introduction: Digital Libraries as Sociotechnical Systems 1

Nancy A. Van House, Ann Peterson Bishop, and Barbara P. Buttenfield

Part I

2 Documents and Libraries: A Sociotechnical Perspective 25

David M. Levy

3 Finding the Boundaries of the Library without Walls 43

Catherine C. Marshall

4 An Ecological Perspective on Digital Libraries 65

Vicki L. O’Day and Bonnie A. Nardi

Part II

5 Designing Digital Libraries for Usability 85

Christine L. Borgman

6 The People in Digital Libraries: Multifaceted Approaches to Assessing Needs

and Impact 119

Gary Marchionini, Catherine Plaisant, and Anita Komlodi

7 Participatory Action Research and Digital Libraries: Reframing

Evaluation 161

Ann Peterson Bishop, Bharat Mehra, Imani Bazzell, and Cynthia Smith

8 Colliding with the Real World: Heresies and Unexplored Questions about

Audience, Economics, and Control of Digital Libraries 191

Clifford Lynch

Part III

9 Information and Institutional Change: The Case of Digital Libraries 219

Philip E. Agre

10 Transparency beyond the Individual Level of Scale: Convergence between

Information Artifacts and Communities of Practice 241

Susan Leigh Star, Geoffrey C. Bowker, and Laura J. Neumann

11 Digital Libraries and Collaborative Knowledge Construction 271

Nancy A. Van House

12 The Flora of North America Project: Making the Case [Study] for Social

Realist Theory 297

Mark A. Spasser

List of Contributors 329

Index 335

vi Contents

Foreword

Bruce Schatz

The world has changed radically with the emergence of the Internet. Information

retrieval used to be a specialized topic known by a few experts and practiced by a

few librarians. Today, millions of ordinary people all over the world routinely

search the Internet in an attempt to find useful information to solve their problems.

This emergence has made the process of organizing and searching digital collec￾tions a critical international need. As the Internet itself becomes increasingly part of

the structure of the world, so will the process of creating useful digital libraries

become a critical part of society (Schatz 1997). Previous generations of the Internet

were focused largely on the technology itself. When the Internet was originally

developed in the 1960s, the focus was on transmitting packets of data correctly

from one machine to another. Such transmitting could be engineered in a value-free

fashion in the abstract world of bits.

Today the focus has shifted dramatically to searching documents usefully across

many collections over the Internet. Such searching must be engineered to meet the

needs of users in the concrete world of people. There are not correct answers to

most queries in information retrieval, merely useful ones.

This shift from correct to useful has correspondingly created a shift in the focus of

projects needed to develop infrastructure for the Internet. Advances in technology

remain important, but considerations in sociology become equally important. The

development of an information-retrieval system is determined largely by technology.

But the deployment is determined largely by sociology.

A successful digital library is a place where a group of users (people) can effec￾tively search a group of documents (collection) via an information system (technol￾ogy). These three components must be in harmony, and all must be effective for the

digital library to be useful.

Shortly after the Internet began to be widely used in the mid-1990s, the first gen￾eration of digital library projects began. Since the technology was still quite new,

these projects were largely research projects by government-sponsored universities

or major library organizations. One major catalyst was the National Science Foun￾dation (NSF), National Aeronautics and Space Administration (NASA), and Defense

Advanced Research and Projects Agency (DARPA)’s Digital Libraries Initiative

(DLI) sponsored from 1994 to 1998.

The first generation of systems tends to be where the major styles are set. This first

generation was technology focused but generally had a goal of fielding some sort of

useful system as well. The tension between cool new technologies and mundane

daily needs was never resolved but was significantly noted by these projects (Schatz

and Chen 1996, 1999).

This volume contains thoughtful descriptions of the sociology research conducted

in several of these projects as well as work that represents the growing community

of researchers investigating social aspects of digital libraries. Fostering the develop￾ment of this community is one of the lasting contributions of the DLI. As principal

investigator of one of the DLI projects, I am pleased to see the final result at last and

glad to have been able to encourage its production with words and monies.

In this volume, we see evidence of the struggle to determine which research

methods to use for which stages of digital library development and deployment. The

process of dealing with conflicting goals over the course of multiyear projects is also

described. These descriptions allow readers to gain some feeling for the balancing

act in information systems between technological and sociological factors.

Some of the descriptions may seem theoretical in nature. Although these first￾generation projects tended to be rather academic, the problems and solutions con￾sidered are much the same as the commercial projects of later generations. As the

initial foray into critical infrastructure, these descriptions are significant practically

in addition to being valuable historically.

Digital libraries will form a major part of the structure of everyday life in the

future. Stakeholders of all types, from system builders to policy makers, will be

forced to deal with their successes and failures.

It is hoped that all will find useful guidance from this book and move closer to the

dream of usefully providing access to all the world’s knowledge.

viii Foreword

References

Schatz, B. 1997. Information Retrieval in Digital Libraries: Bringing Search to the Net.

Science, 275 (January 7) (special issue on Bioinformatics), 327–334.

Schatz, B., and H. Chen. 1996. Building Large-Scale Digital Libraries (Guest Editor’s Intro￾duction to Special Issue on Digital Libraries). IEEE Computer, 29(5), 23–27.

Schatz, B., and H. Chen. 1999. Digital Libraries: Technological Advancements and Social

Impacts. IEEE Computer, 32(2) (special issue on Digital Libraries), 45–50.

Foreword ix

Digital Library Use

1

Introduction: Digital Libraries as Sociotechnical

Systems

Nancy A. Van House, Ann Peterson Bishop, and Barbara P. Buttenfield

This book is about digital libraries as sociotechnical systems—networks of technol￾ogy, information, documents, people, and practices. It is about digital libraries’

interactions with the larger world of work, institutions, knowledge, and society, as

well as with the production of knowledge. And it is about creating, managing, and

evaluating DLs.

The term digital library (DL) encompasses a wide range of working systems and

research prototypes, collections of information and documents, and technologies.1

Much of the discussion about DLs is about technology or about specific applications

(e.g., Joint Conference on Digital Libraries 2001). This book takes a different

approach. We are interested in understanding the social aspects of DLs—not just

social impacts but the web of social and material relations within which DLs operate.

This book originated in the work of its editors and authors in designing, evaluat￾ing, and simply trying to understand DLs and their uses. Initially, this book was to

be about DL evaluation, but it rapidly became apparent to the authors and editors

that we were concerned with much more. We began with the belief that a good DL

is useful. Like a traditional library, a useful DL fits the needs, activities, and contexts

of the people who use it, as well as those of the people who create it, operate it, and

contribute to its content. The more we delved into DLs and their social worlds, the

more we found ourselves drawn into questions not just about DLs but about docu￾ments, collections, and classification; activity, work, and knowledge; politics and

values; institutions; and identity, organizations, and communities.

The contributors to this volume see technology as ‘‘embedded in the social world

in complicated ways, and this is particularly true for digital libraries, which are

intertwined with the cognitive processes of a complex society’’ (Philip E. Agre,

chapter 9). DLs form part of a long history of the mutual constitution of knowledge,

documents, technology, and the social (David M. Levy, chapter 2).

This book has two goals. One is to inform policy and professional practice in DLs

with socially grounded understanding of DLs as part of a web of social relations and

practices. Another is to perform ‘‘technically informed social analysis’’ (Bowker,

Star, Turner, and Gasser 1997, p. xiii) of phenomena of interest to social scientists

that are highlighted by digital libraries, specifically issues of work, groups, and

knowledge.

The chapters in this volume are unified by a sociotechnical approach. In this

context, this phrase has two meanings: the first, already introduced, views digital

libraries as composed of people, activity, artifacts, and technology. The second is an

analytical stance that ‘‘privileges neither the social nor the technological and in

which neither is reducible to the other’’ (Levy, chapter 2). Technology and the social

are instead mutually constituted; the ongoing dynamic of their relationship is one of

the themes of this book.

In this introduction, we consider socially grounded research in digital libraries

generally and discuss why this kind of research is needed. We describe the varied

domains and methods that come together in these chapters and identify major

themes. We outline the book, summarize chapters, and end with some reflections on

the implications of the book and of our approach to DL research.

Interconnections

Computers have escaped from the laboratories that once contained them. They

pervade offices. They have settled into dining rooms, third-grade art rooms, and

botanists’ knapsacks. Information circulates among desktop computers, hand-held

organizers, and mobile phones. With information technology operating in such a

wide sphere of human activity, the consequences of problems in such areas as

usability and access become significant. Information technology and systems simply

have more power to influence our lives, for good or ill. And as the users of infor￾mation technology have widened from professionals to everyone, the gap between

users and designers has widened. So to understand and design for use, we need to

know something about what people are doing at their desks and in the field and

what else rests on those desks and dining room tables.

As information technology becomes more embedded in everyday activities, we

become more aware of its role in social worlds. At the simplest level, people rely on

friends, relatives, and passers-by to learn how to use information technology.

Increasingly, social protocols develop around various kinds of information systems,

2 Nancy A. Van House, Ann Peterson Bishop, and Barbara P. Buttenfield