Principles of network and system administration

Principles of Network

and System Administration

Second Edition

Mark Burgess

Oslo University College, Norway

Principles of Network

and System Administration

Second Edition

Principles of Network

and System Administration

Second Edition

Mark Burgess

Oslo University College, Norway

Second edition copyright
c 2004 John Wiley & Sons Ltd, The Atrium, Southern Gate,

Chichester,

West Sussex PO19 8SQ, England

Telephone (+44) 1243 779777

Email (for orders and customer service enquiries): [email protected]

Visit our Home Page on www.wileyeurope.com or www.wiley.com

First edition copyright
c 2000 John Wiley & Sons Ltd

Cover painting: Man + Air + Space, 1915 (oil on canvas) by Lyubov’ Sergeena Popova

(1889-1924) State Russian Museum, St Petersburg, Russia/Bridgeman Art Gallery

All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or

transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning

or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the

terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London

W1T 4LP, UK, without the permission in writing of the Publisher, with the exception of any material

supplied specifically for the purpose of being entered and executed on a computer system for

exclusive use by the purchase of the publication. Requests to the Publisher should be addressed to

the Permissions Department, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West

Sussex PO19 8SQ, England, or emailed to [email protected], or faxed to (+44) 1243 770620.

This publication is designed to provide accurate and authoritative information in regard to the subject

matter covered. It is sold on the understanding that the Publisher is not engaged in rendering

professional services. If professional advice or other expert assistance is required, the services of a

competent professional should be sought.

Other Wiley Editorial Offices

John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA

Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA

Wiley-VCH Verlag GmbH, Boschstr. 12, D-69469 Weinheim, Germany

John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia

John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809

John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1

Wiley also publishes its books in a variety of electronic formats. Some content that appears

in print may not be available in electronic books.

Library of Congress Cataloging-in-Publication Data

Burgess, Mark, 1966–

Principles of network and system administration / Mark Burgess. – 2nd ed.

p. cm.

ISBN 0-470-86807-4 (Paper : alk. paper)

1. Computer networks – Management. 2. Computer systems. I. Title.

TK5105.5.B863 2003

005.4

3 – dc22

2003019766

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library

ISBN 0-470-86807-4

Typeset in 10/12pt Bookman by Laserwords Private Limited, Chennai, India

Printed and bound in Great Britain by Biddles Ltd, Guildford and King’s Lynn

This book is printed on acid-free paper responsibly manufactured from sustainable forestry

in which at least two trees are planted for each one used for paper production.

Contents

Preface to second edition xi

1 Introduction 1

1.1 What is network and system administration? . . . . . . . . . . . . 1

1.2 Applying technology in an environment . . . . . . . . . . . . . . . 2

1.3 The human role in systems . . . . . . . . . . . . . . . . . . . . . . . 2

1.4 Ethical issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.5 Is system administration a discipline? . . . . . . . . . . . . . . . . 3

1.6 The challenges of system administration . . . . . . . . . . . . . . . 4

1.7 Common practice and good practice . . . . . . . . . . . . . . . . . 5

1.8 Bugs and emergent phenomena . . . . . . . . . . . . . . . . . . . . 6

1.9 The meta principles of system administration . . . . . . . . . . . . 6

1.10 Knowledge is a jigsaw puzzle . . . . . . . . . . . . . . . . . . . . . . 7

1.11 To the student . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.12 Some road-maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 System components 11

2.1 What is ‘the system’? . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2 Handling hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.3 Operating systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.4 Filesystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.5 Processes and job control . . . . . . . . . . . . . . . . . . . . . . . . 43

2.6 Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.7 IPv4 networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.8 Address space in IPv4 . . . . . . . . . . . . . . . . . . . . . . . . . . 63

2.9 IPv6 networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3 Networked communities 75

3.1 Communities and enterprises . . . . . . . . . . . . . . . . . . . . . 75

3.2 Policy blueprints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

3.3 System uniformity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

3.4 User behavior: socio-anthropology . . . . . . . . . . . . . . . . . . 78

3.5 Clients, servers and delegation . . . . . . . . . . . . . . . . . . . . 78

3.6 Host identities and name services . . . . . . . . . . . . . . . . . . . 80

vi CONTENTS

3.7 Common network sharing models . . . . . . . . . . . . . . . . . . . 82

3.8 Local network orientation and analysis . . . . . . . . . . . . . . . . 86

4 Host management 109

4.1 Global view, local action . . . . . . . . . . . . . . . . . . . . . . . . 109

4.2 Physical considerations of server room . . . . . . . . . . . . . . . . 109

4.3 Computer startup and shutdown . . . . . . . . . . . . . . . . . . . 111

4.4 Configuring and personalizing workstations . . . . . . . . . . . . . 114

4.5 Installing a Unix disk . . . . . . . . . . . . . . . . . . . . . . . . . . 121

4.6 Installation of the operating system . . . . . . . . . . . . . . . . . . 124

4.7 Software installation . . . . . . . . . . . . . . . . . . . . . . . . . . 131

4.8 Kernel customization . . . . . . . . . . . . . . . . . . . . . . . . . . 140

5 User management 147

5.1 Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

5.2 User registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

5.3 Account policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

5.4 Login environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

5.5 User support services . . . . . . . . . . . . . . . . . . . . . . . . . . 161

5.6 Controlling user resources . . . . . . . . . . . . . . . . . . . . . . . 163

5.7 Online user services . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

5.8 User well-being . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

5.9 Ethical conduct of administrators and users . . . . . . . . . . . . 173

5.10 Computer usage policy . . . . . . . . . . . . . . . . . . . . . . . . . 186

6 Models of network and system administration 195

6.1 Information models and directory services . . . . . . . . . . . . . . 196

6.2 System infrastructure organization . . . . . . . . . . . . . . . . . . 201

6.3 Network administration models . . . . . . . . . . . . . . . . . . . . 207

6.4 Network management technologies . . . . . . . . . . . . . . . . . . 213

6.5 Creating infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . 219

6.6 System maintenance models . . . . . . . . . . . . . . . . . . . . . . 223

6.7 Competition, immunity and convergence . . . . . . . . . . . . . . . 225

6.8 Policy and configuration automation . . . . . . . . . . . . . . . . . 227

6.9 Integrating multiple OSs . . . . . . . . . . . . . . . . . . . . . . . . 228

6.10 A model checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

7 Configuration and maintenance 235

7.1 System configuration policy . . . . . . . . . . . . . . . . . . . . . . 236

7.2 Methods: controlling causes and symptoms . . . . . . . . . . . . . 237

7.3 Change management . . . . . . . . . . . . . . . . . . . . . . . . . . 239

7.4 Declarative languages . . . . . . . . . . . . . . . . . . . . . . . . . . 240

7.5 Policy configuration and its ethical usage . . . . . . . . . . . . . . 240

7.6 Common assumptions: clock synchronization . . . . . . . . . . . . 241

7.7 Human–computer job scheduling . . . . . . . . . . . . . . . . . . . 242

7.8 Automation of host configuration . . . . . . . . . . . . . . . . . . . 248

7.9 Preventative host maintenance . . . . . . . . . . . . . . . . . . . . 252

CONTENTS vii

7.10 SNMP tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

7.11 Cfengine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

7.12 Database configuration management . . . . . . . . . . . . . . . . . 268

8 Diagnostics, fault and change management 281

8.1 Fault tolerance and propagation . . . . . . . . . . . . . . . . . . . . 281

8.2 Networks and small worlds . . . . . . . . . . . . . . . . . . . . . . . 283

8.3 Causality and dependency . . . . . . . . . . . . . . . . . . . . . . . 285

8.4 Defining the system . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

8.5 Faults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

8.6 Cause trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

8.7 Probabilistic fault trees . . . . . . . . . . . . . . . . . . . . . . . . . 299

8.8 Change management revisited . . . . . . . . . . . . . . . . . . . . . 303

8.9 Game-theoretical strategy selection . . . . . . . . . . . . . . . . . . 304

8.10 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

8.11 System performance tuning . . . . . . . . . . . . . . . . . . . . . . 314

8.12 Principles of quality assurance . . . . . . . . . . . . . . . . . . . . 324

9 Application-level services 331

9.1 Application-level services . . . . . . . . . . . . . . . . . . . . . . . . 331

9.2 Proxies and agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

9.3 Installing a new service . . . . . . . . . . . . . . . . . . . . . . . . . 333

9.4 Summoning daemons . . . . . . . . . . . . . . . . . . . . . . . . . . 333

9.5 Setting up the DNS nameservice . . . . . . . . . . . . . . . . . . . 337

9.6 Setting up a WWW server . . . . . . . . . . . . . . . . . . . . . . . . 353

9.7 E-mail configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

9.8 OpenLDAP directory service . . . . . . . . . . . . . . . . . . . . . . 373

9.9 Mounting NFS disks . . . . . . . . . . . . . . . . . . . . . . . . . . . 374

9.10 Samba . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378

9.11 The printer service . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

9.12 Java web and enterprise services . . . . . . . . . . . . . . . . . . . 382

10 Network-level services 391

10.1 The Internet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391

10.2 A recap of networking concepts . . . . . . . . . . . . . . . . . . . . 392

10.3 Getting traffic to its destination . . . . . . . . . . . . . . . . . . . . 393

10.4 Alternative network transport technologies . . . . . . . . . . . . . 397

10.5 Alternative network connection technologies . . . . . . . . . . . . 400

10.6 IP routing and forwarding . . . . . . . . . . . . . . . . . . . . . . . 401

10.7 Multi-Protocol Label Switching (MPLS) . . . . . . . . . . . . . . . . 407

10.8 Quality of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408

10.9 Competition or cooperation for service? . . . . . . . . . . . . . . . 413

10.10 Service Level Agreements . . . . . . . . . . . . . . . . . . . . . . . . 415

11 Principles of security 423

11.1 Four independent issues . . . . . . . . . . . . . . . . . . . . . . . . 424

11.2 Physical security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426

viii CONTENTS

11.3 Trust relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . 427

11.4 Security policy and definition of security . . . . . . . . . . . . . . . 427

11.5 RFC 2196 and BS/ISO 17799 . . . . . . . . . . . . . . . . . . . . . 430

11.6 System failure modes . . . . . . . . . . . . . . . . . . . . . . . . . . 432

11.7 Preventing and minimizing failure modes . . . . . . . . . . . . . . 440

11.8 Some well-known attacks . . . . . . . . . . . . . . . . . . . . . . . . 445

12 Security implementation 453

12.1 System design and normalization . . . . . . . . . . . . . . . . . . . 453

12.2 The recovery plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

12.3 Data integrity and protection . . . . . . . . . . . . . . . . . . . . . . 454

12.4 Authentication methods . . . . . . . . . . . . . . . . . . . . . . . . 463

12.5 Analyzing network security . . . . . . . . . . . . . . . . . . . . . . . 469

12.6 VPNs: secure shell and FreeS/WAN . . . . . . . . . . . . . . . . . . 477

12.7 Role-based security and capabilities . . . . . . . . . . . . . . . . . 478

12.8 WWW security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478

12.9 IPSec – secure IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480

12.10 Ordered access control and policy conflicts . . . . . . . . . . . . . 483

12.11 IP filtering for firewalls . . . . . . . . . . . . . . . . . . . . . . . . . 485

12.12 Firewalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486

12.13 Intrusion detection and forensics . . . . . . . . . . . . . . . . . . . 493

12.14 Compromised machines . . . . . . . . . . . . . . . . . . . . . . . . 494

13 Analytical system administration 499

13.1 Science vs technology . . . . . . . . . . . . . . . . . . . . . . . . . . 499

13.2 Studying complex systems . . . . . . . . . . . . . . . . . . . . . . . 500

13.3 The purpose of observation . . . . . . . . . . . . . . . . . . . . . . . 502

13.4 Evaluation methods and problems . . . . . . . . . . . . . . . . . . 502

13.5 Evaluating a hierarchical system . . . . . . . . . . . . . . . . . . . 504

13.6 Deterministic and stochastic behavior . . . . . . . . . . . . . . . . 518

13.7 Observational errors . . . . . . . . . . . . . . . . . . . . . . . . . . . 528

13.8 Strategic analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536

13.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536

14 Summary and outlook 539

14.1 Information management in the future . . . . . . . . . . . . . . . . 540

14.2 Collaboration with software engineering . . . . . . . . . . . . . . . 540

14.3 Pervasive computing . . . . . . . . . . . . . . . . . . . . . . . . . . . 541

14.4 The future of system administration . . . . . . . . . . . . . . . . . 541

A Some useful Unix commands 543

B Programming and compiling 549

B.1 Make . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 549

B.2 Perl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553

B.3 WWW and CGI programming . . . . . . . . . . . . . . . . . . . . . . 574

C Example telnet session 581

CONTENTS ix

D Glossary 591

E Recommended reading 597

Bibliography 599

Index 623

Preface to second edition

This book grew originally out of a one-semester course in Network and System

Administration which has now run successfully for six years at Oslo College,

Norway. This first course is an introductory course and involves about thirty

percent theory and seventy percent practical work [40]; it assumes knowledge

equivalent to a typical college course on Operating Systems as well as some basic

computer skills. The purpose of this book was to provide a mixture of theory and

practice for a such course in system administration; to extract those principles

and ideas of system administration which do not change on a day-to-day basis;

and to present them in a defensible manner [188].

In writing the second edition, I have not only corrected shortcomings and

anachronisms in the original edition, but have attempted to compile a textbook

that goes beyond a single introductory course, and paints a larger picture. This

has been a very difficult task, and my book is very imperfect. It attempts to strike

a balance between completeness and selective tasting to satisfy the needs of a

student with a limited budget. It cannot hope to cover everything that a system

administrator ought to know, but it can provide a beginning. The resulting book

forms a sufficient basis for two or more courses at university level, assuming a

previous knowledge of operating systems. Indeed, this book is now the hub of our

Masters Degree in Network and System Administration at Oslo University College.

It makes contact with more traditional areas of computer science and engineering,

and provides an overview of material that will help to bind other more specific

works into a coherent whole. Although it covers material sufficient for more than

one course, it did not seem appropriate to divide the book into smaller parts, as it

also functions as an initial reference work for the field.

On a personal note, I never want to write a book like this again! Maintaining

this book is far harder than maintaining computers – and I can’t do it with

cfengine. The possibility for error and anachronism is enormous and the amount

of work to compile, maintain and generalize these concepts huge. To assemble

the book, I have reviewed the research work of many authors, most of which

has centered around the USENIX organization and its many groundbreaking

conferences. In spite of a desire for completeness, I have resisted the temptation

to include every possible detail and fact which might be useful in the practical

world. Several excellent books already exist, which cover this need, and I see no

reason to compete with them (see the recommended reading list). I have therefore

limited myself to examples of each which are either practical or illustrative. If any

operating systems have been unfairly brought into focus, I hope it is only the Free

xii PREFACE TO SECOND EDITION

operating systems such as GNU/Linux and the BSD’s, from which no one other

than their users will benefit.

For the new edition, I must add my thanks to several individuals. I am most

grateful to Steven Jenkins and Nick Christenson for both thorough, heroic readings

and razor-sharp critiques of the almost finished manuscript. Steve VanDevender

and Æleen Frisch also provided helpful comments and corrections. Thanks to

Jonathan Laventhol for interesting discussions about company policy in the UK

and for providing me with real-world examples, and the permission to adapt and

reproduce them here. Thanks to Hal Miller and Lee Damon for permission to

reproduce their versions of the SAGE code of ethics. Part of the section on SNMP is

based on Jurgen Sch ¨ onw ¨ alder’s excellent writings; I’m grateful to him for allowing ¨

me the indulgence, and for reading the result. Rob Apthorpe also allowed me to

base the discussion of fault trees on his LISA paper that I whipped and beat him

for a year earlier. I have benefited from my lunch discussions with Kyrre Begnum

and Assi Gueye.

From the original edition, I offer my special thanks to Tina Darmohray for

her comments and encouragement, as well as for allowing me to adapt some

firewall examples from her excellent notes. Russ Harvey of the University of

California, Riverside also made very positive and useful criticisms of the early

materials. Special thanks to Per Steinar Iversen for making detailed comments

and constructive criticisms on the manuscript from his near-infinite reservoir of

technical expertise. Thanks also to David Kuncicky, Sigmund Straumsnes and

Kjetil Sahlberg for their careful readings and suggestions for improvement. Any

remaining errors must be entirely someone else’s fault (but I haven’t figured out

who I can blame yet). Thanks to Knut Borge of USIT, University of Oslo, for

moderating the course on which this book is based and for teaching me many

important things over the years; also to Tore Øfsdahl and Harald Hofsæter, our

system administrators at Oslo College who constantly help me in often intangible

ways. Sigmund generated the graphs which appear in this volume. In addition

to them, Runar Jørgensen and Harek Haugerud commented on the manuscript. ˚

Ketil Danielsen has provided me with both tips and encouragement. Thanks to

Greg Smith of the NASA Ames Research Center for performance tips and to

Steve Traugott for discussions on infrastructure. I’m grateful to Cami Edwards of

USENIX for making copies of old LISA proceedings available from the archives.

I was shocked to discover just how true is the panel debate: why do we keep

reinventing the wheel? I should also like to thank all of the students at Oslo

University College who have attended my lectures and have inspired me to do

better than I might otherwise have done. Finally, all credit to the SAGE/USENIX

association for their unsurpassed work in spreading state of the art knowledge

about computing systems of all sizes and shapes.

Mark Burgess

Oslo University College