Tài liệu Pro Oracle Database 11g RAC on Linux docx

Books for professionals by professionals®

US $59.99

Shelve in:

Databases / Oracle

User level:

Intermediate–Advanced

THE APRESS ROADMAP

Pro

Oracle Database 11g

Administration

Pro

Oracle SQL

Beginning

Oracle Database 11g

Administration

Beginning

Oracle SQL

Pro

Oracle Database 11g

RAC on Linux

Expert

Oracle Practices

www.apress.com

SOURCE CODE ONLINE

Companion eBook

Steve Shaw, Author of

Pro Oracle Database 10g

RAC on Linux

Martin Bach

Pro Oracle Database 11g RAC

on Linux

Dear Reader,

You are holding in your hands the result of our experience and best-thinking when

it comes to installing, configuring, and tuning Oracle Database 11g RAC on Linux.

Real Application Clusters, or RAC as it is commonly called, is Oracle’s industry￾leading architecture for building scalable and fault-tolerant databases. RAC pro￾vides redundancy through multiple servers, allowing you to scale up and down

simply by adding or subtracting servers.

Using practical examples and illustrations we take you through all the stages

of building the infrastructure for the latest 11g Release 2 clustered environments

from selecting the right hardware components to installing and configuring Oracle

Enterprise Linux. We detail how to install and configure Oracle VM—Oracle’s

own virtualization solution—to enable anyone to begin working with Oracle RAC

straight away. We show the spectrum of configurations from single server to a fully

virtualized RAC implementation.

Building upon the Linux foundation, you will see how to successfully imple￾ment Oracle’s Grid infrastructure and RAC database software even when upgrad￾ing from an Oracle Database 10g release. You will also learn how to manage and

monitor your new clustered installation through workload management and per￾formance monitoring, and parallel execution.

We make no assumptions on your experience with Oracle 11g RAC Release 2,

or with Linux. Our goal in this book is to provide a complete reference to all of the

information you will need, beginning with the essential grounding of concepts and

architecture. We have comprehensively researched, tested, and detailed every step

of the process so this book can be your guide to taking the next step in the evolu￾tion of grid and cloud computing with Oracle 11g Release 2 RAC on Linux.

Steve Shaw & Martin Bach

The EXPERT’s VOIce® in Oracle

Pro

Oracle Database 11g

RAC on Linux

Installation, Administration, Performance

Steve Shaw and Martin Bach

Create robust and scalable database systems

using Oracle’s clustering and grid technologies

Covers

11g Release 2

Bach

Shaw Oracle Database 11

g RAC on Linux

Companion

eBook Available

Pro

www.it-ebooks.info

www.it-ebooks.info

Pro Oracle Database 11g RAC

on Linux

  

Steve Shaw

Martin Bach

www.it-ebooks.info

Pro Oracle Database 11g RAC on Linux

Copyright © 2010 by Steve Shaw and Martin Bach

All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means,

electronic or mechanical, including photocopying, recording, or by any information storage or retrieval

system, without the prior written permission of the copyright owner and the publisher.

ISBN-13 (pbk): 978-1-4302-2958-2

ISBN-13 (electronic): 978-1-4302-2959-9

Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1

Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol

with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only

in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of

the trademark.

The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are

not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject

to proprietary rights.

President and Publisher: Paul Manning

Lead Editor: Jonathan Gennick

Technical Reviewer: Bernhard Cock Buning and Sandesh Rao

Editorial Board: Clay Andres, Steve Anglin, Mark Beckner, Ewan Buckingham, Gary Cornell,

Jonathan Gennick, Jonathan Hassell, Michelle Lowman, Matthew Moodie, Duncan Parkes,

Jeffrey Pepper, Frank Pohlmann, Douglas Pundick, Ben Renow-Clarke, Dominic Shakeshaft,

Matt Wade, Tom Welsh

Coordinating Editor: Anita Castro

Copy Editor: Patrick Meader and Mary Ann Fugate

Compositor: Bytheway Publishing Services

Indexer: BIM Indexing & Proofreading Services

Artist: April Milne

Cover Designer: Anna Ishchenko

Distributed to the book trade worldwide by Springer Science+Business Media, LLC., 233 Spring Street,

6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer￾sbm.com, or visit www.springeronline.com.

For information on translations, please e-mail [email protected], or visit www.apress.com.

Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use.

eBook versions and licenses are also available for most titles. For more information, reference our

Special Bulk Sales–eBook Licensing web page at www.apress.com/info/bulksales.

The information in this book is distributed on an “as is” basis, without warranty. Although every

precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have

any liability to any person or entity with respect to any loss or damage caused or alleged to be caused

directly or indirectly by the information contained in this work.

www.it-ebooks.info

 CONTENTS AT A GLANCE

iii

Contents at a Glance

 About the Author.................................................................................................. xxi

 About the Technical Reviewer ............................................................................ xxii

 Acknowledgments ............................................................................................. xxiii

 Chapter 1: Introduction...........................................................................................1

 Chapter 2: RAC Concepts ......................................................................................27

 Chapter 3: RAC Architecture.................................................................................63

 Chapter 4: Hardware.............................................................................................97

 Chapter 5: Virtualization.....................................................................................165

 Chapter 6: Linux Installation and Configuration .................................................231

 Chapter 7: Grid Infrastructure Installation .........................................................323

 Chapter 8: Clusterware.......................................................................................379

 Chapter 9: Automatic Storage Management.......................................................455

 Chapter 10: RDBMS Installation and Configuration ............................................505

 Chapter 11: Workload Management ...................................................................559

 Chapter 12: Oracle Performance Monitoring......................................................607

 Chapter 13: Linux Performance Monitoring........................................................653

 Chapter 14: Parallel Execution ...........................................................................687

 Chapter 15: Upgrading to Oracle 11g Release 2 .................................................717

 Index...................................................................................................................771

www.it-ebooks.info

 CONTENTS

iv

Contents

 About the Authors................................................................................................ xxi

 About the Technical Reviewers .......................................................................... xxii

 Acknowledgments ............................................................................................. xxiii

 Chapter 1: Introduction...........................................................................................1

Introducing Oracle Real Application Clusters ....................................................................1

Examining the RAC Architecture........................................................................................3

Deploying RAC ...................................................................................................................4

Maintaining High Availability .................................................................................................................... 5

Defining Scalability................................................................................................................................... 6

Scaling Vertically vs. Horizontally............................................................................................................. 7

Increasing Manageability ......................................................................................................................... 8

Assessing the Cost of Ownership ........................................................................................................... 10

Clustering with Oracle on Linux.......................................................................................13

Running Linux on Oracle..................................................................................................16

Understanding the Role of Unix .............................................................................................................. 16

Liberating Software ................................................................................................................................ 17

Developing Linux .................................................................................................................................... 18

Expanding the Concept of Free with Open Source ................................................................................. 19

Combining Oracle, Open Source, and Linux ....................................................................20

Drilling Down on Unbreakable Linux ...................................................................................................... 21

Creating and Growing Red Hat Enterprise Linux .................................................................................... 22

Extending Red Hat with Oracle Enterprise Linux .................................................................................... 23

Drilling Down on SuSE Linux Enterprise Server ..................................................................................... 24

www.it-ebooks.info

 CONTENTS

v

Taking Linux to Asia ............................................................................................................................... 25

Summary .........................................................................................................................25

 Chapter 2: RAC Concepts ......................................................................................27

Clustering Concepts.........................................................................................................27

Configuring Active/active Clusters ......................................................................................................... 27

Implementing Active/passive Clusters ................................................................................................... 28

Configuring a Shared-All Architecture.................................................................................................... 28

Configuring a Shared-Nothing Architecture ........................................................................................... 29

Exploring the Main RAC Concepts ...................................................................................29

Working with Cluster Nodes ................................................................................................................... 29

Leveraging the Interconnect................................................................................................................... 30

Clusterware/Grid Infrastructure.............................................................................................................. 31

Leveraging Automatic Storage Management ......................................................................................... 39

Installing Real Application Clusters ........................................................................................................ 44

Using the Global Resource Directory (GRD) ............................................................................................ 49

Transferring Data Between Instances with Cache Fusion...................................................................... 51

Achieving Read Consistency................................................................................................................... 52

Synchronizing System Change Numbers ............................................................................................... 52

Exploring the New Features of 11g Release 2.................................................................52

Leveraging Grid Plug and Play................................................................................................................ 53

Modeling Resources with Server Pools.................................................................................................. 55

Ensuring POSIX Compliance with ACFS.................................................................................................. 56

Using Oracle Restart Instead of RAC ...................................................................................................... 57

Simplifying Clusterd Database Access with SCAN Listener ................................................................... 59

Summary .........................................................................................................................60

 Chapter 3: RAC Architecture.................................................................................63

Availability Considerations...............................................................................................63

Deciding the Number of Nodes............................................................................................................... 65

www.it-ebooks.info

 CONTENTS

vi

Online Maintenance and Patching.......................................................................................................... 67

Instance Recovery in RAC....................................................................................................................... 72

Failover Considerations ...................................................................................................74

Transparent Application Failover............................................................................................................ 75

Fast Connection Failover and Fast Application Notification ................................................................... 76

Scalability Considerations ...............................................................................................77

Scalability Enhancers ............................................................................................................................. 78

Scalability Inhibitors ............................................................................................................................... 79

Standby Databases..........................................................................................................81

Introduction to Oracle Standby Databases ............................................................................................. 82

Types of Standby Database .................................................................................................................... 83

Active Data Guard................................................................................................................................... 85

Extended Distance Clusters.................................................................................................................... 90

Oracle Streams ................................................................................................................91

Streams Processing................................................................................................................................ 92

Oracle Streams Prerequisites................................................................................................................. 93

Cluster Topologies ...........................................................................................................94

Summary .........................................................................................................................95

 Chapter 4: Hardware.............................................................................................97

Oracle Availability ............................................................................................................98

Server Processor Architecture.........................................................................................99

x86 Processor Fundamentals ................................................................................................................. 99

Multicore Processors and Hyper-Threading......................................................................................... 103

CPU Cache ............................................................................................................................................ 106

CPU Power Management...................................................................................................................... 109

Virtualization......................................................................................................................................... 111

Memory..........................................................................................................................112

Virtual Memory ..................................................................................................................................... 112

www.it-ebooks.info

 CONTENTS

vii

Physical Memory .................................................................................................................................. 113

NUMA.................................................................................................................................................... 116

Memory Reliability................................................................................................................................ 125

Additional Platform Features .........................................................................................125

Onboard RAID Storage.......................................................................................................................... 126

Machine Check Architectures............................................................................................................... 126

Remote Server Management and IPMI................................................................................................. 127

Network Interconnect Technologies..............................................................................127

Server I/O.............................................................................................................................................. 128

Private Interconnect ............................................................................................................................. 131

Storage Technologies ....................................................................................................136

RAC I/O Characteristics ........................................................................................................................ 137

Hard Disk and Solid State Disk Drive Performance .............................................................................. 143

RAID...................................................................................................................................................... 147

Storage Protocols for Linux .................................................................................................................. 153

Summary .......................................................................................................................164

 Chapter 5: Virtualization.....................................................................................165

Virtualization Definition and Benefits.............................................................................165

Oracle VM ......................................................................................................................168

Oracle VM Server Architecture ............................................................................................................. 168

Oracle VM Design ................................................................................................................................. 174

Oracle VM Server Installation ........................................................................................178

Oracle VM Manager Installation.....................................................................................183

Oracle VM CLI Installation and Configuration ................................................................186

Configuring Oracle VM...................................................................................................187

Network Configuration.......................................................................................................................... 187

Server Pool Configuration..................................................................................................................... 192

Installing and Configuring Guests..................................................................................208

www.it-ebooks.info

 CONTENTS

viii

Importing a Template ........................................................................................................................... 209

Creating a Guest from a Template........................................................................................................ 210

Accessing a Guest ................................................................................................................................ 212

Configuring a Guest for RAC ................................................................................................................. 214

Managing Domains........................................................................................................216

Oracle VM Agent................................................................................................................................... 216

Oracle VM Manager .............................................................................................................................. 218

Oracle VM Manager CLI ........................................................................................................................ 220

The xm Command-Line Interface ......................................................................................................... 222

Summary .......................................................................................................................230

 Chapter 6: Linux Installation and Configuration .................................................231

Selecting the Right Linux Software ...............................................................................231

Reviewing the Hardware Requirements ........................................................................232

Drilling Down on Networking Requirements..................................................................233

Configuring a GNS or a Manual IP ........................................................................................................ 233

Configuring DNS and DHCP .................................................................................................................. 236

Downloading the Linux Software...................................................................................243

Preparing for a Network Install......................................................................................243

Installing Oracle Enterprise Linux 5...............................................................................247

Starting the Installation ........................................................................................................................ 247

Installation Media Check ...................................................................................................................... 247

Anaconda Installation ........................................................................................................................... 247

Install or Upgrade ................................................................................................................................. 248

Disk Partitioning ................................................................................................................................... 248

Configuring the Boot Loader and Network ........................................................................................... 259

Selecting a Time Zone .......................................................................................................................... 260

Configuring the Root Password ............................................................................................................ 261

Reviewing the Package Installation Defaults ....................................................................................... 261

www.it-ebooks.info

 CONTENTS

ix

Selecting a Package Group................................................................................................................... 261

Installing Packages............................................................................................................................... 263

Setting the Final Configuration............................................................................................................. 263

Accepting the License Agreement........................................................................................................ 263

Configuring the Firewall ....................................................................................................................... 263

Configuring SELinux ............................................................................................................................. 263

Enabling kdump.................................................................................................................................... 264

Setting the Date and Time.................................................................................................................... 264

Creating a User..................................................................................................................................... 265

Installing Additional CDs....................................................................................................................... 265

Configuring Oracle Enterprise Linux 5 ...........................................................................265

Configuring a Server with the Oracle Validated RPM ........................................................................... 266

Verifying the Oracle Validated RPM Actions ......................................................................................... 270

Post Oracle Validated RPM Configuration............................................................................................. 282

Completing the Linux Configuration for RAC ........................................................................................ 292

Configuring Shared Storage ................................................................................................................. 298

Discovering and Configuring SAN Disk................................................................................................. 299

Network Channel Bonding.................................................................................................................... 313

I/O Fencing with IPMI............................................................................................................................ 317

Summary .......................................................................................................................322

 Chapter 7: Grid Infrastructure Installation .........................................................323

Getting Ready for Installation ........................................................................................323

Obtain Software Distribution ................................................................................................................ 323

Configure X Environment...................................................................................................................... 324

Determining Configuration Type........................................................................................................... 327

Advanced Installation - Manual Configuration...............................................................327

Network Configuration.......................................................................................................................... 328

DNS Configuration ................................................................................................................................ 329

Choosing an Installation Option............................................................................................................ 330

www.it-ebooks.info

 CONTENTS

x

Selecting an Advanced or Typical Installation Type . ........................................................................... 332

Selecting a Language . ......................................................................................................................... 333

Configuring the Grid Plug and Play. ..................................................................................................... 334

Configuring the Cluster Node Information Page . ................................................................................. 336

Configuring the Network Interface Usage Page . ................................................................................. 337

Configuring the Storage Option Information Page. .............................................................................. 338

Creating an ASM Disk Group . .............................................................................................................. 340

Specifying an ASM Password. ............................................................................................................. 341

Specifying a Username and Password for IPMI. .................................................................................. 342

Configuring Privileged Operating System Groups. ............................................................................... 342

Setting the Installation Location. ......................................................................................................... 344

Specify the Central Inventory’s Location . ............................................................................................ 345

Performing Prerequisite Checks. ......................................................................................................... 345

Reviewing the Summary Page. ............................................................................................................ 351

Setup Page . ......................................................................................................................................... 352

Reviewing Execute Configuration Scripts. ........................................................................................... 352

Monitoring Configuration Assistants . .................................................................................................. 359

Implementing an Advanced Installation for Automatic Configuration ...........................360

Configuring a Network Configuration . ................................................................................................. 360

Configuring DNS . ................................................................................................................................. 362

Configuring DHCP . ............................................................................................................................... 363

Setting up the Grid Plug and Play Information Page. ........................................................................... 364

Configuring the Cluster Node Information Page . ................................................................................. 365

The Summary Page . ............................................................................................................................ 366

Typical Installation.........................................................................................................367

Choosing the Installation Type . ........................................................................................................... 367

Specifying the Cluster Configuration Page . ......................................................................................... 368

Install Locations Page. ......................................................................................................................... 369

Reviewing the Summary Page for a Typical Installation . .................................................................... 370

www.it-ebooks.info

 CONTENTS

xi

Installing a Standalone Server.......................................................................................371

Selecting an Installation Option............................................................................................................ 372

Creating an ASM Disk Group Page ....................................................................................................... 373

Reviewing the Summary Page for a Standalone Installation................................................................ 373

Configuring the Execute Configuration Scripts..................................................................................... 375

Deinstalling the Grid Infrastructure Software................................................................376

Summary .......................................................................................................................377

 Chapter 8: Clusterware.......................................................................................379

Introducing Clusterware ................................................................................................379

Examining the Hardware and Software Requirements..................................................380

Using Shared Storage with Oracle Clusterware ............................................................381

Storing Cluster Information with the Oracle Cluster Registry............................................................... 381

Storing Information in the Oracle Local Registry.................................................................................. 382

Fencing with the Voting Disk................................................................................................................ 382

Recording Information with the Grid Plug and Play Profile................................................................... 383

Using Background Processes ........................................................................................384

Grid Infrastructure Software Stacks..................................................................................................... 384

Drilling Down on the High Availability Stack ........................................................................................ 385

Drilling Down on the Cluster Ready Services Stack ............................................................................. 386

Using Grid Infrastructure Agents .......................................................................................................... 388

Initiating the Startup Sequence............................................................................................................ 389

Managing Oracle Clusterware .......................................................................................391

Using the Enterprise Manager.............................................................................................................. 392

Using the Clusterware Control Utility.................................................................................................... 392

Managing Resources with srvctl .......................................................................................................... 395

Verifying the Cluster with the CVU........................................................................................................ 396

Configuring Network Interfaces with oifcfg.......................................................................................... 400

Administering the OCR and OLR with ocrconfig ................................................................................... 400

www.it-ebooks.info

 CONTENTS

xii

Checking the State of the OCR and its Mirrors with ocrcheck ............................................................. 400

Dumping Contents of the OCR with ocrdump ....................................................................................... 400

Defining Server-Side Callouts........................................................................................401

Protecting Applications with Clusterware .....................................................................403

Managing Resource Profiles................................................................................................................. 403

Configuring Active/Passive Clustering for Oracle Database ................................................................. 404

Configuring Active/Passive Clustering for Apache Tomcat................................................................... 409

Using Oracle Restart......................................................................................................413

Troubleshooting.............................................................................................................415

Resolving Startup Issues ...................................................................................................................... 415

Resolving Problems with Java Utilities................................................................................................. 422

Patching Grid Infrastructure ..........................................................................................422

Adding and Deleting Nodes ...........................................................................................426

Adding Nodes ....................................................................................................................................... 426

Deleting Nodes ..................................................................................................................................... 433

Exploring More Advanced Topics...................................................................................438

Selecting non-Default Listener Ports.................................................................................................... 439

Selecting a non-Default SCAN Listener Endpoint ................................................................................. 442

Changing the SCAN After Installation ................................................................................................... 443

Maintaining Voting Disks...................................................................................................................... 444

Maintaining Local and Cluster Registry................................................................................................ 448

Summary .......................................................................................................................453

 Chapter 9: Automatic Storage Management.......................................................455

Introducing ASM ............................................................................................................455

ASM Terminology ................................................................................................................................. 456

Supported File Types............................................................................................................................ 457

ASM Management ................................................................................................................................ 458

ASM and RDBMS Support..................................................................................................................... 458

www.it-ebooks.info

 CONTENTS

xiii

ASM Installation.............................................................................................................459

ASM Components and Concepts....................................................................................459

ASM Instances...................................................................................................................................... 459

Failure Groups ...................................................................................................................................... 464

ASM Files.............................................................................................................................................. 465

Redundancy.......................................................................................................................................... 468

Striping................................................................................................................................................. 468

Mirroring............................................................................................................................................... 469

Intelligent Data Placement ................................................................................................................... 469

Access Control...................................................................................................................................... 470

Maintaining ASM............................................................................................................475

Creating an ASM Disk Group ................................................................................................................ 475

Extending an ASM Disk Group.............................................................................................................. 478

Dropping Disks from an ASM Disk Group ............................................................................................. 479

Enabling Disk Discovery ....................................................................................................................... 480

Understanding the ASM Header ........................................................................................................... 480

Installing the Grid Infrastructure........................................................................................................... 481

Re-creating the ASM Disks................................................................................................................... 482

ASM Cluster File System ...............................................................................................482

Creating and Mounting an ACFS Using ASMCA .................................................................................... 484

Creating and Mounting an ACFS Using the Command Line.................................................................. 491

Maintaining the ACFS ........................................................................................................................... 494

Using ACFS with Oracle Restart ........................................................................................................... 496

Administering ASM ........................................................................................................496

Using SQL*Plus to Administer ASM ...................................................................................................... 497

ASM Administration Using SRVCTL....................................................................................................... 499

Accessing Files in ASM......................................................................................................................... 500

Using Files Instead of Devices.............................................................................................................. 501

Virtualization and Shared Disks............................................................................................................ 502

www.it-ebooks.info