2-D and 3-D Image Registration for Medical, Remote Sensing, and Industrial Applications pptx

2-D and 3-D

Image Registration

2-D and 3-D

Image Registration

for Medical, Remote Sensing,

and Industrial Applications

A. Ardeshir Goshtasby

A John Wiley & Sons, Inc., Publication

Copyright
c 2005 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by

any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted

under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written

permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the

Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-646-

8600, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed

to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-

6011, fax (201) 748-6008.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in

preparing this book, they make no representations or warranties with respect to the accuracy or

completeness of the contents of this book and specifically disclaim any implied warranties of

merchantability or fitness for a particular purpose. No warranty may be created or extended by sales

representatives or written sales materials. The advice and strategies contained herein may not be suitable

for your situation. You should consult with a professional where appropriate. Neither the publisher nor

author shall be liable for any loss of profit or any other commercial damages, including but not limited to

special, incidental, consequential, or other damages.

For general information on our other products and services please contact our Customer Care Department

within the U.S. at 877-762-2974, outside the U.S. at 317-572-3993 or fax 317-572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print, however,

may not be available in electronic format.

Library of Congress Cataloging-in-Publication Data:

Goshtasby, Ardeshir.

2-D and 3-D image registration for medical, remote sensing, and industrial applications /

A. Ardeshir Goshtasby.

p. cm.

“Wiley-Interscience publication.”

Includes bibliographical references and index.

ISBN 0-471-64954-6 (cloth : alk. paper)

1. Image processing–Digital techniques. 2. Image analysis–Data processing. I. Title.

TA1637.G68 2005

621.36’7–dc22

2004059083

Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

To My Parents

and Mariko and Parviz

Contents

Preface xi

Acknowledgments xiii

Acronyms xv

1 Introduction 1

1.1 Terminologies 3

1.2 Steps in Image Registration 4

1.3 Summary of the Chapters to Follow 5

1.4 Bibliographical Remarks 5

2 Preprocessing 7

2.1 Image Enhancement 7

2.1.1 Image smoothing 7

2.1.2 Deblurring 11

2.2 Image Segmentation 15

2.2.1 Intensity thresholding 15

2.2.2 Boundary detection 17

2.3 Summary 39

vii

viii CONTENTS

2.4 Bibliographical Remarks 40

3 Feature Selection 43

3.1 Points 43

3.2 Lines 51

3.2.1 Line detection using the Hough transform 52

3.2.2 Least-squares line fitting 53

3.2.3 Line detection using image gradients 56

3.3 Regions 58

3.4 Templates 59

3.5 Summary 60

3.6 Bibliographical Remarks 60

4 Feature Correspondence 63

4.1 Point Pattern Matching 63

4.1.1 Matching using scene coherence 64

4.1.2 Matching using clustering 67

4.1.3 Matching using invariance 70

4.2 Line Matching 74

4.3 Region Matching 77

4.3.1 Shape matching 78

4.3.2 Region matching by relaxation labeling 82

4.4 Chamfer Matching 86

4.4.1 Distance transform 87

4.5 Template Matching 92

4.5.1 Similarity measures 92

4.5.2 Gaussian-weighted templates 99

4.5.3 Template size 100

4.5.4 Coarse-to-fine methods 101

4.6 Summary 103

4.7 Bibliographical Remarks 103

5 Transformation Functions 107

5.1 Similarity Transformation 112

CONTENTS ix

5.2 Projective and Affine Transformations 115

5.3 Thin-Plate Spline 116

5.4 Multiquadric 120

5.5 Weighted Mean Methods 123

5.6 Piecewise Linear 129

5.7 Weighted Linear 131

5.8 Computational Complexity 134

5.9 Properties of the Transformation Functions 136

5.10 Summary 139

5.11 Bibliographical Remarks 140

6 Resampling 143

6.1 Nearest Neighbor 144

6.2 Bilinear Interpolation 145

6.3 Cubic Convolution 147

6.4 Cubic Spline 149

6.5 Radially Symmetric Kernels 150

6.6 Summary 153

6.7 Bibliographical Remarks 154

7 Performance Evaluation 155

7.1 Feature Selection Performance 156

7.2 Feature Correspondence Performance 160

7.3 Transformation Function Performance 161

7.4 Registration Performance 163

7.5 Summary 164

7.6 Bibliographical Remarks 164

8 Image Fusion 167

8.1 Fusing Multi-Exposure Images 168

8.1.1 Image blending 168

8.1.2 Examples 172

8.2 Fusing Multi-Focus Images 175

8.3 Summary 177

x CONTENTS

8.4 Bibliographical Remarks 177

9 Image Mosaicking 181

9.1 Problem Description 182

9.2 Determining the Global Transformation 182

9.3 Blending Image Intensities 185

9.4 Examples 186

9.5 Mosaicking Range Images 189

9.6 Evaluation 192

9.7 Summary 193

9.8 Bibliographical Remarks 194

10 Stereo Depth Perception 197

10.1 Stereo Camera Geometry 198

10.2 Camera Calibration 202

10.3 Image Rectification 204

10.4 The Correspondence Process 207

10.4.1 Constraints in stereo 207

10.4.2 Correspondence algorithms 210

10.5 Interpolation 217

10.6 Summary 219

10.7 Bibliographical Remarks 220

Glossary 223

References 229

Index 255

Preface

Image registration is the process of spatially aligning two or more images of a scene.

This basic capability is needed in various image analysis applications. The alignment

process will determine the correspondence between points in the images, enabling

the fusion of information in the images and the determination of scene changes.

If identities of objects in one of the images are known, by registering the images,

identities of objects and their locations in another image can be determined. Image

registration is a critical component of remote sensing, medical, and industrial image

analysis systems.

This book is intended for image analysis researchers as well as graduate students

who are starting research in image analysis. The book provides details of image

registration, and each chapter covers a component of image registration or an appli￾cation of it. Where applicable, implementation strategies are given and related work

is summarized.

In Chapter 1, the main terminologies used in the book are defined, an example of

image registration is given, and image registration steps are named. In Chapter 2,

preprocessing of images to facilitate image registration is described. This includes

image enhancement and image segmentation. Image enhancement is used to remove

noise and blur from images and image segmentation is used to partition images into

regions or extract region boundaries or edges for use in feature selection.

Chapters 3–5 are considered the main chapters in the book, covering the image

registration steps. In Chapter 3, methods and algorithms for detecting points, lines,

and regions are described, in Chapter 4, methods and algorithms for determining the

correspondence between two sets of features are given, and in Chapter 5, transforma￾xi

xii PREFACE

tion functions that use feature correspondences to determine a mapping function for

image alignment are discussed.

In Chapter 6 resampling methods are given and in Chapter 7 performance evalu￾ation measures, including accuracy, reliability, robustness, and speed are discussed.

Chapters 8–10 cover applications of image registration. Chapter 8 discusses cre￾ation of intensity and range image mosaics by registering overlapping areas in the

images, and Chapter 9 discusses methods for combining information in two or more

registered images into a single highly informative image. In particular, fusion of

multi-exposure and multi-focus images is discussed. Finally, Chapter 10 discusses

registration of stereo images for depth perception. Camera calibration and correspon￾dence algorithms are discussed in detail and examples are given.

Some of the discussions such as stereo depth perception apply to only 2-D images,

but many of the topics covered in the book can be applied to both 2-D and 3-D

images. Therefore, discussions on 2-D image registration and 3-D image registration

continue in parallel. First the 2-D methods and algorithms are described and then

their extensions to 3-D are provided.

This book represents my own experiences on image registration during the past

twenty years. The main objective has been to cover the fundamentals of image

registration in detail. Applications of image registration are not discussed in depth.

A large number of application papers appear annually in Proc. Computer Vision and

Pattern Recognition, Proc. Int’l Conf. Computer Vision, Proc. Int’l Conf. Pattern

Recognition, Proc. SPIE Int’l Sym. Medical Imaging, and Proc. Int’l Sym. Remote

Sensing of Environment. Image registration papers frequently appear in the following

journals: Int’l J. Computer Vision, Computer Vision and Image Understanding, IEEE

Trans. Pattern Analysis and Machine Intelligence, IEEE Trans. Medical Imaging,

IEEE Trans. Geoscience and Remote Sensing, Image and Vision Computing, and

Pattern Recognition.

The figures used in the book are available online and may be obtained by visitingthe

website http://www.imgfsr.com/book.html. The software implementing the methods

and algorithms discussed in the book can be obtained by visiting the same site. Any

typographical errors or errata found in the book will also be posted on this site. The

site also contains other sources of information relating to image registration.

A. ARDESHIR GOSHTASBY

Dayton, Ohio, USA

Acknowledgments

I would like to thank NASA for providing the satellite images and Kettering Med￾ical Center, Kettering, Ohio, for providing the medical images used in this book.

I also would like to thank Shree Nayar of Columbia University for providing the

multi-exposure images shown in Figs 8.2–8.5; Max Lyons for providing the multi￾exposure images shown in Fig. 8.6; Paolo Favaro, Hailin Jin, and Stefano Saotto of

University of California at Los Angeles for providing the multi-focus images shown in

Fig. 8.7; Cody Benkelman of Positive Systems for providing the aerial images shown

in Fig. 9.4; Yuichi Ohta of Tsukuba University for providing the stereo image pair

shown in Fig. 10.10; and Daniel Scharstein of Middlebury College and Rick Szeliski

of Microsoft Research for providing the stereo image pair shown in Fig. 10.11. My

Ph.D. students, Lyubomir Zagorchev, Lijun Ding, and Marcel Jackowski, have con￾tributed to this book in various ways and I appreciate their contributions. I also would

like to thank Libby Stephens for editing the grammar and style of this book.

A. A. G.

xiii