Nonlinear programming : concepts, algorithms, and applications to chemical processes

Nonlinear

Programming

MP10_Biegler_FM-A.indd 1 7/6/2010 11:34:54 AM

This series is published jointly by the Mathematical Optimization Society and the Society for

Industrial and Applied Mathematics. It includes research monographs, books on applications,

textbooks at all levels, and tutorials. Besides being of high scientific quality, books in the series

must advance the understanding and practice of optimization. They must also be written clearly

and at an appropriate level.

Editor-in-Chief

Thomas Liebling

École Polytechnique Fédérale de Lausanne

Editorial Board

William Cook, Georgia Tech

Gérard Cornuejols, Carnegie Mellon University

Oktay Gunluk, IBM T.J. Watson Research Center

Michael Jünger, Universität zu Köln

C.T. Kelley, North Carolina State University

Adrian S. Lewis, Cornell University

Pablo Parrilo, Massachusetts Institute of Technology

Daniel Ralph, University of Cambridge

Éva Tardos, Cornell University

Mike Todd, Cornell University

Laurence Wolsey, Université Catholique de Louvain

Series Volumes

Biegler, Lorenz T., Nonlinear Programming: Concepts, Algorithms, and Applications to

Chemical Processes

Shapiro, Alexander, Dentcheva, Darinka, and Ruszczynski, Andrzej, Lectures on Stochastic

Programming: Modeling and Theory

Conn, Andrew R., Scheinberg, Katya, and Vicente, Luis N., Introduction to Derivative-Free

Optimization

Ferris, Michael C., Mangasarian, Olvi L., and Wright, Stephen J., Linear Programming with MATLAB

Attouch, Hedy, Buttazzo, Giuseppe, and Michaille, Gérard, Variational Analysis in Sobolev

and BV Spaces: Applications to PDEs and Optimization

Wallace, Stein W. and Ziemba, William T., editors, Applications of Stochastic Programming

Grötschel, Martin, editor, The Sharpest Cut: The Impact of Manfred Padberg and His Work

Renegar, James, A Mathematical View of Interior-Point Methods in Convex Optimization

Ben-Tal, Aharon and Nemirovski, Arkadi, Lectures on Modern Convex Optimization: Analysis,

Algorithms, and Engineering Applications

Conn, Andrew R., Gould, Nicholas I. M., and Toint, Phillippe L., Trust-Region Methods

MOS-SIAM Series on Optimization

´

MP10_Biegler_FM-A.indd 2 7/6/2010 11:34:54 AM

Nonlinear

Programming

Concepts, Algorithms, and

Applications to Chemical Processes

Lorenz T. Biegler

Carnegie Mellon University

Pittsburgh, Pennsylvania

Society for Industrial and Applied Mathematics

Philadelphia

Mathematical Optimization Society

Philadelphia

MP10_Biegler_FM-A.indd 3 7/6/2010 11:34:54 AM

Copyright © 2010 by the Society for Industrial and Applied Mathematics and the Mathematical

Optimization Society

10 9 8 7 6 5 4 3 2 1

All rights reserved. Printed in the United States of America. No part of this book may be reproduced,

stored, or transmitted in any manner without the written permission of the publisher. For information,

write to the Society for Industrial and Applied Mathematics, 3600 Market Street, 6th Floor, Philadelphia,

PA 19104-2688.

Trademarked names may be used in this book without the inclusion of a trademark symbol. These

names are used in an editorial context only; no infringement of trademark is intended.

AIMMS is a registered trademark of Paragon Decision Technology B.V.

AMPL is a trademark of AMPL Optimization LLC.

Excel is a trademark of Microsoft Corporation in the United States and/or other countries.

GAMS is a trademark of Gams Development Corp.

gPROMS is a trademark of Process Systems Enterprise, Ltd.

MATLAB is a registered trademark of The MathWorks, Inc. For MATLAB product information, please

contact The MathWorks, Inc., 3 Apple Hill Drive, Natick, MA 01760-2098 USA, 508-647-7000,

Fax: 508-647-7001 [email protected], www.mathworks.com.

TOMLAB is a registered trademark of Tomlab Optimization.

Library of Congress Cataloging-in-Publication Data

Biegler, Lorenz T.

Nonlinear programming : concepts, algorithms, and applications to chemical processes / Lorenz T.

Biegler.

p. cm.

Includes bibliographical references and index.

ISBN 978-0-898717-02-0

1. Chemical processes. 2. Nonlinear programming. I. Title.

TP155.75.B54 2010

519.7’6--dc22 2010013645

is a registered trademark.

MP10_Biegler_FM-A.indd 4 7/6/2010 11:34:54 AM

In memory of my father

To my mother

To Lynne and to Matthew

To all my students



MP10_Biegler_FM-A.indd 5 7/6/2010 11:34:54 AM

book_tem

2010/7/27

page vii

✐

✐

✐

✐

✐

✐

✐

✐

Contents

Preface xiii

1 Introduction to Process Optimization 1

1.1 Scope of Optimization Problems .................... 1

1.2 Classification of Optimization Problems ................ 3

1.3 Optimization Applications in Chemical Engineering .......... 5

1.4 Nonlinear Programming Examples in Chemical Engineering ...... 6

1.4.1 Design of a Small Heat Exchanger Network ....... 7

1.4.2 Real-Time Optimization of a Distillation Column .... 9

1.4.3 Model Predictive Control . . . . . . . . . . . . . . . . . 11

1.5 A Motivating Application . . . . . . . . . . . . . . . . . . . . . . . . 13

1.6 Summary and Notes for Further Reading . . . . . . . . . . . . . . . . 15

1.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2 Concepts of Unconstrained Optimization 17

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.2.1 Vectors and Matrices . . . . . . . . . . . . . . . . . . . . 19

2.2.2 Quadratic Forms . . . . . . . . . . . . . . . . . . . . . . 22

2.2.3 Classification of Functions . . . . . . . . . . . . . . . . . 25

2.3 Optimality Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.4 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.4.1 Direct Search Methods . . . . . . . . . . . . . . . . . . . 30

2.4.2 Methods That Require Derivatives . . . . . . . . . . . . 33

2.5 Summary and Notes for Further Reading . . . . . . . . . . . . . . . . 37

2.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3 Newton-Type Methods for Unconstrained Optimization 39

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.2 Modification of the Hessian Matrix . . . . . . . . . . . . . . . . . . . 40

3.3 Quasi-Newton Methods . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.4 Line Search Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.5 Trust Region Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.5.1 Convex Model Problems . . . . . . . . . . . . . . . . . 53

3.5.2 Nonconvex Model Problems . . . . . . . . . . . . . . . . 56

vii

book_tem

2010/7/27

page viii

✐

✐

✐

✐

✐

✐

✐

✐

viii Contents

3.6 Summary and Notes for Further Reading . . . . . . . . . . . . . . . . 60

3.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

4 Concepts of Constrained Optimization 63

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

4.1.1 Constrained Convex Problems . . . . . . . . . . . . . . . 64

4.2 Local Optimality Conditions—A Kinematic Interpretation . . . . . . . 68

4.3 Analysis of KKT Conditions . . . . . . . . . . . . . . . . . . . . . . 72

4.3.1 Linearly Constrained Problems . . . . . . . . . . . . . . 75

4.3.2 Nonlinearly Constrained Problems . . . . . . . . . . . . 76

4.3.3 Second Order Conditions . . . . . . . . . . . . . . . . . 79

4.4 Special Cases: Linear and Quadratic Programs . . . . . . . . . . . . . 84

4.4.1 Description of Linear Programming . . . . . . . . . . . . 84

4.4.2 Description of Quadratic Programming . . . . . . . . . . 85

4.4.3 Portfolio Planning Case Study . . . . . . . . . . . . . . . 86

4.5 Summary and Notes for Further Reading . . . . . . . . . . . . . . . . 89

4.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

5 Newton Methods for Equality Constrained Optimization 91

5.1 Introduction to Equality Constrained Optimization . . . . . . . . . . . 91

5.2 Newton’s Method with the KKT Matrix . . . . . . . . . . . . . . . . 92

5.2.1 Nonsingularity of KKT Matrix . . . . . . . . . . . . . . 94

5.2.2 Inertia of KKT Matrix . . . . . . . . . . . . . . . . . . . 95

5.3 Taking Newton Steps . . . . . . . . . . . . . . . . . . . . . . . . . . 96

5.3.1 Full-Space Newton Steps . . . . . . . . . . . . . . . . . 96

5.3.2 Reduced-Space Newton Steps . . . . . . . . . . . . . . . 99

5.4 Quasi-Newton Methods . . . . . . . . . . . . . . . . . . . . . . . . . 102

5.4.1 A Quasi-Newton Full-Space Method . . . . . . . . . . . 103

5.4.2 A Quasi-Newton Reduced-Space Method . . . . . . . . . 105

5.5 Globalization for Constrained Optimization . . . . . . . . . . . . . . 109

5.5.1 Concepts of Merit Functions . . . . . . . . . . . . . . . . 109

5.5.2 Filter Method Concepts . . . . . . . . . . . . . . . . . . 112

5.5.3 Filter versus Merit Function Strategies . . . . . . . . . . 113

5.6 Line Search Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 114

5.6.1 Line Search with Merit Functions . . . . . . . . . . . . . 115

5.6.2 Line Search Filter Method . . . . . . . . . . . . . . . . . 119

5.7 Trust Region Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 122

5.7.1 Trust Regions with Merit Functions . . . . . . . . . . . . 123

5.7.2 Filter Trust Region Methods . . . . . . . . . . . . . . . . 126

5.8 Combining Local and Global Properties . . . . . . . . . . . . . . . . 128

5.8.1 The Maratos Effect . . . . . . . . . . . . . . . . . . . . . 128

5.9 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 130

5.10 Notes for Further Reading . . . . . . . . . . . . . . . . . . . . . . . . 131

5.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

6 Numerical Algorithms for Constrained Optimization 133

6.1 Constrained NLP Formulations . . . . . . . . . . . . . . . . . . . . . 133

book_tem

2010/7/27

page ix

✐

✐

✐

✐

✐

✐

✐

✐

Contents ix

6.2 SQP Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

6.2.1 The Basic, Full-Space SQP Algorithm . . . . . . . . . . 137

6.2.2 Large-Scale SQP . . . . . . . . . . . . . . . . . . . . . . 144

6.2.3 Extensions of SQP Methods . . . . . . . . . . . . . . . . 148

6.3 Interior Point Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 151

6.3.1 Solution of the Primal-Dual Equations . . . . . . . . . . 154

6.3.2 A Line Search Filter Method . . . . . . . . . . . . . . . . 155

6.3.3 Globalization with Trust Region Methods . . . . . . . . . 158

6.4 Nested Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

6.4.1 Gradient Projection Methods for Bound Constrained

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 164

6.4.2 Linearly Constrained Augmented Lagrangian . . . . . . . 167

6.5 Nonlinear Programming Codes . . . . . . . . . . . . . . . . . . . . . 168

6.5.1 SQP Codes . . . . . . . . . . . . . . . . . . . . . . . . . 169

6.5.2 Interior Point NLP Codes . . . . . . . . . . . . . . . . . 170

6.5.3 Nested and Gradient Projection NLP Codes . . . . . . . . 171

6.5.4 Performance Trends for NLP Codes . . . . . . . . . . . . 171

6.6 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 175

6.7 Notes for Further Reading . . . . . . . . . . . . . . . . . . . . . . . . 176

6.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

7 Steady State Process Optimization 181

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

7.2 Optimization of Process Flowsheets . . . . . . . . . . . . . . . . . . . 183

7.2.1 Importance of Accurate Derivatives . . . . . . . . . . . 188

7.2.2 Ammonia Process Optimization . . . . . . . . . . . . . . 191

7.3 Equation-Oriented Formulation of Optimization Models . . . . . . . . 193

7.3.1 Reformulation of the Williams–Otto Optimization

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 196

7.4 Real-Time Optimization . . . . . . . . . . . . . . . . . . . . . . . . . 200

7.4.1 Equation-Oriented RTO Models . . . . . . . . . . . . . . 201

7.4.2 Case Study of Hydrocracker Fractionation Plant . . . . . 203

7.5 Equation-Oriented Models with Many Degrees of Freedom . . . . . . 206

7.6 Summary and Notes for Further Reading . . . . . . . . . . . . . . . . 209

7.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

8 Introduction to Dynamic Process Optimization 213

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

8.2 Dynamic Systems and Optimization Problems . . . . . . . . . . . . . 214

8.3 Optimality Conditions for Optimal Control Problems . . . . . . . . . 220

8.3.1 Optimal Control without Inequalities . . . . . . . . . . . 223

8.3.2 Optimal Control with Inequality Constraints . . . . . . . 225

8.4 Handling Path Constraints . . . . . . . . . . . . . . . . . . . . . . . . 232

8.4.1 Treatment of Equality Path Constraints . . . . . . . . . . 232

8.4.2 Treatment of State Path Inequalities . . . . . . . . . . . . 237

8.5 Singular Control Problems . . . . . . . . . . . . . . . . . . . . . . . 239

8.6 Numerical Methods Based on NLP Solvers . . . . . . . . . . . . . . . 243

book_tem

2010/7/27

page x

✐

✐

✐

✐

✐

✐

✐

✐

x Contents

8.7 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 246

8.8 Notes for Further Reading . . . . . . . . . . . . . . . . . . . . . . . . 247

8.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

9 Dynamic Optimization Methods with Embedded DAE Solvers 251

9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

9.2 DAE Solvers for Initial Value Problems . . . . . . . . . . . . . . . . . 253

9.2.1 Runge–Kutta Methods . . . . . . . . . . . . . . . . . . . 255

9.2.2 Linear Multistep Methods . . . . . . . . . . . . . . . . . 256

9.2.3 Extension of Methods to DAEs . . . . . . . . . . . . . . 259

9.3 Sensitivity Strategies for Dynamic Optimization . . . . . . . . . . . . 260

9.3.1 Direct Sensitivity Calculations . . . . . . . . . . . . . . 261

9.3.2 Adjoint Sensitivity Calculations . . . . . . . . . . . . . . 262

9.3.3 Evolution to Optimal Control Problems . . . . . . . . . . 265

9.4 Multiple Shooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271

9.4.1 Dichotomy of Boundary Value Problems . . . . . . . . . 273

9.5 Dynamic Optimization Case Study . . . . . . . . . . . . . . . . . . . 276

9.6 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 282

9.7 Notes for Further Reading . . . . . . . . . . . . . . . . . . . . . . . . 283

9.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284

10 Simultaneous Methods for Dynamic Optimization 287

10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

10.2 Derivation of Collocation Methods . . . . . . . . . . . . . . . . . . . 288

10.2.1 Polynomial Representation for ODE Solutions . . . . . . 289

10.2.2 Collocation with Orthogonal Polynomials . . . . . . . . 290

10.3 NLP Formulations and Solution . . . . . . . . . . . . . . . . . . . . . 295

10.3.1 Treatment of Finite Elements . . . . . . . . . . . . . . . 296

10.3.2 Treatment of Unstable Dynamic Systems . . . . . . . . . 299

10.3.3 Large-Scale NLP Solution Strategies . . . . . . . . . . . 302

10.3.4 Parameter Estimation for Low-Density Polyethylene

Reactors . . . . . . . . . . . . . . . . . . . . . . . . . . 304

10.4 Convergence Properties of Simultaneous Approach . . . . . . . . . . 309

10.4.1 Optimality with Gauss–Legendre Collocation . . . . . . . 312

10.4.2 Optimality with Radau Collocation . . . . . . . . . . . . 313

10.4.3 Convergence Orders for NLP Solutions . . . . . . . . . . 314

10.4.4 Singular Optimal Controls . . . . . . . . . . . . . . . . . 315

10.4.5 High-Index Inequality Path Constraints . . . . . . . . . . 317

10.5 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 322

10.6 Notes for Further Reading . . . . . . . . . . . . . . . . . . . . . . . . 322

10.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323

11 Process Optimization with Complementarity Constraints 325

11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325

11.2 MPCC Properties and Formulations . . . . . . . . . . . . . . . . . . . 327

11.2.1 Solution Strategies . . . . . . . . . . . . . . . . . . . . . 331

11.2.2 Comparison of MPCC Formulations . . . . . . . . . . . 333

book_tem

2010/7/27

page xi

✐

✐

✐

✐

✐

✐

✐

✐

Contents xi

11.3 Complementary Models for Process Optimization . . . . . . . . . . . 336

11.4 Distillation Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . 343

11.4.1 Multicomponent Column Optimization with

Phase Changes . . . . . . . . . . . . . . . . . . . . . . . 343

11.4.2 Tray Optimization . . . . . . . . . . . . . . . . . . . . . 345

11.5 Optimization of Hybrid Dynamic Systems . . . . . . . . . . . . . . . 349

11.6 Dynamic MPCC Case Studies . . . . . . . . . . . . . . . . . . . . . . 352

11.6.1 Reformulation of a Differential Inclusion . . . . . . . . . 352

11.6.2 Cascading Tank Problem . . . . . . . . . . . . . . . . . 356

11.7 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 359

11.8 Notes for Further Reading . . . . . . . . . . . . . . . . . . . . . . . . 360

11.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361

Bibliography 363

Index 391

book_tem

2010/7/27

page xiii

✐

✐

✐

✐

✐

✐

✐

✐

Preface

Chemical engineering applications have been a source of challenging optimization problems

for over 50 years. For many chemical process systems, detailed steady state and dynamic

behavior can now be described by a rich set of detailed nonlinear models, and relatively small

changes in process design and operation can lead to significant improvements in efficiency,

product quality, environmental impact, and profitability. With these characteristics, it is not

surprising that systematic optimization strategies have played an important role in chemical

engineering practice. In particular, over the past 35 years, nonlinear programming (NLP)

has become an indispensable tool for the optimization of chemical processes. These tools

are now applied at research and process development stages, in the design stage, and in the

online operation of these processes. More recently, the scope of these applications is being

extended to cover more challenging, large-scale tasks including process control based on

the optimization of nonlinear dynamic models, as well as the incorporation of nonlinear

models into strategic planning functions.

Moreover, the ability to solve large-scale process optimization models cheaply, even

online, is aided by recent breakthroughs in nonlinear programming, including the devel￾opment of modern barrier methods, deeper understanding of line search and trust region

strategies to aid global convergence, efficient exploitation of second derivatives in algorith￾mic development, and the availability of recently developed and widely used NLP codes,

including those for barrier methods [81, 391, 404], sequential quadratic programming (SQP)

[161, 159], and reduced gradient methods [119, 245, 285]. Finally, the availability of op￾timization modeling environments, such as AIMMS, AMPL, and GAMS, as well as the

NEOS server, has made the formulation and solution of optimization accessible to a much

wider user base. All of these advances have a huge impact in addressing and solving pro￾cess engineering problems previously thought intractable. In addition to developments in

mathematical programming, research in process systems engineering has led to optimiza￾tion modeling formulations that leverage these algorithmic advances, with specific model

structure and characteristics that lead to more efficient solutions.

This text attempts to make these recent optimization advances accessible to engineers

and practitioners. Optimization texts for engineers usually fall into two categories. First,

excellent mathematical programming texts (e.g., [134, 162, 294, 100, 227]) emphasize fun￾damental properties and numerical analysis, but have few specific examples with relevance

to real-world applications, and are less accessible to practitioners. On the other hand, equally

good engineering texts (e.g., [122, 305, 332, 53]) emphasize applications with well-known

methods and codes, but often without their underlying fundamental properties. While their

approach is accessible and quite useful for engineers, these texts do not aid in a deeper un￾derstanding of the methods or provide extensions to tackle large-scale problems efficiently.

xiii

book_tem

2010/7/27

page xiv

✐

✐

✐

✐

✐

✐

✐

✐

xiv Preface

To address the modeling and solution of large-scale process optimization problems,

it is important for optimization practitioners to understand

• which NLP methods are best suited for specific applications,

• how large-scale problems should be formulated and what features should be empha￾sized, and

• how existing methods can be extended to exploit specific structures of large-scale

optimization models.

This book attempts to fill the gap between the math programming and engineering

texts. It provides a firm grounding in fundamental algorithmic properties but also with

relevance to real-world problem classes through case studies. In addressing an engineering

audience, it covers state-of-the-art gradient-based NLP methods, summarizes key character￾istics and advantages of these methods, and emphasizes the link between problem structure

and efficient methods. Finally, in addressing a broader audience of math programmers it

also deals with steady state and dynamic models derived from chemical engineering.

The book is written for an audience consisting of

• engineers (specifically chemical engineers) interested in understanding and applying

state-of-the-art NLP algorithms to specific applications,

• experts in mathematical optimization (in applied math and operations research) who

are interested in understanding process engineering problems and developing better

approaches to solving them, and

• researchers from both fields interested in developing better methods and problem

formulations for challenging engineering problems.

The book is suitable for a class on continuous variable optimization, but with an

emphasis on problem formulation and solution methods. It is intended as a text for graduate

and advanced undergraduate classes in engineering and is also suitable as an elective class for

students in applied mathematics and operations research. It is also intended as a reference

for practitioners in process optimization in the area of design and operations, as well as

researchers in process engineering and applied mathematics.

The text is organized into eleven chapters, and the structure follows that of a short

course taught to graduate students and industrial practitioners which has evolved over the

past 20 years. Included from the course are a number of problems and computer projects

which form useful exercises at the end of each chapter.

The eleven chapters follow sequentially and build on each other. Chapter 1 pro￾vides an overview of nonlinear programming applications in process engineering and sets

the motivation for the text. Chapter 2 defines basic concepts and properties for nonlinear

programming and focuses on fundamentals of unconstrained optimization. Chapter 3 then

develops Newton’s method for unconstrained optimization and discusses basic concepts

for globalization methods; this chapter also develops quasi-Newton methods and discusses

their characteristics.

Chapter 4 then follows with fundamental aspects of constrained optimization, building

on the concepts in Chapter 2. Algorithms for equality constrained optimization are derived

in Chapter 5 from a Newton perspective that builds on Chapter 3. Chapter 6 extends this