Statistical Analysis of Financial Data in S-Plus

Springer Texts in Statistics

Advisors:

George Casella Stephen Fienberg Ingram Olkin

This page intentionally left blank

Rene´ A. Carmona

Statistical Analysis of

Financial Data in S-Plus

With 144 Figures

Rene´ A. Carmona

Department of Statistics

University of Princeton

Princeton, NJ 08544-5263

USA

[email protected]

Editorial Board

George Casella Stephen Fienberg Ingram Olkin

Department of Statistics Department of Statistics Department of Statistics

University of Florida Carnegie Mellon University Stanford University

Gainesville, FL 32611-8545 Pittsburgh, PA 15213-3890 Stanford, CA 94305

USA USA USA

Library of Congress Cataloging-in-Publication Data

Carmona, R. (Rene´)

Statistical analysis of financial data in S-PLUS / Rene´ A. Carmona.

p. cm. — (Springer texts in statistics)

Based on the author’s lecture notes for a course at Princeton University.

Includes bibliographical references and index.

ISBN 0-387-20286-2 (alk. paper)

1. Finance—Mathematical models. 2. Finance—Econometric models. 3. S-Plus. I. Title.

II. Series.

HG106.C37 2003

332′01′51955—dc22 2003066218

ISBN 0-387-20286-2 Printed on acid-free paper.

 2004 Springer-Verlag New York, Inc.

All rights reserved. This work may not be translated or copied in whole or in part without the

written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York,

NY 10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use

in connection with any form of information storage and retrieval, electronic adaptation, computer

software, or by similar or dissimilar methodology now known or hereafter developed is forbidden.

The use in this publication of trade names, trademarks, service marks, and similar terms, even if

they are not identified as such, is not to be taken as an expression of opinion as to whether or not

they are subject to proprietary rights.

Printed in the United States of America. (MVY)

987654321 SPIN 10953280

Springer-Verlag is a part of Springer Science+Business Media

springeronline.com

To Chanel, Chelsea and Stephanie ´

This page intentionally left blank

Preface

This book grew out of lectures notes written for a one-semester junior statistics

course offered to the undergraduate students majoring in the Department of Oper￾ations Research and Financial Engineering at Princeton University. Tidbits of the

history of this course will shed light on the nature and spirit of the book.

The purpose of the course is to introduce the students to modern data analysis

with an emphasis on a domain of application that is of interest to most of them:

financial engineering. The prerequisites for this course are minimal, however it is

fair to say that all of the students have already taken a basic introductory statistics

course. Thus the elementary notions of random variables, expectation and correlation

are taken for granted, and earlier exposure to statistical inference (estimation, tests

and confidence intervals) is assumed. It is also expected that the students are familiar

with a minimum of linear algebra as well as vector and matrix calculus.

Because of my background, the course is both computational and mathematical

in nature. Most problems considered are formulated in a rigorous manner. Mathe￾matical facts are motivated by applications, stated precisely, justified at an intuitive

level, but essentially never proven rigorously. The emphasis is more on the relevance

of concepts and on the practical use of tools, rather than on their theoretical under￾pinnings.

I chose to illustrate concepts, manipulate data, build models, and implement es￾timation and prediction procedures in the S-Plus computer environment. For this

reason an introduction to S and S-Plus (reproduced in appendix) is offered at the

beginning of the course each semester, and many lectures are sprinkled with the S

commands needed to perform the analyses discussed in class. The first two incarna￾tions of this course were using S-Plus on Unix platforms and not all the students

were able to cope with the steep learning curve. Moreover, the two textbooks used

for the class did not seem to be of very much help to the students. So I decided

vii

viii PREFACE

to prepare lecture notes focused on the material covered in class, and to switch to

Windows in order to work with a friendlier implementation of S.

The present manuscript is a polished version of the class notes. It is divided

into three parts. Part I, Exploratory Data Analysis, reviews the most commonly used

methods of statistical data exploration. Part II, Regression, introduces the students to

modern regression with an emphasis on robustness and non-parametric techniques.

Part III, Time Series and State Space Models, is concerned with the theories of time

series and of state space models.

Contents

Part I is a patchwork of many exploratory data analysis techniques. It begins with

a discussion of various methods of density estimation, including histograms and ker￾nel density estimators. Since the emphasis of the course is on financial applications,

the notion of heavy tail is immediately showcased with examples. A good part of the

first chapter is concerned with the practical estimation of heavy tailed distributions,

their detection, their estimation and their simulation. We use the statistical concept

of percentile to introduce the notion of value-at-risk so important in the financial in￾dustry, and we demonstrate its use on a couple of illustrative examples. The second

chapter is concerned with multivariate distributions and the various concepts of de￾pendence. We study the classical correlation coefficients, but we also spend a good

amount of time understanding the notion of copula, and the important role it plays

when the marginal distributions have heavy tails. As in the univariate case, we learn

how to detect unusual dependencies, to estimate them, and to simulate them. We also

give a complete discussion of principal component analysis and illustrate its power

on two applications to fixed income markets.

Part II is concerned with regression, and it is naturally divided into two chap￾ters: the first devoted to parametric methods, and the second to non-parametric ones.

Chapter 3 deals with linear models and their applications. The notion of robust￾ness is introduced and examples are used to illustrate the differences between least

squares and least absolute deviations regressions. Applications of linear models in￾clude polynomial and more general nonlinear regressions. We use financial exam￾ples throughout and we analyze the term structure of interest rates in detail. Chapter

4 is concerned with nonparametric regression. We compare the properties of data

smoothers for univariate data, and we analyze in detail the multivariate kernel re￾gression and density estimation for intermediate values of the dimension. For large

values of the dimension we consider projection pursuit. To illustrate, we analyze

energy forward curves and intra-day tick data on S&P 500 futures contracts. The

last part of this chapter is devoted to a demonstration of the use of semi-parametric

and nonparametric methods in option pricing. We review the derivation of the clas￾sical Black-Scholes pricing formula, we illustrate its shortcomings, and we walk the

reader through the implementation of modern regression techniques as pricing al￾ternatives. The actual implementations are done on liquid S&P 500 futures option

data.

Preface ix

The first chapter of Part III is devoted to the classical linear models for time

series, and to the idiosyncrasies of the S-Plus objects and methods needed to fit

them. We discuss auto regressive and moving-average models, and we give examples

of their use in practice. The main application of the material of this chapter is con￾cerned with the analysis of temperature data. Even if it may not appear to be much

of a financial application at first, we recast this analysis in the framework of finan￾cial risk management via a thorough discussion of the booming market of weather

derivatives. We give practical examples to illustrate the use of the statistical tech￾niques introduced in this chapter to the pricing of these new financial instruments.

In the following two chapters, we turn to the analysis of partially observed state

space systems. Chapter 6 deals with linear models and the classical Kalman filter.

For illustration purposes, we study two financial applications, one related to an ex￾tension of the CAPM model, and a second dealing with the analysis of quarterly

company earnings. Chapter 7 is devoted to the analysis of nonlinear time series. We

first consider the natural generalizations of the linear time series models and we pro￾vide an extensive review of the theory and the practice of the famous ARCH and

GARCH models. We also consider models from continuous time finance through

their discretized forms. A special section is devoted to the use of scenarios for eco￾nomic modeling. We concentrate on scenarios for a stock index and the short and

long interest rates. These scenarios are of crucial importance in risk management

where they are used as input to large stochastic optimization programs. Finally, we

revisit the theory presented in the case of partially observed linear systems, and we

extend the filtering paradigm to nonlinear systems with the help of recent advances

in Monte Carlo techniques. We give several applications of this material, including

to the estimation of stochastic volatility and commodity convenience yield.

Each chapter contains a problem section. Most problems are of a financial na￾ture. They are preceded with symbols
E ,
S , and/or
T to indicate if they are of

an empirical, simulation, and/or theoretical nature. Each chapter ends with a sec￾tion called Notes & Complements that includes bibliographic references which can

be used by readers interested in acquiring a deeper understanding of the topics of

that chapter. The book ends with two appendices as a suite of indexes. Appendix

A contains the introductory session set up to initiate the students to S-Plus at the

beginning of the course, and Appendix B gives information on how to download the

library EVANESCE and the home-grown functions used in the text, as well as the

data sets used in the text and in the problems.

The code together with the data used in the text can be downloaded from the

author web page at the URL:

http://www.princeton.edu/˜rcarmona/safd/

This web page will be updated regularly, and corrections, complements, new data

sets, updates, etc., will be posted frequently.

Acknowledgments

First and foremost, I want to thank all the students who suffered painfully through

early versions of the course, and primitive drafts of the lecture notes. Their patience

x PREFACE

and their encouragement helped me persevere, and over the years, figure out the

recipe for the form and the content of the course. I feel guilty to have used them as

guinea pigs, but I am glad that the process finally converged. My Chairman Erhan

C¸ inlar trusted me with this course, and gave me total freedom to reshape it. What

seemed like foolishness to some, may have been great insight with what needed to

be done. I am grateful for his confidence and his relentless encouragements.

My interest in computational statistics was sparked over twenty years ago by two

dear friends: Anestis Antoniadis and Jacques Berruyer. Time and distance pulled us

apart, but what their collaboration taught me will remain with me forever. The first

part of the book would not have been possible without Julia Morrison’s contribu￾tion. It was a real pleasure to work with her on the development of the S-Plus li￾brary EVANESCE. I am very grateful for this experience. I also want to thank Yacine

Ait-Sahalia for enlightening discussions on nonparametric asset pricing. I am also

indebted to Cliona Golden for a superb job proofreading an early version of the

manuscript. Finally, I want to thank my wife Debra for tolerating my insane working

habits, and my three wonderful daughters Stephanie, Chelsea and Chanel for their

limitless patience and unconditional love. I may not deserve it, but I sure am proud

of it.

Rene Carmona ´

Princeton, N.J.

November 4, 2003

Contents

Part I DATA EXPLORATION, ESTIMATION AND SIMULATION

1 UNIVARIATE EXPLORATORY DATA ANALYSIS ............... 3

1.1 Data, Random Variables and Their Distributions . . ............... 3

1.1.1 The PCS Data ....................................... 4

1.1.2 The S&P 500 Index and Financial Returns ............... 5

1.1.3 Random Variables and Their Distributions ............... 7

1.1.4 Examples of Probability Distribution Families . . .......... 8

1.2 First Exploratory Data Analysis Tools . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.2.1 Random Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.2.2 Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.3 More Nonparametric Density Estimation . . . . . . . . . . . . . . . . . . . . . . . 16

1.3.1 Kernel Density Estimation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1.3.2 Comparison with the Histogram . . . . . . . . . . . . . . . . . . . . . . . . 19

1.3.3 S&P Daily Returns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1.3.4 Importance of the Choice of the Bandwidth . . . . . . . . . . . . . . 22

1.4 Quantiles and Q-Q Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

1.4.1 Understanding the Meaning of Q-Q Plots . . . . . . . . . . . . . . . . 24

1.4.2 Value at Risk and Expected Shortfall . . . . . . . . . . . . . . . . . . . . 25

1.5 Estimation from Empirical Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

1.5.1 The Empirical Distribution Function . . . . . . . . . . . . . . . . . . . . 28

1.5.2 Order Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

1.5.3 Empirical Q-Q Plots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

1.6 Random Generators and Monte Carlo Samples . . . . . . . . . . . . . . . . . . 31

1.7 Extremes and Heavy Tail Distributions. . . . . . . . . . . . . . . . . . . . . . . . . 35

1.7.1 S&P Daily Returns, Once More . . . . . . . . . . . . . . . . . . . . . . . . 35

1.7.2 The Example of the PCS Index . . . . . . . . . . . . . . . . . . . . . . . . . 37

1.7.3 The Example of the Weekly S&P Returns . . . . . . . . . . . . . . . . 41

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Notes & Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

xii CONTENTS

2 MULTIVARIATE DATA EXPLORATION ....................... 49

2.1 Multivariate Data and First Measure of Dependence . . . . . . . . . . . . . 49

2.1.1 Density Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

2.1.2 The Correlation Coefficient . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

2.2 The Multivariate Normal Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 56

2.2.1 Simulation of Random Samples . . . . . . . . . . . . . . . . . . . . . . . . 57

2.2.2 The Bivariate Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.2.3 A Simulation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

2.2.4 Let’s Have Some Coffee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

2.2.5 Is the Joint Distribution Normal? . . . . . . . . . . . . . . . . . . . . . . . 62

2.3 Marginals and More Measures of Dependence . . . . . . . . . . . . . . . . . . 63

2.3.1 Estimation of the Coffee Log-Return Distributions . . . . . . . . 64

2.3.2 More Measures of Dependence . . . . . . . . . . . . . . . . . . . . . . . . . 68

2.4 Copulas and Random Simulations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

2.4.1 Copulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

2.4.2 First Examples of Copula Families . . . . . . . . . . . . . . . . . . . . . . 72

2.4.3 Copulas and General Bivariate Distributions. . . . . . . . . . . . . . 74

2.4.4 Fitting Copulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

2.4.5 Monte Carlo Simulations with Copulas . . . . . . . . . . . . . . . . . . 77

2.4.6 A Risk Management Example . . . . . . . . . . . . . . . . . . . . . . . . . 80

2.5 Principal Component Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

2.5.1 Identification of the Principal Components of a Data Set . . . 84

2.5.2 PCA with S-Plus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

2.5.3 Effective Dimension of the Space of Yield Curves . . . . . . . . . 87

2.5.4 Swap Rate Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Appendix 1: Calculus with Random Vectors and Matrices . . . . . . . . . . . . . 92

Appendix 2: Families of Copulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Notes & Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Part II REGRESSION

3 PARAMETRIC REGRESSION ................................ 105

3.1 Simple Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

3.1.1 Getting the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

3.1.2 First Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

3.1.3 Regression Set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

3.1.4 Simple Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

3.1.5 Cost Minimizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

3.1.6 Regression as a Minimization Problem . . . . . . . . . . . . . . . . . . 114

3.2 Regression for Prediction & Sensitivities . . . . . . . . . . . . . . . . . . . . . . . 116

3.2.1 Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

3.2.2 Introductory Discussion of Sensitivity and Robustness . . . . . 118

Contents xiii

3.2.3 Comparing L2 and L1 Regressions . . . . . . . . . . . . . . . . . . . . . 119

3.2.4 Taking Another Look at the Coffee Data . . . . . . . . . . . . . . . . . 121

3.3 Smoothing versus Distribution Theory . . . . . . . . . . . . . . . . . . . . . . . . . 123

3.3.1 Regression and Conditional Expectation . . . . . . . . . . . . . . . . . 123

3.3.2 Maximum Likelihood Approach . . . . . . . . . . . . . . . . . . . . . . . . 124

3.4 Multiple Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

3.4.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

3.4.2 The S-Plus Function lm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

3.4.3 R2 as a Regression Diagnostic . . . . . . . . . . . . . . . . . . . . . . . . . 131

3.5 Matrix Formulation and Linear Models . . . . . . . . . . . . . . . . . . . . . . . . 133

3.5.1 Linear Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

3.5.2 Least Squares (Linear) Regression Revisited . . . . . . . . . . . . . 134

3.5.3 First Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

3.5.4 Testing the CAPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

3.6 Polynomial Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

3.6.1 Polynomial Regression as a Linear Model . . . . . . . . . . . . . . . 146

3.6.2 Example of S-Plus Commands . . . . . . . . . . . . . . . . . . . . . . . 146

3.6.3 Important Remark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

3.6.4 Prediction with Polynomial Regression . . . . . . . . . . . . . . . . . . 148

3.6.5 Choice of the Degree p . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

3.7 Nonlinear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

3.8 Term Structure of Interest Rates: A Crash Course . . . . . . . . . . . . . . . . 154

3.9 Parametric Yield Curve Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

3.9.1 Estimation Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

3.9.2 Practical Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

3.9.3 S-Plus Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

3.9.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Appendix: Cautionary Notes on Some S-Plus Idiosyncracies . . . . . . . . . 166

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

Notes & Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

4 LOCAL & NONPARAMETRIC REGRESSION .................. 175

4.1 Review of the Regression Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

4.2 Natural Splines as Local Smoothers . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

4.3 Nonparametric Scatterplot Smoothers. . . . . . . . . . . . . . . . . . . . . . . . . . 178

4.3.1 Smoothing Splines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

4.3.2 Locally Weighted Regression . . . . . . . . . . . . . . . . . . . . . . . . . . 181

4.3.3 A Robust Smoother . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

4.3.4 The Super Smoother. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

4.3.5 The Kernel Smoother . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

4.4 More Yield Curve Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

4.4.1 A First Estimation Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

4.4.2 A Direct Application of Smoothing Splines . . . . . . . . . . . . . . 188

4.4.3 US and Japanese Instantaneous Forward Rates . . . . . . . . . . . . 188

xiv CONTENTS

4.5 Multivariate Kernel Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

4.5.1 Running the Kernel in S-Plus . . . . . . . . . . . . . . . . . . . . . . . . 192

4.5.2 An Example Involving the June 1998 S&P Futures Contract 193

4.6 Projection Pursuit Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

4.6.1 The S-Plus Function ppreg . . . . . . . . . . . . . . . . . . . . . . . . . 198

4.6.2 ppreg Prediction of the S&P Indicators. . . . . . . . . . . . . . . . . 200

4.7 Nonparametric Option Pricing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

4.7.1 Generalities on Option Pricing . . . . . . . . . . . . . . . . . . . . . . . . . 205

4.7.2 Nonparametric Pricing Alternatives . . . . . . . . . . . . . . . . . . . . . 212

4.7.3 Description of the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

4.7.4 The Actual Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

4.7.5 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

Appendix: Kernel Density Estimation & Kernel Regression . . . . . . . . . . . . 222

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

Notes & Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Part III TIME SERIES & STATE SPACE MODELS

5 TIME SERIES MODELS: AR, MA, ARMA, & ALL THAT......... 239

5.1 Notation and First Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

5.1.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

5.1.2 Regular Time Series and Signals . . . . . . . . . . . . . . . . . . . . . . . 240

5.1.3 Calendar and Irregular Time Series . . . . . . . . . . . . . . . . . . . . . 241

5.1.4 Example of Daily S&P 500 Futures Contracts . . . . . . . . . . . . 243

5.2 High Frequency Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

5.2.1 TimeDate Manipulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

5.3 Time Dependent Statistics and Stationarity . . . . . . . . . . . . . . . . . . . . . 253

5.3.1 Statistical Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

5.3.2 The Notion of Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

5.3.3 The Search for Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

5.3.4 The Example of the CO2 Concentrations . . . . . . . . . . . . . . . . 261

5.4 First Examples of Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

5.4.1 White Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

5.4.2 Random Walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

5.4.3 Auto Regressive Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . 268

5.4.4 Moving Average Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . 272

5.4.5 Using the Backward Shift Operator B . . . . . . . . . . . . . . . . . . . 275

5.4.6 Linear Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

5.4.7 Causality, Stationarity and Invertibility . . . . . . . . . . . . . . . . . . 277

5.4.8 ARMA Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

5.4.9 ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

5.5 Fitting Models to Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

5.5.1 Practical Steps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282