Bayesian Econometrics

Bayesian Econometrics

This Page Intentionally Left Blank

Bayesian Econometrics

Gary Koop

Department of Economics

University of Glasgow

Copyright
c 2003 John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester,

West Sussex PO19 8SQ, England

Telephone (C44) 1243 779777

Email (for orders and customer service enquiries): [email protected]

Visit our Home Page on www.wileyeurope.com or www.wiley.com

All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or

transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning

or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the

terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London

W1T 4LP, UK, without the permission in writing of the Publisher. Requests to the Publisher should

be addressed to the Permissions Department, John Wiley & Sons Ltd, The Atrium, Southern Gate,

Chichester, West Sussex PO19 8SQ, England, or emailed to [email protected], or faxed to (C44)

1243 770620.

This publication is designed to provide accurate and authoritative information in regard to the subject

matter covered. It is sold on the understanding that the Publisher is not engaged in rendering

professional services. If professional advice or other expert assistance is required, the services of a

competent professional should be sought.

Other Wiley Editorial Offices

John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA

Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA

Wiley-VCH Verlag GmbH, Boschstr. 12, D-69469 Weinheim, Germany

John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia

John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809

John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1

Wiley also publishes its books in a variety of electronic formats. Some content that appears

in print may not be available in electronic books.

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library

ISBN 0-470-84567-8

Typeset in 10/12pt Times by Laserwords Private Limited, Chennai, India

Printed and bound in Great Britain by Biddles Ltd, Guildford and King’s Lynn

This book is printed on acid-free paper responsibly manufactured from sustainable forestry

in which at least two trees are planted for each one used for paper production.

To Lise

Contents

Preface xiii

1 An Overview of Bayesian Econometrics 1

1.1 Bayesian Theory 1

1.2 Bayesian Computation 6

1.3 Bayesian Computer Software 10

1.4 Summary 11

1.5 Exercises 11

2 The Normal Linear Regression Model with Natural Conjugate

Prior and a Single Explanatory Variable 15

2.1 Introduction 15

2.2 The Likelihood Function 16

2.3 The Prior 18

2.4 The Posterior 19

2.5 Model Comparison 23

2.6 Prediction 26

2.7 Empirical Illustration 28

2.8 Summary 31

2.9 Exercises 31

viii Contents

3 The Normal Linear Regression Model with Natural Conjugate

Prior and Many Explanatory Variables 33

3.1 Introduction 33

3.2 The Linear Regression Model in Matrix Notation 34

3.3 The Likelihood Function 35

3.4 The Prior 36

3.5 The Posterior 36

3.6 Model Comparison 38

3.7 Prediction 45

3.8 Computational Methods: Monte Carlo Integration 46

3.9 Empirical Illustration 47

3.10 Summary 54

3.11 Exercises 54

4 The Normal Linear Regression Model with Other Priors 59

4.1 Introduction 59

4.2 The Normal Linear Regression Model with Independent

Normal-Gamma Prior 60

4.3 The Normal Linear Regression Model Subject

to Inequality Constraints 77

4.4 Summary 85

4.5 Exercises 86

5 The Nonlinear Regression Model 89

5.1 Introduction 89

5.2 The Likelihood Function 91

5.3 The Prior 91

5.4 The Posterior 91

5.5 Bayesian Computation: The Metropolis–Hastings Algorithm 92

5.6 A Measure of Model Fit: The Posterior Predictive P-Value 100

5.7 Model Comparison: The Gelfand–Dey Method 104

5.8 Prediction 106

5.9 Empirical Illustration 107

5.10 Summary 112

5.11 Exercises 113

Contents ix

6 The Linear Regression Model with General Error

Covariance Matrix 117

6.1 Introduction 117

6.2 The Model with General
118

6.3 Heteroskedasticity of Known Form 121

6.4 Heteroskedasticity of an Unknown Form: Student-t Errors 124

6.5 Autocorrelated Errors 130

6.6 The Seemingly Unrelated Regressions Model 137

6.7 Summary 143

6.8 Exercises 144

7 The Linear Regression Model with Panel Data 147

7.1 Introduction 147

7.2 The Pooled Model 148

7.3 Individual Effects Models 149

7.4 The Random Coefficients Model 155

7.5 Model Comparison: The Chib Method of Marginal

Likelihood Calculation 157

7.6 Empirical Illustration 162

7.7 Efficiency Analysis and the Stochastic Frontier Model 168

7.8 Extensions 176

7.9 Summary 177

7.10 Exercises 177

8 Introduction to Time Series: State Space Models 181

8.1 Introduction 181

8.2 The Local Level Model 183

8.3 A General State Space Model 194

8.4 Extensions 202

8.5 Summary 205

8.6 Exercises 206

x Contents

9 Qualitative and Limited Dependent Variable Models 209

9.1 Introduction 209

9.2 Overview: Univariate Models for Qualitative and Limited

Dependent Variables 211

9.3 The Tobit Model 212

9.4 The Probit Model 214

9.5 The Ordered Probit Model 218

9.6 The Multinomial Probit Model 221

9.7 Extensions of the Probit Models 229

9.8 Other Extensions 230

9.9 Summary 232

9.10 Exercises 232

10 Flexible Models: Nonparametric and Semiparametric Methods 235

10.1 Introduction 235

10.2 Bayesian Non- and Semiparametric Regression 236

10.3 Mixtures of Normals Models 252

10.4 Extensions and Alternative Approaches 262

10.5 Summary 263

10.6 Exercises 263

11 Bayesian Model Averaging 265

11.1 Introduction 265

11.2 Bayesian Model Averaging in the Normal

Linear Regression Model 266

11.3 Extensions 278

11.4 Summary 280

11.5 Exercises 280

12 Other Models, Methods and Issues 283

12.1 Introduction 283

12.2 Other Methods 284

12.3 Other Issues 288

12.4 Other Models 292

12.5 Summary 308

Contents xi

Appendix A: Introduction to Matrix Algebra 311

Appendix B: Introduction to Probability and Statistics 317

B.1 Basic Concepts of Probability 317

B.2 Common Probability Distributions 324

B.3 Introduction to Some Concepts in Sampling Theory 330

B.4 Other Useful Theorems 333

Bibliography 335

Index 347

This Page Intentionally Left Blank

Preface

Bayesian methods are increasingly becoming attractive to researchers in many

fields. Econometrics, however, is a field in which Bayesian methods have had

relatively less influence. A key reason for this absence is the lack of a suitable

advanced undergraduate or graduate level textbook. Existing Bayesian books are

either out-dated, and hence do not cover the computational advances that have

revolutionized the field of Bayesian econometrics since the late 1980s, or do not

provide the broad coverage necessary for the student interested in empirical work

applying Bayesian methods. For instance, Arnold Zellner’s seminal Bayesian

econometrics book (Zellner, 1971) was published in 1971. Dale Poirier’s influ￾ential book (Poirier, 1995) focuses on the methodology and statistical theory

underlying Bayesian and frequentist methods, but does not discuss models used

by applied economists beyond regression. Other important Bayesian books, such

as Bauwens, Lubrano and Richard (1999), deal only with particular areas of

econometrics (e.g. time series models). In writing this book, my aim has been

to fill the gap in the existing set of Bayesian textbooks, and create a Bayesian

counterpart to the many popular non-Bayesian econometric textbooks now avail￾able (e.g. Greene, 1995). That is, my aim has been to write a book that covers a

wide range of models and prepares the student to undertake applied work using

Bayesian methods.

This book is intended to be accessible to students with no prior training in

econometrics, and only a single course in mathematics (e.g. basic calculus). Stu￾dents will find a previous undergraduate course in probability and statistics useful;

however Appendix B offers a brief introduction to these topics for those without

the prerequisite background. Throughout the book, I have tried to keep the level

of mathematical sophistication reasonably low. In contrast to other Bayesian and

comparable frequentist textbooks, I have included more computer-related mate￾rial. Modern Bayesian econometrics relies heavily on the computer, and devel￾oping some basic programming skills is essential for the applied Bayesian. The

required level of computer programming skills is not that high, but I expect that

this aspect of Bayesian econometrics might be most unfamiliar to the student

xiv Preface

brought up in the world of spreadsheets and click-and-press computer packages.

Accordingly, in addition to discussing computation in detail in the book itself, the

website associated with the book contains MATLAB programs for performing

Bayesian analysis in a wide variety of models. In general, the focus of the book

is on application rather than theory. Hence, I expect that the applied economist

interested in using Bayesian methods will find it more useful than the theoretical

econometrician.

I would like to thank the numerous people (some anonymous) who gave me

helpful comments at various stages in the writing of this book, including: Luc

Bauwens, Jeff Dorfman, David Edgerton, John Geweke, Bill Griffiths, Frank

Kleibergen, Tony Lancaster, Jim LeSage, Michel Lubrano, Brendan McCabe,

Bill McCausland, Richard Paap, Rodney Strachan, and Arnold Zellner. In addi￾tion, I would like to thank Steve Hardman for his expert editorial advice. All

I know about Bayesian econometrics comes through my work with a series of

exceptional co-authors: Carmen Fernandez, Henk Hoek, Eduardo Ley, Kai Li,

Jacek Osiewalski, Dale Poirier, Simon Potter, Mark Steel, Justin Tobias, and

Herman van Dijk. Of these, I would like to thank Mark Steel, in particular, for

patiently responding to my numerous questions about Bayesian methodology and

requests for citations of relevant papers. Finally, I wish to express my sincere

gratitude to Dale Poirier, for his constant support throughout my professional

life, from teacher and PhD supervisor, to valued co-author and friend.

1

An Overview of Bayesian

Econometrics

1.1 BAYESIAN THEORY

Bayesian econometrics is based on a few simple rules of probability. This is

one of the chief advantages of the Bayesian approach. All of the things that an

econometrician would wish to do, such as estimate the parameters of a model,

compare different models or obtain predictions from a model, involve the same

rules of probability. Bayesian methods are, thus, universal and can be used any

time a researcher is interested in using data to learn about a phenomenon.

To motivate the simplicity of the Bayesian approach, let us consider two ran￾dom variables, A and B.

1 The rules of probability imply:

p.A; B/ D p.AjB/p.B/

where p.A; B/ is the joint probability2 of A and B occurring, p.AjB/ is the

probability of A occurring conditional on B having occurred (i.e. the conditional

probability of A given B), and p.B/ is the marginal probability of B. Alterna￾tively, we can reverse the roles of A and B and find an expression for the joint

probability of A and B:

p.A; B/ D p.BjA/p.B/

Equating these two expressions for p.A; B/ and rearranging provides us with

Bayes’ rule, which lies at the heart of Bayesian econometrics:

p.BjA/ D p.AjB/p.B/

p.A/ (1.1)

1This chapter assumes the reader knows the basic rules of probability. Appendix B provides a

brief introduction to probability for the reader who does not have such a background or would like

a reminder of this material. 2We are being slightly sloppy with terminology here and in the following material in that we

should always say ‘probability density’ if the random variable is continuous and ‘probability function’

if the random variable is discrete (see Appendix B). For simplicity, we simply drop the word ‘density’

or ‘function’.