Cases, Numbers, Models: International Relations Research Methods

Cases, Numbers, Models:

International Relations Research

Methods

edited by

Detlef F. Sprinz and Yael Wolinsky

REVISED, November 2002

2

Contents

List of Tables 4

List of Graphs 5

1 Introduction: Methodology in International Relations Research,

Detlef F. Sprinz and Yael Wolinsky 6

I CASE STUDY METHODS

2 Case Study Methods: Design, Use, and Comparative Advantages,

Andrew Bennett 27

3 Case Study Methods in International Political Economy,

John S. Odell 65

4 Oualitative Research Design in International Environmental Policy,

Ronald Mitchell and Thomas Bernauer

91

5 Case Study Methods in International Security Studies,

Arie M. Kacowicz 119

II QUANTITATIVE METHODS

6 Empirical-Quantitative Approaches to the Study of International

Relations, Bear F. Braumoeller and Anne E. Sartori 139

7 Quantitative Approaches to the International Political Economy,

Edward D. Mansfield 164

8 Quantitative Analysis of International Environmental Policy,

Detlef F. Sprinz 190

3

9 Testing Theories of International Conflict: Questions of Research

Design for Statistical Analysis, Paul Huth and Todd Allee 207

III FORMAL METHODS

10 Formal Models of International Politics, Duncan Snidal 242

11 International Political Economy and Formal Models of Political

Economy, Helen Milner 284

12 Consumption, Production and Markets: Applications of

Microeconomics to International Politics, John A.C. Conybeare 311

13 Game Theory and International Environmental Policy,

D. Marc Kilgour and Yael Wolinsky 339

14 Formal Analysis and Security Studies: The Art of Shaker

Modeling, Andrew Kydd 370

15 Conclusion, Detlef F. Sprinz and Yael Wolinsky 396

4

List of Tables

Chapter 1: Introduction

Table 1: Organization of the Book and Chapter Authors

Chapter 2: Bennett

Table 1: Equivalent Terms for Types of Case Studies

Chapter 4: Bernauer and Mitchell

Table 1: Criteria for High Quality QER Research

Chapter 6: Braumoeller and Sartori

Table 1. Relationship between Y and X from Anscombe (1973)

Table 2: A significant regression coefficient with 50,000 observations

5

List of Figures

Chapter 1. Introduction

Figure 1: Trends in Methodology of International Relations Research

Chapter 6: Braumoeller and Sartori

Figure 1. Four datasets consistent with results in Table 1.

Figure 2: Data summarized in Table 2.

Chapter 8: Sprinz

Figure 1: Measuring Regime Effectiveness

Chapter 9: Huth and Allee

Figure 1. The Evolution of International Disputes

Figure 2: The Dispute Initiation Stage

Figure 3: The Challenge the Status Quo Stage

Figure 4: The Negotiations Stage

Figure 5: The military Escalation Stage

Chapter 10: Snidal

Figure 1a. Stable Richardson Arms Race

Figure 1b. Unstable Richardson Arms Race

Figure 2: Arms Race as a Prisoners’ Dilemma

Figure 3: Multiple Cooperative Equilibria

Figure 4: Extensive Form of Trust Game

Figure 5: Normal Form of Trust Game

Figure 6: Normal Form Threat Game

Figure 7: Extensive Form Threat Game

Chapter 12: Conybeare.

Figure 1: War and Expected Utility

Chapter 13: Kilgour and Wolinsky

Figure 1: Asymmetric Deterrence Game (adapted from Zagare and Kilgour 2000)

Figure 2: Perfect Bayesian Equilibria of Asymmetric Deterrence Game (adapted

from Zagare and Kilgour 2000)

Chapter 14: Kydd

Figure 1: The Bargaining Range

Figure 2: The Game Tree (Complete Information)

Figure 3: The New Bargaining Range

Figure 4: The Game Tree (Incomplete Information)

Figure 5: War in the Incomplete Information Bargaining Game

6

1 Introduction: Methodology in International Relations

Research1

Detlef F. Sprinz and Yael Wolinsky

Studies of International Relations try to explain a broad range of political interactions

among countries, societies, and organizations. From the study of war and peace, to

exploring economic cooperation and environmental conflict, furthering a

methodologically-guided understanding of international politics requires a systematic

approach to identifying fundamental processes and forces of change. With the growing

importance of economic interdependence and the profound changes in the international

system during the last few decades, the analysis of International Relations has expanded

in three main directions. First, scholars have ventured into new issue areas of

International Relations including international environmental politics, international

ethics, and globalization. Second, new methods have emerged within the study of

International Relations (e.g., two-level game analysis and spatial analysis), and the scope

of methodologies has substantially broadened over the past decades to include greater use

of rational choice analysis and statistical methods. Finally, aiming at a more precise

understanding of complex interactions among players at the international level, students

of the field have developed greater specialization within both substantive sub-fields and

methodological approaches. These developments have undoubtedly enriched

International Relations research and have drawn more attention to additional areas of

study such as compliance with international treaties and the explanation of civil wars.

At the same time the combination of new themes of research, broadening scope of

methodologies, and greater specialization within sub-fields has overshadowed common

methodological concerns of students of the field. While general courses on research

methodologies have become part of the standard curriculum in Political Science at both

the advanced undergraduate level and the graduate level, serious discussions of

methodological problems common to the analysis of International Relations are still

7

comparatively rare. This volume aims to fill this gap by presenting theoretical and

empirical studies that deal with central methodological issues in the study of International

Relations while also examining recent debates in the field. The authors explain the

application of three different methods of research to the study of International Relations:

case studies, quantitative analyses, and formal methods2

. The use of these methods is

evaluated in the context of different substantive sub-fields of International Relations (e.g.

international security, international political economy). The authors also engage in a

discussion of how the different methods have influenced central debates in International

Relations such as whether and why democratic countries are unlikely to fight each other,

and what determines the effectiveness of international regimes.

Following many years of debate on which method has the leading edge in

studying International Relations, this book is written in a very different spirit. It argues

that enough knowledge has now been accumulated to foster a serious dialogue across

different methodological approaches and sub-fields. Such a dialogue will generate a

better understanding of the advantages and limits of different methods and thus could

lead to more fruitful research on International Relations.

Recently, leading scholars of the field have elaborated upon the need for a more

robust discourse on methodology in International Relations. In particular, two former

presidents of the International Studies Association, Michael Brecher and Bruce Bueno de

Mesquita, have attempted to motivate such a dialogue. In his 1999 Presidential Address

to the International Studies Association, Brecher states that the field must move away

from intolerance of competing paradigms, models, methods and findings. He emphasizes

the importance of both cumulation of knowledge and research that bridges across

methods (Brecher 1999). Bueno de Mesquita outlines the comparative advantages of the

three major methods used in international relations (case study, quantitative, and formal

methods) and suggests that “[s]cientific progress is bolstered by and may in fact require

the application of all three methods” (Bueno de Mesquita 2002).

For decades International Relations scholars have debated methodological issues

such as the level of analysis dilemma: Should policy and politics be explained by

focusing on decision makers as individuals, the state organizations involved, or factors at

the international system level? And while such issues are still important, the

8

accumulation of methodologically oriented research now allows for a more integrative

approach to the study of International Relations. Indeed, the growing interest in diverse

aspects of international politics in both academia and public policy may be enhanced by

greater discourse among scholars in the field.

This book offers a unique combination of an introduction to the major strands of

methodology and an examination of their application in dominant sub-fields of

International Relations. Throughout the book the emphasis is on the merits of employing

case study, quantitative analysis, and formal methods in International Relations research

and the trade-offs involved in using each method. Subsequent to the introduction to each

method, separate chapters illustrate the application of the particular method in three sub￾fields of International Relations: international political economy, international

environmental politics, and international security. These sub-fields were chosen for

several reasons.

International security has been at the heart of the traditional study of International

Relations and still is a core sub-field. Many of the main intellectual challenges of

scholars in the field center on international security, beginning with the study of war and

its causes at the individual (leader), state, and international system levels. Over the past

half century, scholars have broadened the range of questions to include the implications

of nuclear deterrence for the stability of the international system, causes of civil wars,

how and why international alliances form, and whether and why democratic countries are

less likely to go to war against each other (the democratic peace thesis).

International political economy (IPE) is another central sub-field of International

Relations. Much current scholarship on international politics deals with questions of

international political economy, specifically the politics of international trade and

monetary relations. Many studies in this field focus on foreign economic policy-making,

but broader definitions of the field also include the study of international institutions and

cooperation3

. International political economy has been at the center of the modern study

of International Relations largely due to the growing importance of economic interactions

among countries, but even more so as a result of the flourishing global economy since the

end of World War II and the contemporary wave towards globalization.

9

International environmental politics is a relatively new sub-field that has emerged

with the growing importance of global and transboundary environmental issues including

climate change, transboundary air pollution, and threats to the world’s biodiversity. Its

significance derives from the possibility that perfectly “normal” human activities now

have the potential to destroy the basis of life on a truly global scale. Students of the field

study motivations and policies of both traditional players such as governments and

international organizations and non-traditional players, primarily the rapidly growing

number of international non-governmental organizations, who have come to play a

prominent role in international environmental politics. Given the emerging nature of this

field, a candid discussion of methodological problems and a comparison across methods

and fields can help facilitate the advancement of a diverse research agenda.

The idea of this book was born following a discourse among some of the authors

in the 1997 annual meeting of the International Studies Association (ISA) in Toronto,

Canada. Following that conference, the editors invited the authors to write a paper on

their methodological area of expertise. In addition to presenting these papers and

discussing issues on ISA panels, the authors also met in March 1999 for a workshop that

focused on the role and limitations of the different methodologies in advancing

International Relations research. Although the group was not able to agree on every issue,

we benefited from these serious and thoughtful conversations. The interaction among

authors continued during the drafting, review, and revision of the chapters, as we read

each other’s chapters. Accordingly, these chapters (perhaps with one exception) represent

original work written specifically for this volume.

Theory and Methodology

There are three main elements that portray the state of the art and the intellectual progress

of an academic field. The first element is the set of empirical phenomena and questions

being studied; the second criterion is the development of theory, and the third is the ways

in which methodology is used to evaluate theoretical claims and their empirical

implications. This book focuses on methodology but the authors also discuss the first two

10

elements and how methodology affects both empirical debates and theoretical issues. The

links between theory and methodology are complex and deserve some deliberation4

.

Theory is defined by the American Heritage Dictionary as

[s]ystematically organized knowledge applicable in a relatively wide variety of

circumstances, esp. a system of assumptions, accepted principles, and rules of

procedure devised to analyze, predict, or otherwise explain the nature or behavior

of a specified set of phenomena (The American Heritage Dictionary 1985, 1260).

Theory provides clear and precise explanations of important phenomena. It focuses

scholarly attention on important puzzles that set the research agenda for students of the

field. Ideally, theory should also offer a set of testable and falsifiable hypotheses thus

encouraging systematic re-evaluation of its main arguments by a variety of methods.

Methodology refers to systematically structured or codified ways to test theories.

Methodology is thus critical in facilitating the evaluation of theory and the evolution of

research. It is particularly useful in the context of a progressive research program where

hypotheses lend themselves to falsification (Lakatos 1986). In these cases, methodology,

especially case studies and quantitative analysis, can assist in testing existing theories.

Methodology also helps in generating or expanding the scope of received theories

as can be seen sometimes in formal modeling. Given a range of assumptions about the

properties of actors and their interactions, various hypotheses can be deduced and,

ideally, corroborated – or rejected – by empirical case studies or in quantitative research.

Formal models can also be used to probe and cast doubts on the internal validity of

theories (see chapters 10 and 14). Ideally, theories would be supported by studies that use

different methods.

Theory and methodology are most beneficial when they accompany each other for

the advancement of knowledge. While theory provides explanations for particular

phenomena based on specific assumptions, purely axiomatic knowledge, turned into

theories, is rarely useful in explaining real “world politics”. Theoretical arguments have

to be augmented with systematic methods of testing that can also help guard against

chance and selection bias. Besides formal models, it is mainly case study research, which

can help generate new hypotheses to advance theory building. Both case studies and

11

quantitative methods are often used to test propositions. Carefully crafted research

designs permit the assessment of regularities between variables, detection of their

limitations (e.g., scope of the relationship in time and space) and point to the possibility

of generalization as well as replicability and reliability of the findings (see appendix 1 for

more details on research design).

Political methodology has undergone many changes over the last century. King

(1991) suggested a five-part history of political methodology during the 20th century.

Beginning with the early 1920s, and inspired by a scientific tradition in North American

social sciences, pioneers used direct empirical observation; subsequently, the “behavioral

revolution” of the mid-1960s accounted for a sharp increase in empirical-quantitative

analyses by applying statistical methods to data that was collected mostly by researches

from outside of the field. It was only during the 1970s that political scientists began to

create their own datasets rather than rely on externally generated data as in the earlier

phases. The late 1970s witness a substantial increase in borrowing quantitative methods

from outside of the political science discipline. Finally, since the 1980s political science

methodologists have improved existing methods and developed new tools specifically

geared to answering political science questions.

The history of quantitative studies in international relations resembles that of

political science at large, but since the 1970s case study methodology has also

proliferated in international relations, particularly in studies that reach into the

comparative politics field. In addition, the growth of rational choice approaches first in

economics and subsequently in political science has now had a marked impact on the

study of international politics. Since the 1980s, both mathematical models and soft

rational choice approaches have contributed to the development and refinement of central

ideas in the field such as hegemonic stability theory and the democratic peace (Goldmann

1995; Wæver 1998). Also emerging during the 1980s and 1990s were constructivist,

poststructuralist, and postmodern approaches to international relations, although it

remains debatable whether these approaches actually have developed a methodology of

their own.

12

In order to gain more insight about the prevalence of different methodological

approaches in International Relations, the editors of this volume conducted a survey of all

articles published in some of the leading journals in the field during the last twenty-five

years. The survey included articles published between 1975-2000 in the following

journals: American Political Science Review,5

International Organization, International

Security,6

International Studies Quarterly, the Journal of Conflict Resolution, and World

Politics (see Figure 1).7

The articles were classified into five categories according to the method of

analysis employed:

i. descriptive approach based on historical analysis (and lacking clear

methodology)

ii. case studies - analytical empirical research containing some methodological

components (at least justification for case selection and method of analysis as

well as possibly descriptive inference)

iii. quantitative (statistical) analysis - ranging from simple correlation/covariance,

factor analysis to regression analysis and more advanced statistical methods

iv. formal modeling - ranging from simple models to simulations and more

sophisticated mathematical game models

v. combination of at least two methodologies (ii – iv), (esp. quantitative analysis

with formal modeling).

\Insert Figure 1 about here\

The broad trajectory over the past 25 years, grouped as five-year intervals (with the

exception of the most recent group which comprises six years) demonstrates important

methodological trends in International Relations. The most profound trend evident in

Figure 1 is the continuing decline in the number of articles using a descriptive-historical

approach (and lacking serious consideration of methodology). While in the late 1970s

about half of all the articles published in these journals lacked any methodological

component, in the late 1990s less than one third of the articles surveyed could be

classified as such. This trend reflects an important development in the way International

13

Relations scholars conduct their research and supports the notion that International

Relations as a field has become more methods-oriented than before. In particular,

International Studies Quarterly, International Security, and World Politics all currently

publish far fewer articles that pursue a descriptive-historical approach than twenty-five

years ago. For instance, during the late 1970s over 70 percent of the articles published in

World Politics applied a descriptive or historical approach while in late 1990s this ratio

declined to less than 30 percent. Another interesting finding is the fairly constant

frequency of articles using case studies, which has remained roughly constant at around

13 percent throughout the last quarter century.

In contrast, there has been a sharp increase in the number of articles using either

quantitative or formal methods or a combination of both. In the population of articles

published within the surveyed journals, the percentage of articles pursuing statistical

analysis rose from about 26 percent during the late 1970s to about 43 percent during the

late 1990s. This trend is most pronounced in International Organization and World

Politics. Edward Mansfield made a similar discovery with respect to the frequency of

statistical analysis in articles on international political economy (see chapter 7). It is

remarkable that close to half of all articles recently published in these six prominent

journals use quantitative methods of research. While this trend can be partly explained by

the greater availability of large data sets, the increased popularity of statistical methods in

International Relations undoubtedly reflects better methodological training of students

and scholars.

The number of articles using formal methods increased from less than 9 percent

during the late 1970s to about 14 percent in the late 1990s. While International

Organization, International Studies Quarterly, and World Politics all currently publish

more articles using formal methods than they did 25 years ago, the most significant

increase in formal methods is concentrated in journals that have traditionally published

more quantitative work, specifically the Journal of Conflict Resolution and the American

Political Science Review. Although recent claims about formal theory suggest that game

theory is becoming more influential in the study of international politics, articles using

formal methods still constitute a relatively small portion of International Relations

publications, on par with case-study analysis.

14

The survey of these leading journals also confirms that few scholars in the field

engage multi-method research. Although this figure has been slowly rising, during the

late 1990s still less than four percent of all articles published in the journals surveyed

used both statistical and formal methods. Cross method analysis obviously requires more

training (or alternatively, cross-field collaboration). However, it allows scholars to

investigate alternative explanations, compensate for weaknesses in each of these

methods, and corroborate research results. Cross-method analysis will not eliminate all

uncertainty from (theoretical) claims in the study of International Relations, but it would

increase the reliability of theoretical research.

This volume aims at increasing the dialogue among scholars of International

Relations and reducing the costs of cross-method discourse. It does so by providing in￾depth discussions of methodological concerns associated with using case study,

quantitative analysis and formal methods. Throughout the book, the authors also

emphasize the trade offs involved in deploying these methods to different substantive

sub-fields of International Relations. This book is intended for students and scholars of

various sub-fields of International Relations who specialize in different research methods.

As it introduces methodology without assuming prior formal education in social scientific

methods, it can also be used in advanced undergraduate and graduate courses.

Plan of the Book

The book is organized around three methodological approaches to the study of

International Relations: case studies, quantitative analyses, and formal methods. Each

methodological section begins with an introductory essay that presents an overview of the

method and explains its advantages and its limitations. Following the introductory

chapter, each methodological section includes several chapters that focus on applications

of the respective method in different sub-fields of International Relations, namely

international political economy, international environmental politics, and international

security. The chapters evaluate the contribution of the various methods to central debates

in the field as well as to theory building. They do so not by following a uniform format,

15

rather by discussing the literature and specific methodological issues, or sometimes by

focusing on a more detailed theoretical framework of analysis.

The chapters are united in their emphasis on exploring common methodological

concerns, providing a critical evaluation of central ideas from a methodological

perspective, and stimulating discourse among International Relations scholars. In order to

provide additional guidance to readers each chapter recommends five main studies for

further readings. The concluding chapter of the volume evaluates some of the merits and

limits of the different methodologies presented for studying International Relations.

The following table details the structure of the book and the authors of the respective

chapters (see Table 1).

\Insert Table 1 about here\

All the chapters in the book were written for the purpose of offering an evaluation

and critique of the analysis of International Relations. Reading the entire book provides

the reader with the benefit of a broad perspective on the use of the main methods of

analysis in different sub-fields of International Relations as well as discussions of key

substantive debates. The chapters can also be read in alternative ways. Each chapter

stands on its own merits and can be read separately; in addition, the book can be read by

methodological sections (parts of the book) or by substantive fields. For instance, readers

can choose to focus on how a particular method has been applied in several sub-fields of

International Relations. This focus on a particular method may be more useful for classes

on research methods (reading by row in Table 1). Alternatively, readers interested in a

particular sub-field can compare how the different methods have been applied in that

particular field (reading by column in Table 1). Such reading of the book is most useful

for classes in a particular sub-field, for instance students in a class on international

political economy will benefit from reading about the application of the three different

methodological approaches in their sub-field. Finally, a more introductory course can use

the introductory chapters to each part of the book, which offer an overview of each