World Politics

Research Note

Improving Forecasts of State Failure

Gary Kinga1 and Langche Zenga2

a1 Harvard University

a2 George Washington University

This article offers the first independent scholarly evaluation of the claims, forecasts, and causal inferences of the State Failure Task Force and its efforts to forecast when states will fail. State failure refers to the collapse of the authority of the central government to impose order, as in civil wars, revolutionary wars, genocides, politicides, and adverse or disruptive regime transitions. States that sponsor terrorism or allow it to be organized within their borders are all failed states. This task force, set up at the behest of Vice President Gore in 1994, has been led by a group of distinguished academics working as consultants to the U.S. CIA. State Failure Task Force reports and publications have received attention in the media, in academia, and from public decision makers. The article identifies several methodological errors in the task force work that cause its reported forecast probabilities of conflict to be too large, its causal inferences to be biased in unpredictable directions, and its claims of forecasting performance to be exaggerated. However, the article also finds that the task force has amassed the best and most carefully collected data on state failure to date, and the required corrections provided in this article, although very large in effect, are easy to implement. The article also demonstrates how to improve forecasting performance to levels significantly greater than even corrected versions of its models. Although the matter is still a highly uncertain endeavor, the authors are nevertheless able to offer the first accurate forecasts of state failure, along with procedures and results that may be of practical use in informing foreign policy decision making. The article also describes a number of strong empirical regularities that may help in ascertaining the causes of state failure.

Gary King is Professor of Government at Harvard University; Senior Science Advisor, Information and Evidence for Policy Cluster, World Health Organization; and Director of the Harvard-MIT Data Center. He the author of A Solution to the Ecological Inference Problem: Reconstructing Individual Behaviorfrom Aggregate Data (1997) and Unifying Political Methodology: The Likelihood Theory of Statistical Inference (1989), and he is coauthor (with Robert Keo-hane and Sidney Verba) of Designing Social Inquiry: Scientific Inference in Qualitative Research (1994). He has also written several widely used public domain statistical software packages.

Langche Zeng is Associate Professor of Political Science at George Washington University. She has contributed articles to various social science journals and is the author or coauthor of several public domain statistical software packages on neural networks, heteroskedastic logit models, and rare event logit. Her research interest centers on quantitative methods.