Forecasting and Ethics, Ctd.

A few additional thoughts (original thoughts here) inspired by posts on Duck of Minerva and Dart-Throwing Chimp.

First, I just want to agree wholeheartedly with Jay Ulfelder’s conclusion on Dart-Throwing Chimp:

Look, these decisions are going to be made whether or not we produce statistical forecasts, and when they are made, they will be informed by many things, of which forecasts—statistical or otherwise—will be only one. That doesn’t relieve the forecaster of ethical responsibility for the potential consequences of his or her work. It just means that the forecaster doesn’t have a unique obligation in this regard. In fact, if anything, I would think we have an ethical obligation to help make those forecasts as accurate as we can in order to reduce as much as we can the uncertainty about this one small piece of the decision process. It’s a policymaker’s job to confront these kinds of decisions, and their choices are going to be informed by expectations about the probability of various alternative futures. Given that fact, wouldn’t we rather those expectations be as well informed as possible?

And I just want to underscore something that Daniel Nexon on the Duck referred to as the peformativity problem, referencing some interesting work in economics. I do agree that an interesting intellectual question is if political scientists ever get good at forecasting–a big ifand those forecasts do generate policy interventions, then the forecasts might become self-defeating or self-fulfilling. The goal is for them to become self-defeating, but one could imagine the opposite: military intervention or aid could destabilize a perilous society or warnings of risk could lead to better data collection which could lead to categorization of a problem as being more serious than in an area without warning. Either self-defeating or self-fulfilling predictions present a host of empirical problems. It’s worth a longer discussion, but I think it’s worth starting the discussion. My hunch is political science is shifting methodologically to a fight between the forecasters (probably bolstered by big data) and the causal inferencers. The issues associated with causal inference have been fleshed out in some detail, but the issues associated with forecasting are still relatively unexplored. Let’s start exploring.

Forecasting and Ethics

Idean Salehyean has a provocative post over at the Monkey Cage arguing that forecasting is not a value neutral enterprise. If we forecast some outcome, we should expect policymakers to take some steps as a result, and those steps may or may not be acceptable ethically. Okay. But not forecasting may or may not be acceptable ethically either. I’m a bit of an ethics novice, so I assume people are coming at morals from either some consequentialist or deontological frame. It’s difficult for me to see how forecasting is directly a rights or rules violation, so we’re automatically in some realm of consequentialism when we are worried about forecasting. And once we are down that path, we have to explicitly consider the counterfactual of not forecasting. And not forecasting when one has some ability to do so might lead to bad moral outcomes.

We probably aren’t in the business of chaining Nate Silver, Andrew Gelman, and Gary King to their desks to forecast genocide even if it would achieve salutary consequences (probably for good deontological reasons about letting people make choices about their lives). But I think Idean Salehyean’s point ends up being a banal one because more or less everything we do is not value neutral.

Nothing is special about forecasting, I would stress. Observational studies—the democratic peace, for instance—might lead one to conclude that democracy should be spread, by force if need be.

We are inhabitants of the world. We happen not to be particularly influential inhabitants so the chain of causation from our studies to positive and negative consequences is lengthy. We shouldn’t be paralyzed or fascinated by our presence in the moral universe.

Defending the Science of Politics: Wait, We Were Trying to Forecast?

I don’t want to dwell for too long on Jacqueline Stevens’s commentary on political science from this weekend’s New York Times Sunday Review. Professor Stevens deserves credit, not scorn, for engaging the public intellectual sphere in a serious manner, an explicit goal of the political scientists of this blog. While I am certain that colleagues of mine will discuss some of the substantive portions of her piece in greater detail, I merely wish to highlight two points of hers that I struggled to comprehend from an ontological perspective.

First, I cannot fully understand Professor Stevens’s own motivations for her support of the proposed bill to end political science funding. She cites the following as the fundamental reason behind her support:

The bill incited a national conversation about a subject that has troubled me for decades: the government — disproportionately — supports research that is amenable to statistical analyses and models even though everyone knows the clean equations mask messy realities that contrived data sets and assumptions don’t, and can’t, capture.

In other words, Professor Stevens wants to cut off all political science funding because she believes that the type of research that receives the majority of NSF-funding–quantitative research–is inherently flawed. Even if we accept her belief that quantitative political science research is inferior, is it worth it to cut off all political science funding? Isn’t Professor Stevens’s proposal the equivalent of cutting off our proverbial noses to spite our faces? While I agree with Professor Stevens’s contention that we should be careful about “clean equations” that purport to explain more than they can, I do not think that cutting off NSF research from all of political science is a productive way to begin such a conversation. Ultimately, Professor Stevens says that

Government can — and should — assist political scientists, especially those who use history and theory to explain shifting political contexts, challenge our intuitions and help us see beyond daily newspaper headlines.

This proposal suggests a serious discussion about funding allocation, not funding elimination. As I stated above, I think that this is a conversation worth having. However, this  conversation does not mean support for the proposed bill that is making its way through Congress. This bill would eliminate political science funding, including such funding that might have gone to political scientists “who use history and theory.”

Second, I am a bit puzzled as to why Professor Stevens seems to believe that political science is fundamentally about prediction. True, positivism lies at the heart of the social scientific enterprise. However, political science, like all science, is about explanation, understanding, and knowledge. After all, the Latin scientia, from which science derives its meaning, translates into “knowledge,” not prediction. Political scientists are human beings and, as such, struggle to predict the future. For this reason, political scientists, more than most journalists and public intellectuals, are usually careful about the extent to which they can claim to predict future events.

Consider James Fearon and David Laitin’s 2003 article that is the principal subject of Professor Stevens’s critique. As Professor Stevens suggests, their contribution is twofold: to dispel the notion that ethnic grievances cause civil wars and to promote the idea that the same conditions that favor insurgency (i.e. as in weak states), favor civil wars. However, let’s look at the exact language that Fearon and Laitin use. Against the grievances argument, they write that

we find little evidence that one can predict where a civil war will break out by looking for where ethnic or other broad political grievances are strongest.

Isn’t this the exact type of language and research for which Professor Stevens advocates? They present a study about the failures of political science prediction. If anything, they caution against arguments, from political scientists or others, that use grievances to predict civil wars. Moreover, they state that

measures of cultural diversity and grievances fail to postdict civil war onset, while measures of conditions that favor insurgency do fairly well.

Again, notice both the language and caution that Fearon and Laitin utilize. “Fail to postdict” suggests that history has not provided any evidence in favor of the argument that grievances predict civil wars. The use of the word “postdict” suggests Fearon and Laitin’s own hesitation about the performance of their own variables.

The Fearon and Laitin article is just one of many examples of political science research executed with due diligence and circumspection. There are countless others. Of course, there are also countless works of political science that are not nearly as well done and fail to resist the urge to predict beyond the limits of social science. We need to have a frank conversation about the place of such research in the world of political science. However, to support the elimination of all political science research, including that which both I and Professor Stevens would consider of top quality, for the sole reason of having this conversation seems ill-advised.

For more of his musings on political science, follow William on Twitter.