Why Judges Decide Differently: The Theoretical Implications of Empirical Models

• In Event: Methodology in the Study of Political Institutions

Abstract

Much of the judicial decision making research has focused on inter-judge inconsistency. How much more likely is a panel of all Democratic appointees to decide in favor of civil rights plaintiffs than is a panel of all Republican appointees? How does the presence of a black judge on a panel affect the likelihood that an affirmative action plan will be found constitutional? Due to the “credibility revolution” in the social sciences, many judicial behavior researchers have focused on answering these questions by leveraging the quasi-random assignment of cases to judges. However, these identification strategies can yield empirical results that are removed from the theories they are meant to address. If we are interested, for example, in the theoretical question of how political preferences influence decisions in civil rights cases, an ATE of Democratic appointees on on the percentage of pro-plaintiff decisions is much less informative than is generally appreciated. The problem is twofold: (a) it may be something associated with the treatment---and not the treatment---that is actually the cause of the measured effect and (b) the measured outcome may mask the dimensions of disagreement. Recent research has begun to address the first problem. But researchers have essentially ignored the second problem: most simply assume that their chosen outcome (e.g., liberal or conservative, reverse or affirm, etc.) captures the nature of judicial disagreement. For example, while it may seem obvious that differences in “liberal” decision rates between judges reflects differences in ideology, it may instead simply reflect differences in general reversal proclivity. In this paper, we build on our recent work in which we develop a new method for estimating inconsistency between judges. Using an original dataset of all appeals filed in the Ninth Circuit between 1995 and 2013, we show how our method can be a diagnostic tool: discrepancies between a theory-driven average treatment effect and our agnostic measure of inconsistency can alert a researcher to potential disconnects between theory and evidence.

Authors

• Ryan Copus, Berkeley Law, JSP

• Ryan Hubert, University of California, Davis