Country Fixed Effects with a Binary Dependent Variable: MLE or LPM?

• In Event: Poster Session: Political Methodology

Abstract

Models using country fixed effects in panel data with a binary outcome variable require certain sacrifices that researchers are often unwilling to make. In maximum likelihood with rare events data, a substantial portion of the sample may be lost due to lack of within-group variation. Coefficients are also notoriously inaccurate when a large number of covariates is added to the model. The deficiencies of the linear probability model (LPM), on the other hand, are equally well-known: heteroscedasticity, predicted probabilities outside the 0-1 range, and the assumption that relationships are linear when they may not be. Researchers using the LPM, at least in political science, admit that it is less accurate than maximum likelihood and often cite simplicity of interpretation as the reason for using it, rather than advocating for it as the most appropriate estimation technique according to their theory and data (see Harding and Stasavage, 2014). Because of the glaring deficiencies of both approaches with regards to fixed effects specifications, researchers often ignore fixed effects altogether and opt for alternative models.

While the reasoning is justified, I argue that this is wrong conclusion. Fixed effects estimation can be very useful in rare events data, but the right steps need to be taken to produce reliable results. In this paper, I argue that the linear probability model produces, on the whole, more accurate estimates than maximum likelihood when using fixed effects specifications. The differences are relatively minor, on average, when the sample includes between 25 and 75 percent of ones (the more observations per group, the more similar results are within this range). However, when the sample has either less than 25 percent or more than 75 percent of ones, the LPM is notably superior than the conditional logit or the logit with dummies in producing accurate predicted probabilities. Moreover, I argue, the functional form of the prediction needs to be understood in order to produce the proper interpretation of fixed effects estimates. I provide a novel technique to estimate the threshold at which the LPM should be preferred over MLE. I also show how to estimate the correct functional form of the relationship and predict where the LPM can be underestimating and overestimating the results. The actual ‘correction’ cannot be performed, but the researcher can gain a lot of information from the direction of the bias alone.

This paper is motivated, first, by the tentative and cautious use of fixed effects in the applied literature. Broadly speaking, researchers are often caught between the common use of maximum likelihood, the difficulties that MLE presents for rare events and for calculating marginal effects, and the generally discredited LPM. This paper provides a focused discussion of these issues and proposes a solution to the riddle. Second, there exists a growing need to use fixed effects in comparative politics and some subfields of international relations to avoid false positives arising from omitted variable bias. Cross-country comparisons based on panel data, which are pervasive in comparative politics, are particularly susceptible to omitted variable bias, as differences among countries are substantial and difficult to model. Fixed effects models provide a good, if blunt, solution to this problem. Researchers who run the fixed effects equivalent of their model and encounter a smaller coefficient should be particularly alert to omitted variable bias. Running fixed effects models can be of great help to avoid common Type-I errors (a more elaborate review of the literature is available in the paper).

To show the differences between LPM and MLE with fixed effects in rare events data, I use Monte Carlo simulations with randomly generated data. I create 15 separate datasets consisting of 20, 50 and 100 groups, and 20, 30, 50, 75 and 100 observations per group. MLE with fixed effects is especially sensitive to overall sample size and to the number of observations within each group. These multiple datasets allow us to compare estimation of rare events data with fixed effects at different sample and group sizes. Next, I perform 11 different simulations for each dataset aimed to differences between the LPM and MLE in estimating coefficients in rare events data. Each time the dependent variable is generated to include, at random, a certain percent of ones: 1, 3, 5, 10, 25, 50, 75, 90, 95, 97 and 99 percent. This process generates 11 different sets of coefficients and predicted probabilities that can be plotted. When observations per group are above 30, the MLE and LPM models produce practically identical predicted probabilities at 50% of ones. When the percentage of ones is 25 or 75, both models produce similar probabilities. Below 25 or above 75 percent, however, the LPM produces a predicted probability much closer to the real value, while the MLE model produces biased predictions.

Author

• Joan Carreras Timoneda, University of Maryland