The Rescaling Problem in Nonlinear Probability Models

• In Event: Nationalization and Other Long-Term Trends in Voting

Abstract

Nonlinear probability models that deal with binary dependent variables, such as logit and probit models, are important workhorses in modern empirical social science research. The same holds true for their ordered and categorical extensions. Yet comparing raw coefficients of such models across different specifications and different samples routinely presents the problem that estimates of the same underlying relationship change drastically with minor specification changes or different samples.

The problem occurs because the estimated parameters in such models are only identified “up to a scale,” which means that the estimated coefficients are scaled by the standard deviation of the unobserved disturbance term. For example, any additional covariate would mean a decrease in the residual variance, which then inflates all the estimated coefficients even if their true values have not changed.

In the paper, we study alternative estimators that address such rescaling problem, including standardized coefficients, AME (Average Marginal Effect), and the method developed by Karlson, Breen, and Holm (2011, hereafter KHB). A series of Monte Carlo simulations show that AME outperforms other alternatives in most cases with wide range of error distributions for the disturbances, while KHB works better only under limited conditions. We supply practical advice for practitioners on how to better compare results of nonlinear probability models across models and different samples.

Authors

• Christopher H. Achen, Princeton University

• Won-ho Park, Seoul National University