The Tower Method: a Novel Approach to Missingness

• In Event: Missing Data and Data Augmentation

Abstract

Most methods for handling missing data are designed for statistical inference contexts, yet many of today's applications are to predictive analytics. Here we introduce a new method, designed specifically for use in prediction, based on the Tower Property from probability theory. Empirical investigation presented here is quite promising. Our the predictive accuracy of our approach is as good as, often better than, the leading imputation techniques, and is better computationally: unlike leading implementations mice and Amelia, our approach, which we term the Tower Method, does not have convergence issues, and has better runtime than mice (as in, ‘multiple imputation by chained equations’). Indeed our Tower Method's speed is much closer to listwise deletion, which is both fast and convenient but the most suspect for either inference or prediction. We introduce a new R package, polyanNA, which simplifies implementation of the Tower Property in a variety of regression contexts. Finally, we provide empirical benchmarks using several data sets such as the World Values Study, ANES, and Votivate.

Authors

• Pete Mohanty, Stanford University

• Norman Matloff