A New Matching Frontier to Help Balance Priorities When Designing Observational Studies

• In Event: Observational and Experimental Causal Inference

Abstract

Despite the popularity of matching-based methods for causal inference in observational settings, there is no consensus on how best to define matching goals, how to operationalize fundamental formal assumptions, and how to assess the usefulness of matched data. King et al. (2017) propose their matching frontier, which optimizes a trade-off between sample size and a measure of covariate imbalance when estimating average treatment effects for treated units. We build upon this by describing an alternative approach to frontier visualization that helps researchers make well-informed matching design choices. Our approach recommends researchers apply their knowledge and expertise to define their goal(s) and objective(s), but not to try to identify the best matching model (e.g., to select variable importance weights) to achieve those ends. Researchers should use general-purpose machine-learning optimization algorithms (such as genetic algorithms) to optimize matching models. We believe the comparative advantages of our approach are: (a) it encourages researchers to think expansively and strategically about their inferential goals and consider objectives and trade-offs of practical interest beyond sample size and imbalance; (b) it uses information about observed outcomes to help researchers consider and visualize trade-offs between a diverse set of matching objectives, so they may assess sensitivity and stability of their treatment effect estimates and optimize matching models for desirable properties (e.g. reduced model-dependence and sensitivity to unobserved confounding) without encouraging unprincipled results-hacking, and (c) itis a balance-maximizing matching method that matches unidirectionally(from treated units to control units). We also introduce an imbalance measure that is new to this context—the supremum of all linear functions of covariate imbalance, of a certain magnitude—and explore its properties in our empirical example, the classic Lalonde-Dehejia-Wahba NSW dataset. Software to implement our proposed methods is available in R.

Authors

• Alexis J. Diamond, Minerva Schools at KGI

• Idriss Bennis

• Mikulas Plesak, Minerva Schools at KGI