Shrinkage Estimation for Factorial Experiments

• In Event: Causal Inference: Mediation, Experiments, and Sensitivity

Abstract

In randomized experiments with multi-dimensional treatments, estimation of interaction effects and non-linearity may be important for understanding causal structure, but costly in terms of variance. I present a structured approach to analysis of factorial designs, incorporating assumptions that are likely common in factorial experiments in the social sciences: interactions and non-linearity in such cases are likely to be small, but not exactly zero, and increased complexity of interactions or non-linearity is associated with smaller relative effects. I then develop a hierarchical ridge-type shrinkage estimator, under which increasing penalties are applied to the L2 norm based on a series expansion of the treatment variables.

Consider as an example a randomized factorial experiment, where each factor is assigned independently and consists of a baseline level and several experimental levels. Suppose the effects of interactions are close to, but are not exactly equal to, zero; and as the complexity of interactions increases, the relative effects are likely to move closer to zero. This setting is motivated by cases where there is a natural baseline, and we have evidence from meta-analyses regarding the hierarchical structure of effects.

While this setting may not be universally representative, it does not seem far-fetched for most factorial experiments in the social sciences. Based on literature in the field of political science, non-linearities and interactions are expected to be relatively small; in a get-out-the-vote campaign, after having received one letter, the effect on a subject of receiving a second letter is of smaller magnitude; having received a phone call and a letter, the interaction effect, whether positive or negative, is unlikely to completely negate or to more than double the main effect of either direct treatment. I will consider implications when these assumptions are violated, with comparisons to conventional estimators, including lasso, standard ridge regression, and elastic net.

In current literature in the social sciences, common approaches in analysis of factorial experiments are either to ignore interactions and non-linearities in favor of estimating only main effects, or to apply penalties to the L1 norm on all interactions, treating estimation as a variable selection problem (e.g., Imai and Ratkovic, 2013). Given the behavior of L1-based estimators when parameter components are close to zero (Leeb and Potscher, 2008), such estimators that exploit sparsity have the potential for introducing a region of maximum risk. As well, the non-regularity of penalized estimators implies that the structure of potential interactions should be taken into account. In such complex sparsifying models, no distinction is made between higher- and lower-order interactions. The estimator proposed here addresses both concerns and will tend to have better properties than unstructured, L1-based estimators in many real-world settings.

This approach is applied to a canonical get-out-the-vote field experiment (Gerber and Green, 2000), with multiple factorial treatments and a natural baseline condition; results are compared to those of Imai and Ratkovic (2013), analyzing the same data. A second application considers a conjoint experiment on preferences over a ballot measure on campaign finance reform, conducted on Amazon Mechanical Turk.

Author

• Molly Rachael Offer-Westort, Yale University