Beyond Latin Squares: Methods for Generalized Orthogonal Randomized Assignment

• In Event: Advanced Methods for Causal Inference and Power Analysis

Abstract

Despite their power, simple random assignment strategies such as complete Bernoulli randomizations and random allocations often cannot meet all of researchers' design demands. Experimental designs frequently need to incorporate complex features, such as simultaneous assignment constraints in multiple dimensions. We describe the traditional Latin square problem, then generalize the foundational Latin square solution to cases with arbitrary numbers of units, conditions, and assignment periods. We further examine additional types of across- and within-unit constraints that experimental researchers can put on their assignment algorithms. We provide concrete solutions for these types of orthogonality requirements, contrast our solutions to extant approaches, and demonstrate advantages over other methods. We show how orthogonal designs can decrease the entropy in treatment condition distributions by 71-76% over weaker designs, for example. While other software approaches to random assignment exist in tools that excel at improving assignment through, for example, covariate blocks, we address a somewhat different set of constraints that researchers often face. Our approach can address the needs of researchers hoping to block on a small set of discrete covariates. 

We detail applications in two actual field experiments conducted with governments, one in social policy, one in fire and emergency services. The former experiment tests a strategy of messaging to participants in the US federal disability insurance program, to encourage job training and work experience. Using a combined sample of nearly one million beneficiaries, this experiment distributed four different modes and messages to beneficiaries, finding no effect on program participation, but increases in calls to the program helpline. The latter experiment tests how risk scores from a predictive model might encourage street-level bureaucrats to exercise discretion in ways that improve public safety. When (blinded) inspectors received lists consisting entirely of buildings with high probabilities of fire, they recorded no more fire-causing violations during their discretionary inspections, compared to when they received lists with a set of buildings stratified by risk score.

Author

• Ryan T. Moore, American University
  Presenting Author