Using Randomized Re-assignment to Address Selection Bias in Group Comparisons

• In Event: 315 - Open Refereed Roundtables Session. B. Table 01 Shared and Contested Spaces
  In Refereed Roundtable 100min: Table 07. Education and Educators

Abstract

When seeking to conduct group comparisons in which the two groups are composed of potentially different subject, there is the concern that differences in these two groups may affect the comparisons of these two groups. There are multiple ways to address this concern, such as only comparing across groups those differences which exist within-groups or by using relative variation measures, such as the coefficient of variation. I now consider a third option: difference-in-difference (DiD) models via randomized re-assignment.

The most common set up is where outcomes are recorded for two groups across two time periods. One group—the treatment group—is exposed to some type of treatment in between Time 1 and Time 2. In contrast, the other group—the control group—is not exposed to the treatment at all. Because the same subjects are recorded at the start and end of the experiment, for each group, the average change in scores from Time 1 to Time 2 in the control group can be subtracted from the average change in scores from Time 1 to Time 2 for the treatment group. This, in essence, eliminates concerns of biases stemming from group differences, and enables researchers to isolate the effect of the treatment.

In this example, I compare the distribution of teachers across two education labor markets--private and public schools--with different organizational structures, showing that sector-level differences are not due to selection bias, but rather, to the different organizational structures of each system.

Author

• Ryan Seebruck, University of Arizona