Scaling Up a Brief Student Mindset Intervention: Lessons Learned During COVID

• In Event: Rigorous Research in the Time of COVID: Lessons for Rethinking Educational Research

Abstract

Objective. We address how the COVID-19 pandemic disrupted the scaling up of a brief student mindset intervention. We discuss observed differences across three years of implementation, focusing on lessons we learned administering writing exercises that students complete as part of a normal “classroom” activity.

Perspectives & Methods: Project Description. Our multisite randomized controlled trial was designed to replicate, at scale, a brief mindset intervention shown to mitigate the negative impact of stereotype threat for 7th grade students of color. The planned study included two consecutive cohorts of 7th grade students (2019-20, 2020-21) with students randomly assigned to experimental conditions within school and race/ethnicity blocks.

The intervention consists of three to four 15-minute in-class writing exercises, which prompt students to choose two or three personally important values from a list of 11 (e.g., family, music, sports) and write about why they find them important. Control students engage in the same amount of reading and writing but are asked to write about neutral topics that are unlikely to induce self-affirmation. Teachers, who were blind to randomization and the true purpose of the exercises, distributed intervention materials as if they were normal classroom activities. All students remained blind to their condition and to the hypotheses of the study.

Results: COVID-Related Changes. Although schools were engaged in remote learning in the 2020-21 school year, our partnering sites implemented the intervention and allowed us to collect student survey outcomes via an online platform. Of the 33 Cohort 1 schools, 30 schools agreed to continue with the study in 2020-21. Although student response rates declined during the 2020-21 school year, using online administration of materials, response rates for the intervention mirrored the completion of other student schoolwork (approximately 60-70%).

Another change we made to the study was to add a third cohort of students and approximately half of our schools agreed to participate. These schools also elected to continue with online administration of the intervention and therefore we are able to compare response rates and outcomes for online administration of the intervention across two learning environments: fully remote and typical in-school learning.

Significance: Lessons Learned. First, we learned that our partnering schools were deeply invested in our study, for which we remain grateful. Second, we observed that with only minor changes to the wording of the intervention, the writing exercises could be administered by teachers, and completed by students, via an online platform. Third, we learned that response rates were similar regardless of whether the intervention was delivered via an online platform or paper—when students were in school. Response rates were slightly lower when students were engaged in remote learning. Another discovery was that when using an online platform to deliver the intervention, students wrote nearly 4 times as much as they did with paper administration. Finally, and importantly, treatment students affirmed at similar rates whether participating via online or paper administration.

Authors

• Trisha Borman, American Institutes for Research
  Presenting Author

• Geoffrey D. Borman, Arizona State University
  Non-Presenting Author