Can We Change the Mindsets of a Nation? Evidence From the National Mindset Study

• In Event: 25.015 - Improving the Evaluation and Scalability of Social Psychological Interventions in Schools

Abstract

The National Mindset Study evaluates a motivational intervention delivered in the first 10 weeks of ninth grade in a national probability sample of regular U.S. public high schools. The particular motivational approach we employ is a growth mindset of intelligence intervention (Dweck, 2006; Paunesku et al., 2015; Yeager & Dweck, 2012). Growth mindset is particularly useful for the present purpose because researchers have recently developed efficient methods for delivering this intervention via the internet at scale with minimal cost or professional development, yielding a highly scalable intervention. Moreover, the intervention lends itself to generalizable experimentation.

Randomized experiments rely on the logic that units randomly assigned to one or another group differ only on the presence of the treatment (within sampling error). But causal inference alone does not allow for generalization, because causal effects may be moderated (i.e., influenced by the context and population in which they are observed). For example, the types of schools that typically participate in experiments testing social or behavioral interventions may also be the types of schools that are hand-selected to have structures or resources that facilitate student responsiveness to treatments. Experiments in education have almost exclusively been carried out in samples of convenience (Olsen, Orr, Bell & Stuart, 2013). As a result, research has not provided a statistically defensible way to justify claims about the types of schools in which experimental interventions to improve motivation can alter a ninth grader’s educational trajectory.

In contrast, much social science that does not seek to make causal claims routinely uses the logic of randomization in a different way: by constructing scientific samples using probability methods. This sort of random sampling has been the bedrock of survey research for nearly a century. It allows the American government, for example, to efficiently measure the number of people who are unemployed in the country, the physical health of its residents, and much more. If random sampling is done, whereby every member of a defined population has a known, non-zero probability of selection, a researcher has a solid scientific basis to justify generalizing findings (Kruskal & Mosteller, 1980).

The present research combines the tools of random assignment to experimental condition, which allow for causal inference, with the tools of random selection of schools, which allow for generalizability. This allows us to address three questions that have the potential to inform the basic assumptions of many social and behavioral interventions.

Data come from the National Mindset Study, which is an experiment from 76 U.S. public high schools, involving more than 19,000 students. Results show that a growth mindset can be delivered with high fidelity (94% student response rate; 92% rate of successful completion of materials) to a large sample, and produce effect sizes on immediate self-reports (d = .26) that are quite strong given the sample heterogeneity.

Authors

• David Scott Yeager, Stanford University

• Paul Hanselman, The University of Texas - Austin

• Chris S. Hulleman, University of Virginia

• Elizabeth Tipton, Teachers College, Columbia University