Accepting the Null: Exploring the Varieties of Value Construction in College Statistics

• In Event: 30.032 - Opening the File Drawer for Innovation in Task Value Intervention

Abstract

Since the National Research Council (2003) called for education researchers to help make curricula relevant to students’ lives and backgrounds, considerable progress has been made in the development of “relevance” or task-value interventions (for overview, see Harackiewicz et al., 2014). Such advances have provided empirical support for the value of relevance interventions in promoting students’ motivation and achievement across student populations and disciplinary contexts; however, in some cases, well-regarded interventions have failed to replicate prior findings, which are rarely reported in professional journals. Aside from preventing file drawer bias and further failed efforts, reported null findings can spark discourse and innovation critical for scientific advancement (Cooper, 2015). As James Joyce put it, “Mistakes are the portals of discovery.” We will report results of a failure to replicate a popular relevance intervention and present qualitative research findings that provide valuable insights for improving future intervention efforts.
We implemented a widely recognized self-generated relevance intervention (Hulleman et al., 2010), using a randomized control design and propensity matching in an introductory statistics course at a large, Midwestern public research university. The intervention required students to complete surveys and write short essays at two time points either (a) relating course content to their lives (relevance condition) or (b) describing statistical concepts (control condition). Prior studies have found that students who wrote about the relevance of (e.g., high school science, college psychology) coursework to their lives showed increased interest, subjective task value, and performance (Harackiewicz et al., 2014). We collected data from 2,000 students, including extensive background information on file with the university, approximately 4,000 essays, and self-reported survey data. In the spring, we analyzed survey and performance data and found the intervention did not replicate prior results. Given the great care taken to replicate Hulleman et al.’s (2010) research methodology, we turned to students’ intervention essays in search of evidence to explain the null results, which revealed several important insights that can be used to refine future intervention efforts.
Our approach consisted of extrapolating data by developing a novel coding scheme based on exploratory analyses of students’ “self-generated” task value essays. Our results showed that students demonstrated varying levels of compliance, ranging from completely hypothetical to fully actualized task value applications. In addition, we found evidence of multiple types of subjective task value (as conceptualized in expectancy-value theory; e.g., Eccles et al., 1983), which evidence and theory suggests may have conflicting implications for students’ motivation (cf. Vansteenkiste et al., 2009). Further, students made few applications to the self and identified value primarily for professional research (as opposed to a broad scope of “real-life” domains). In this presentation, we provide qualitative evidence of these issues and discuss prudent future approaches in relevance intervention research. In particular, we offer a novel rubric to help future researchers target and identify more precise and theoretically important motivational variables.

Authors

• Jeffrey R. Albrecht, University of Michigan

• Nicole Rausch, University of Michigan - Ann Arbor

• Stuart A. Karabenick, University of Michigan