Validating a New (Practical) Measure of Community College Student Motivation and Engagement: Results by Age and Race/Ethnicity

• In Event: Bridging Research and Practice With “Practical Measurement”: The Case of Improving Developmental Mathematics in Community Colleges

Abstract

The purpose of this presentation is to demonstrate the validity of the practical measure of community college student motivation and engagement. Measures were examined based on the four primary drivers to student success in math: 1) engaging nature of the course material, 2) students’ skills and habits for college success, 3) students’ views of themselves as math learners, and 4) students’ ties to peers. Regression analyses of these measures by age and race/ethnicity were used to predict students’ performance scores on an embedded assessment in the pilot survey and their reported math grades.

Research supports the notion that specific skills and mindsets that students acquire predict student motivation, engagement, and achievement more than others. For instance, students who perceive course material as personally interesting or relevant (Hulleman & Harackiewicz, 2009; Destin & Oyserman, 2010), acquire the necessary skills and habits to effectively complete college work (Zimmerman et al., 2011; Duckworth et al., 2011), perceive themselves as capable, growing math learners (Blackwell, Trzesniewski, & Dweck, 2007), and perceive a sense of belonging and connectedness in their social environment (Bahr, 2010; Walton & Cohen, 2011) tend to exhibit higher motivation, engagement, and achievement in school.

Methods:
Our analytic method is unique in that it privileges validity over internal consistency reliability (Cronbach’s alphas). In fact, internal consistency reliability is virtually uncorrelated with validity (McCrea, Kurtz, Yamagata, & Terracciano, 2011). Hence, we focus on objective validity criteria (i.e., test performance) to establish conclusions about the utility of our measure. Multiple regression analyses were conducted with test performance on a 20-item Math Test and self-reported average Math Grades in high school and college as the dependent variables.

Participants:
A diverse national sample of 500 college students was randomly selected from an online panel maintained by Luth Research. They were 25% Black, 25% Hispanic / Latino, 25% Asian, and 25% White. Half were over 30 and half were under 30. As a result, we were adequately powered to test for differential validity within Age X Race/Ethnicity sub-groups.

Results:
Multiple regression results showed that six of the nine independent variables (interest and relevance, long-term goals, anxiety in math and statistics, negative attributions, negative stereotypes, and social ties to peers) significantly contributed to the prediction of student performance on a 20-item Math Test, all ps < .05, Adjusted R2 = .35.

Furthermore, nearly identical results were found in a model predicting self-reported math grades. Six of the nine independent variables (interest and relevance, long-term goals, choice, basic study skills, anxiety in math and statistics, and negative stereotypes) significantly contributed to the prediction of student math grades, all ps < .05, Adjusted R2 = .41.
Interaction effects in regressions demonstrated that our measures were equally predictive regardless of Race / Ethnicity or age group.

These findings indicate that this brief (modal response time: 3 minutes) self-report measure accounted for nearly 35% of the variance in predicting students’ performance on a 20-item Math Test and 41% of the variance in predicting college math grades.

Author

• David Scott Yeager, Stanford University