Student and Teacher Ratings of Student Motivation: Multiple Measures or Multiple Traits?

• In Event: 65.071 - Measuring Social and Emotional Learning Skills Using the Mission Skills Assessment

Abstract

Intrinsic motivation, broadly defined, refers to being moved to action due to inherent interest or enjoyment. The Mission Skills Assessment (MSA) includes a measure which captures a general enjoyment of school and love of learning. This operationalization is consistent with Gottfried (1985), who found intrinsic motivation could indicate a general orientation toward enjoyment of learning.

General measurement best practice recommends multiple measures of student attitudes such as teacher ratings of students alongside students’ self-ratings. However, numerous studies have found that student and teacher ratings are only moderately related, indicating they may not actually measure the same trait (Cleary & Callan, 2013; Marsh, Smith, Barnes, & Smith, 1983; Park & Peterson, 2006). The purpose of this study was to explore the relationship between teacher and student ratings of student intrinsic motivation.

We utilized multilevel structural equation modeling (e.g., Preacher, Zyphur, & Zhang, 2010), which allowed us to investigate the measurement properties of the scale while taking into account variability within and between teachers.

The sample included 15590 students and 1294 teachers across 65 schools. On average each teacher rated 10 students, and one teacher rated each student. The 28-item intrinsic motivation scale included 16 items measuring school-specific intrinsic motivation (e.g., “I look forward to learning new things in school”) and 12 items measuring a general love of learning (e.g., “I love learning”). Teachers responded to seven of the items that focused on love of learning, with adapted language (e.g., “Loves learning”). All items had a 4-point response scale ranging from (1) “Rarely or never” to (4) “Usually or always”. Teachers also reported how well they knew the student and how exceptional they thought the student was.

Teachers varied in their congruence with student ratings, and there was a general trend of teachers rating their students higher than students rated themselves. We compared numerous multilevel measurement models. Typical measures of model fit indicated a two-factor solution with one student factor and one teacher factor best fit the data (CFI=.98, TLI=.97, RMSEA=.04). The two factors were correlated at .35 at the within-teacher level and .15 at the between level. The teacher items had large ICCs, with 20-30% of the variance occurring between teachers, whereas only 2-4% of the variance occurred between teachers for the student items. Within-teacher variability was significantly predicted by numerous factors.

This is the first large-scale investigation of congruence between teacher and student ratings of motivation. We found a moderate correlation between teacher and student ratings, as is consistent with prior research, but our large sample allowed us to further explore the relationship. This revealed sizeable variability in rating congruence, suggesting that many teachers may be evaluating a different construct than students. Cumulative GPA and teachers’ exceptional ratings explained the most variance in teacher ratings, implying that teachers may rely too heavily on student achievement information to infer student intrinsic motivation.

Authors

• Jessica Kay Flake, York University

• Kevin Terrance Petway, Educational Testing Service