Momentary Engagement Profiles: A Person-in-Context Approach to Studying Student Engagement Using Experience Sampling Data

• In Event: 30.030 - Intensive Data Approaches to Students' Day-to-Day, Moment-to-Moment Emotions, Motivation, and Engagement

Abstract

Objectives. The purpose of this paper is to apply person-oriented analytic techniques to intensive data gathered via the Experience Sampling Method (ESM) in order to examine profiles of students’ momentary engagement in high school science classrooms. Specific goals are to: 1) test the viability of multidimensional engagement profiles using cluster analytic techniques; 2) explore whether engagement profiles are related to instructional practices; and 3) explore how engagement profiles are related to person-level characteristics like gender and ethnicity.

Perspectives. The study is framed by literatures on academic engagement and person-oriented approaches to understanding human experience. Engagement is generally construed as a multidimensional construct that is dependent on context (Fredricks et al., 2004), yet few studies measure engagement in ways that accommodate both of these defining characteristics. The ESM is ideal for gathering in-the-moment indicators on multiple dimensions of engagement, and generates sufficient data to examine how these dimensions coalesce in authentic learning environments. This person-oriented approach provides a complementary perspective to variable-oriented approaches which tend to isolate independent effects of particular dimensions, rather than consider them holistically. This study extends work utilizing person-oriented approaches by developing profiles that capture the person in a particular context, thus enabling detailed analysis of both the personal and situational factors associated with intrapersonal variation.

Data Sources and Methods. Data were collected from 244 students in 12 high school science classrooms.
ESM. In response to pagers that signaled students at randomly selected time points over 10 days of class, students responded to multiple items which were used to construct composite measures of behavioral, cognitive, and affective engagement that are aligned with existing measures. In total, 4,136 ESM responses were collected.
Video. Classrooms were videotaped on the days students completed the ESM and ESM data were linked to video data detailing classroom activities at the time of each signal.
Analytic Approach. Students’ momentary composite reports of behavioral, cognitive and affective engagement were used to identify profiles using a two-step cluster analysis procedure with hierarchical and k-means clustering (Bergman & El-Khouri, 1999). ANOVA was used to test the relationship between profiles and instructional activities. Within-person variation in engagement profiles was examined using additional clustering techniques.

Results. The final cluster solution included 7 momentary profiles which represent unique combinations of behavioral, cognitive and affective engagement. Profiles were validated using internal and external criteria. The cluster solution provides some empirical support for an existing theoretical taxonomy of engagement (Conner and Pope, 2013, see Figure 1). Students’ profiles differed as a function of their instructional activity at the time of their report. While gender did not account for appreciable variation in students’ profiles, race and ethnicity did, as did the interaction of gender and ethnicity.

Significance. This paper presents a novel way to measure student engagement and a unique approach to analyzing ESM data. Results deepen our understanding of the nature of student engagement, advance the relatively new field of person-oriented approaches to research, and provide useful information about student experience for practitioners interested in creating optimally engaging learning environments.

Authors

• Jennifer A. Schmidt, Michigan State University

• Joshua Michael Rosenberg, The University of Tennessee - Knoxville

• Patrick Neil Beymer, Michigan State University