How Engagement During Out-of-School-Time STEM Programs Predicts Changes in Motivation in STEM

• In Event: Data-Intensive Approaches to Studying Engagement in Education: Exploring Their Current Potential

Abstract

Out-of-school time (OST) STEM programs have started focusing on STEM content due to the lack of individuals choosing to focus on STEM careers. Although adaptive outcomes have been found while youth are attending OST STEM programs, little research has focused on outcomes after the program has ended. Often, the goal of OST STEM programs is to increase youth’s long-term interest and competence in STEM. Therefore, examining youth interest and competence in STEM after spending time in the program is key. Framing the study around Emergent Motivation Theory (Csikszentmihalyi, 1990), we use a profile-oriented approach to investigate the relationship between youths’ momentary engagement and their interest and perceived competence at program completion. Research questions include: 1) What momentary profiles emerge? 2) What profiles are predictive of interest and perceived competence after attending an OST STEM program?
Data were collected from 203 youth in nine OST programs, each lasting four weeks. Through an Experience Sampling Method (ESM) approach, youth were signaled through mobile phones, yielding 2,463 total ESM responses. Youth were asked to complete a survey before and after the program. A two-step cluster analysis was used to identity momentary profiles. Multilevel Modeling was used to account for the nesting of momentary responses within the nine OST STEM programs.
Four profiles were selected on the basis of fit indices, cross-validation, and concerns of parsimony and interpretability in light of theoretical accounts (See Figure 1). The profiles represent both the conditions of engagement (as indicated by students’ perceptions of challenge and skill) and the experience of engagement (as indicated by behavioral, cognitive, and affective dimensions). For subsequent analyses, the profile assignments were aggregated at the individual-level as proportions, so that each student had a value between 0 and 1 representing the proportion of their responses in each of the profiles. Results of multilevel modeling with Universally low specified as the comparison group, suggest that the proportion of youth’s responses in Skilled but not challenged and Fully engaged predicted positive and statistically significant changes in interest and competence. However, the magnitude of the effect for the relationship between Skilled but not challenged and change in interest (𝛽 = 1.01) and competence (𝛽 = 0.75) was much stronger than Fully engaged, with more modest relations with interest (𝛽 = 0.76) and competence (𝛽 = 0.55).
This study provides evidence that in the context of OST STEM programs, not only engagement, but also its conditions - perceptions of challenge and skill - foster changes in youth's interest and competence. Although spending more time in profiles with high levels of challenge, skill, and engagement was associated with positive, significant changes in interest and competence, these changes were smaller than those in the profile with more modest levels of engagement, high levels of skill, and low levels of challenge, pointing to potentially unique features of the OST context. The use of ESM in combination with a person-oriented approach helped to provide a richer and more complete understanding of outcomes from engaging experiences and their conditions.

Authors

• Joshua Michael Rosenberg, The University of Tennessee - Knoxville

• Patrick Neil Beymer, Michigan State University

• Jennifer A. Schmidt, Michigan State University