Optimal Learning Moments in Elementary Science Using in Situ Surveys: A Repeated Measures and Validation Study

• In Event: How Can Elementary Science Curriculum Support Student Learning, Teacher Practices, and Inclusiveness in Classrooms?

Abstract

Objectives
This paper explores an optimal learning survey (OLM) that measures 3rd graders' engagement (i.e., social and emotional responses) in situ during their science classes. It examines the item validity of the surveys using a series of repeated measures for one of four units over five weeks.

Theoretical Framework
Few studies have examined the impact of engagement on elementary science learning based on involvement in a new science curriculum throughout an academic year with repeated validated measures. This study aims to further research that has shown positive relationships between students’ engagement in the classroom and science achievement (Grabau & Ma, 2017). Engagement is defined as interest, skill, and challenge (Authors, 2016). Interest is defined as students’ response to phenomena experienced during lessons and their investment in explaining the phenomenon. Skills are the competencies students need for task accomplishment. Finally, challenge provides opportunities wherein students must take risks in their learning. Challenge and interest influence students’ motivation to use their skills to accomplish tasks (Authors, 2016). Learning surveys were created using the experience sampling (ESM) method, a type of measurement used to explore engagement as it occurs in context (Csikszentmihalyi, et al., 2006).

Methods and Data Sources
The ESES team designed ‘in situ’ surveys to be given after six focal lessons in each of the four units in the curriculum. This study focuses on one of the four units, forces and motion. Teachers administered the survey, consisting of questions pertaining to the driving questions, modeling, investigation, and development of a final artifact to 3rd-grade students immediately following the lesson. Three questions were based on engagement: interest, skill, and challenge, and a fourth question rotated between collaboration, persistence, agency, time, and outcome by lesson. The items consisted of a 4-point Likert-scale, strongly agree, agree, disagree, strongly disagree). Students showed agreement circling icons of thumbs up and thumbs down for disagreement, if more than one response was given a median value was assigned. If 2 and 3 were both circled a student was assigned a score of 2.5.

Results
The 595 student sample from 25 classrooms produced 3,369 responses to engagement prompts across the four ESES units and 1,019 (126 students within 12 classrooms) within the Force and Motion Unit (Tables 1 and 4). Tables 2 and 5 show response frequencies for interest, skill, challenge, and the varying component (Q4) for the overall survey and the Force and Motion unit respectively. The reliability of the OLM survey for the Force and Motion unit, using Cronbach’s Alpha is 0.50 (Table 6), just slightly lower than the overall reliability of 0.53 (Table 3). An exploratory factor analysis showed the OLM was unidimensional, therefore, measuring one underlying latent trait: engagement.

Significance
Measuring students' engagement, in situ gives education professionals a deeper understanding of what is happening in a student’s subjective emotions during an activity. Obtaining a snapshot of curriculum materials that are positively affecting students’ emotions, can provide valuable information for creating engaging lessons and activities now and in the future.

Authors

• Kayla Bartz, Michigan State University
  Presenting Author

• Cory S MIller, Michigan State University
  Non-Presenting Author

• Kathryn M. Bateman, Museum of Science - Boston
  Non-Presenting Author

• Lydia Bradford, Michigan State University
  Non-Presenting Author