A Text Messaging–Based Experience Sampling Method Study of Students' Interest in Introductory Computer Science

• In Event: Multimethod Approaches to Understanding STEM Outcomes: The Importance of Scientific Thinking, Motivation, and Engagement

Abstract

Computer science (CS) is growing as an area of study and work, driving much of the growth in STEM majors and careers (National Academies of Sciences, Engineering, and Medicine, 2018). However, there are long-standing issues around how accessible and supportive the CS discipline is for those who wish to pursue it (Margolis et al., 2000; Sax et al., 2017). These concerns have become even more salient as CS has come into focus at the K-12 levels (Codding et al., 2019; Leonard et al., 2018) and as CS occupations have become lucrative (Lewis et al., 2019). While research has shown how what students believe about themselves (particularly their competence-related beliefs) can shape their pursuit of an interest or major in CS (Lishinski et al., 2016; Wilson, 2002), scholars have not addressed the way that finer grained experiences contribute to sparking students’ interest in the field.

One way to study these more fine-grained, situation-specific experiences is with the Experience Sampling Method (ESM), which involves collecting data on individuals’ experiences at much more frequent intervals than traditional survey research. To study the factors that influence the development of interest in CS for novice undergraduate students, we developed an app Short Message Survey that enabled us to send short surveys to students via text message immediately after class, in order to access students immediate experiences in CS, 14 times over the semester. This ESM data enabled us to understand how students; experiences changed over the course of the semester and related to their semester-long outcomes.

Using our Short Message Survey app, we asked students to rate their current feelings of frustration, accomplishment, challenge, and confidence in their abilities using 5-point likert scale items. Our data collection was successful, with a 73.2% response rate, and we also succeeded at keeping the ESM data collection as unobtrusive as possible. Participants were students in one of four laboratory sections of an introductory CS class at a large university in the Southeastern United States. 32 students participated in the ESM data collection. The ESM data were analyzed using multivariate multilevel models to account for the nesting of ESM responses within individual students. The models revealed that the majority of the variance was found at the student level rather than the time point, indicating that differences in feelings of frustration, accomplishment, challenge, and confidence were mostly due to differences between students than differences in what is happening in class week-over-week.

The results of our data analysis showed a significant, positive correlation between students’ feelings of accomplishment and confidence week over week with their change in interest in CS from the beginning of the semester to the end of the semester. These results suggest that students whose experiences in class make them feel accomplished and confident are more likely to become more interested in the field of CS. These results speak to how motivation in CS can be understood using ESM and highlight how specific, momentary experiences can spark students' interest from before to after an introductory CS course.

Authors

• Alex Lishinski, The University of Tennessee, Knoxville

• Joshua Michael Rosenberg, The University of Tennessee - Knoxville

• Omiya Sultana

• Michael Mann, University of Tennessee

• Joshua Dunn, The University of Tennessee - Knoxville