Evaluation of the Mobile Making Program: Scalability and Sustainability

• In Event: Scaling Out, Up, and Deep: Mobile Making Afterschool Program across Four California State University Campuses

Abstract

Afterschool programs play an important role in broadening participation in STEM and Making (Krishnamurthi, Ballard, & Noam, 2014), however, sustaining and scaling high-quality afterschool programs continue to be a challenge due to funding, resources, and staffing (Bevan & Michalchik, 2013). The evaluation of the Mobile Making program across four CSU campuses is guided by the following questions:
1. What aspects of the after-school Making program implementation hold constant, and which are required to change, as the program is scaled across different contexts?
2. Which design principles previously identified are essential for the success of the Mobile Making program?
3. What new design principles emerge as the program is implemented across different settings and contexts?
4. What quality indicators prove necessary for scaling and sustaining the Mobile Making program?
Drawing from the literature on best practices for the scalability and sustainability of STEM afterschool programs (Koch & Penuel, 2014; Weis, Eisenhart, & Cipollone, 2015), we share the development and revisions of the youth and undergraduate measures used across four CSU campuses to document youth’s STEM and Making interest, self-efficacy, and perceived relevance as well as undergraduate students’ development of 21st-century skills, STEM teaching interest and self-efficacy. In addition, we will discuss the unique challenges faced in different demographic and geographic settings, such as urban, suburban, and rural communities, and how these factors influence the implementation and outcomes of the program. By exploring the diverse contexts and environments, we aim to identify the core design principles that contribute to the program’s success and those that require adaptation. Findings will be discussed within the broader context of supplementary and contradictory roles of research and evaluation, highlighting the importance of flexible program design and ongoing assessment to meet the evolving needs of participants and facilitators. This comprehensive approach ensures that the Mobile Making program not only reaches a wide audience but also maintains its impact and relevance across varying informal learning environments.

Authors

• James Marshall, San Diego State University

• Sinem Siyahhan, California State University - San Marcos

• Edward Price, California State University - San Marcos