Assessing Youths' Engineering Skills in a Participatory Action Research: Manifestation of Engineering Habits of Mind in a Museum Fellowship Program

• In Event: Transfer for Learning Outside the Classroom: Informal Contexts to Examine STEM Learning Skills and Strategies

Abstract

One way to motivate youths to pursue engineering careers is to help them better understand how engineers respond to the challenges of everyday life in order to develop a way of thinking, knowing, and doing that is unique in engineering practices. The National Research Council's Committee (2009) outlines these engineering principles as the "engineering habits of mind" such as collaboration and system thinking. However, it has been challenging for educators and researchers to assess students’ understanding and development of these principles in real-life context.

In this study, we invited two groups of high school- and college-aged museum educators to participate in a year-long fellowship project in which they were learners as well as educators of engineering habits of mind in a science museum. The fellows first learned about engineering habits of mind with project researchers and then worked together with museum staff to design and facilitate engineering-focused events, programs, and activities for museum visitors. The goal was to examine how creating and delivering these experiences for visitors could have a positive impact on the youth participants’ understanding of the engineering disciplines, and on visitors’ perceptions of engineering and its relationship to everyday life.

We used quasi-experimental design to assess the fellows’ understanding of engineering habits of mind by three aspects. First, we adopted a scenario-based problem-solving task, the Engineering Professional Skill Assessment (EPSA), developed by McCormack and colleagues (2014), to assess fellows’ engineering professional skills indicated by the Accreditation Board for Engineering and Technology (ABET). Second, we collected qualitative data of youths’ understanding and implementation of engineering concepts through their activity design process. The process of designing and developing engineering activities and programs indicated how the fellows conceptualized and communicated engineering concepts. We also conducted pre- and post-interviews to gather the youths’ self-reflection of their learning experience. Lastly, we collected feedback from visitors who participate the engineering programs via short interviews and surveys.

The preliminary data of the first year of the study suggested that a) the project successfully increased the fellows’ understanding of engineering habits of mind and related engineering skills, comparing the pre and post EPSA, during the project year; b) the qualitative data analysis showed that the youths not only incorporated engineering habits of mind when designing the program activities, but also used the engineering habits of mind as criteria to evaluate the quality of their activities; and c) even though visitors valued engineering education and informal learning opportunities, they may not see the role that young museum educators play in providing quality, engaging engineering learning activities. The full analysis of year-one results and preliminary year-two results will be available to share at the conference.

The study showed the potential of using unique triangulation of data to conduct holistic and contextual assessment of youths’ understanding of engineering concepts. The project also highlighted the benefits of having these youths as stakeholders and leaders who took the responsibility of designing and developing public events and programs, and brought their knowledge, passions, and personal stories to improve visitors’ learning experience.

Author

• ChangChia James Liu, New York Hall of Science