Communicating About Computational Thinking: Understanding Affordances of Portfolios for Assessing High School Students’ Computational Thinking and Participation Practices

• In Event: Assessing Computational Thinking: Updates and Advances in the Field

Abstract

While the assessment of computational thinking concepts, practices, and perspectives is at the forefront of K-12 CS education, supporting student communication about computation has received relatively little attention. Communication about computation involves describing and explaining computational artifacts and related processes and behaviors, and can be considered part of the larger category of computational participation (Kafai & Burke, 2014). This practice attends to Conley and Darling-Hammond’s (2013) broad call for assessments that include communication of ideas through the use of appropriate vocabulary in a subject area as well as the presentation of artifacts to a broader audience.

In this paper, we report on the use of portfolios in 15 introductory computer science high school classrooms from largely underserved communities in two major metropolitan school districts in the U.S. Southwest. We ask: 1) To what extent did students communicate about their computational projects and processes? 2) What can we learn about students’ engagement with computational practices, especially rarely assessed activities such as testing, iteration, and debugging, through these personalized reports on process?

The context of the study was the implementation of the 10-12 week electronic textiles curricular unit (http://exploringcs.org/e-textiles), an optional end-of-year unit that takes place within the introductory computer science course, Exploring Computer Science. Electronic textiles connect sewable microcontrollers with conductive thread to actuators such as LEDs as well as sound, light or touch sensors to make interactive craft projects. For their portfolios, students reported on the function of a final project in the e-textile unit as well as challenges they encountered during the process of making it, namely any problems or design changes they handled.

We examined the portfolios of 248 high school students in 15 introductory CS classrooms across 14 schools (school populations with 72-99% ethnically underrepresented groups, 47%-97% free and reduced lunch). A two-part analysis utilized both a formal rubric (top-down, part of the curriculum) to code computational communication and an open-coding scheme (bottom-up) to identify computational practices described. For the bottom-up analysis we applied a two-step, open-coding approach (Charmaz, 2006) to develop a coding scheme, before obtaining inter-rater reliability (.75) across both the rubric and coding scheme.

Our findings revealed that students who created process-based portfolios documenting their e-textiles projects generally illustrated a solid ability to write and visualize computational ideas within their portfolios, albeit with room for improvement, especially with annotating visual evidence. Students were also able to report on their computational processes, demonstrating engagement with practices such as testing and/or refining something iteratively, isolating problems and root causes, debugging, and even collaborating and designing for an audience (i.e., users). Because these portfolios were able to capture students’ engagement with these often under-assessed computational thinking practices, we consider these tools a powerful way to evaluate students’ computational performance. In the discussion, we consider the implications of using portfolios as assessments that complement more traditional measures such as tests, project evaluations, short programming tasks, or surveys. Portfolios meet a critical need to promote and assess computational communication skills.

Authors

• Deborah Fields, Utah State University

• Debora Lui, University of Pennsylvania

• Yasmin B. Kafai, University of Pennsylvania

• Gayithri Jayathirtha, University of Oregon

• Justice Toshiba Walker, University of Maryland

• Mia Shaw, New York University