A Principled Approach for Designing Computational Thinking and Computer Science Assessments: Repurposing Summative CT (Computational Thinking) Assessments as Standards-Aligned Computer Science Formative Assessments

• In Event: Assessing Computational Thinking: Updates and Advances in the Field

Abstract

Computational thinking (CT) and computer science (CS) skills are increasingly important in today’s digital world. Our paper in the 2021 special issue on assessing CT describes the design, development, and validation of separate assessments of programming concepts and generic CT practices for 4th-6th grade students that were used with about 15,000 students in Hong Kong. In this session, we will discuss how we repurposed select tasks from our summative CT assessments as standards-aligned CS formative assessments. While summative assessments are typically used to determine students’ overall mastery and/or assign grades, formative assessments are designed to help teachers diagnose where students are in their learning so that they can determine the instructional next steps to best support their students.

Assessments designed and validated for summative purposes are not necessarily appropriate for formative purposes, but the use of a principled design process such as Evidence Centered Design (Mislevy & Haertel, 2006) can support the process of repurposing assessment tasks and identifying what needs to change to make tasks appropriate for a different purpose. Our goal for developing CS formative assessments was to support middle school CS teachers in better understanding their students’ learning and challenges on concepts underlying five CS standards from the ‘Algorithms and Programming’ strand, namely algorithms, decomposition, variables, combined control structures, and procedures.

One critical step in this process of adapting our existing CT assessments was identifying tasks appropriate for repurposing. Similar to our CT assessment design where we had decomposed each CT concept and practice into fine-grained learning targets, we also started our standards-aligned CS assessment design by decomposing each CS standard into a set of granular learning targets. We then compared the two sets of learning targets to determine their overlaps. While CT and CS are often used interchangeably, there are several differences between the two domains and CS encompasses a broader set of skills than CT. All five of our target middle school CS standards map onto either CT concepts (variables, repetition, conditionals, procedures) or CT practices (algorithmic thinking, decomposition, reusing and remixing), but CS standards are broader in scope. Tasks aligned to overlapping learning targets were identified and then examined for age appropriateness.

Once tasks were identified, the main modification involved the evaluation of student responses. For summative CT assessments, scoring guides were generated to assign points based on student responses. With these scoring guides there may be multiple ways to get to a specific score point. While we can say that students with higher scores demonstrated a greater level of ability, we cannot necessarily compare students with the same scores. For evaluating student responses for formative purposes, rubrics were developed that instead of assigning scores, categorized students in relation to the challenges they demonstrated. Two different types of rubrics were developed, one in which students were placed into one category that best described what students were able to do and what challenges they had. For the other type of rubric, students could be assigned multiple categories, each category corresponding to a specific challenge that students demonstrate.

Authors

• Satabdi Basu, SRI International

• Daisy Wise Rutstein, edCount, LLC