Assessing Computational Thinking: Practices Versus Concepts

• In Event: 61.039 - Assessing Computational Thinking: A "Landscape" Symposium About Where We Are at and Where to Go

Abstract

Objectives
The CS K-12 framework specifies that computational thinking (CT) can be thought of as as the collection of 4 computer science (CS) practices (K-12 Computer Science Framework, 2016). The assessments of practices can be challenging, as an assessment should measure the degree to which a student is engaging with the practices and not just what they know about the practice. In addition, one challenge when developing assessments of CT is should these practices be assessed separately from assessing CS concepts, and if so how?

Background
In the Principled Assessments for Computational Thinking (PACT) project, end-of-unit assessments were developed for the Exploring Computer Science curriculum (ECS), an introductory high school CS course. Assessments were designed to focus on practices, but drew from the content covered in the curriculum in such a way that the scores represent scores for student’s ability related to the unit and not solely on the students ability on CS practices. A different approach was developed for the Coolthink@JC project, in which assessments were developed for an upper elementary school CS course. In this project, two separate assessments were developed, one for CS Constructs (e.g., repetition and conditionals), and one for CS practices (e.g. algorithmic thinking).

Methods & Data Sources
Both projects used an evidence-centered design (ECD) approach (Mislevy & Haertel, 2006). One of the strengths of this approach is that it highlights the need to clearly define what it is that is being measured, and encourages the development of specifications around what evidence is needed, and how can tasks be structured to collect this evidence. One step in this process is the development of a design pattern, which includes the specification of the focal knowledge, skills and abilities (FKSAS) that are to be measured, and guidance on how these FKSAS can be measured. It is here that the relationship between content and practice can be specified.

Results & Next Steps
One FKSAs that we wanted to measure in the Coolthink@JC project was the ability to compare multiple approaches to solving a problem. This was measured separately from the knowledge of a codebase, and it was specified that tasks should contain simple solutions, not tied to a coding language, and should include trade-offs that students could use to compare. In the PACT project we had a similar goal, but also wanted to measure students’ ability to relate code structures to solutions. Therefore, for the PACT project the solutions presented was tied to a programming language.
While there are some practices, or parts of practices, that are easier to find ways to measure separate from CS concepts, there are others (e.g. testing computational artifacts) that are more challenging. Further discussion will involve when it is appropriate to measure practice separate from concepts, and what are ways in which these can be measured separately.

Author

• Daisy Wise Rutstein, SRI International