Students' Thinking Processes, Strategies, and Performance When Responding to PA Tasks of CT

• In Event: On the Comparability of Cross-National Assessment of Critical Thinking

Abstract

1. Objectives/Purposes
We examine the students' thinking processes and strategies of Colombian and Swiss teacher students for a specific PA of CT, “Migrants”. We ask: (1) Are different types of thinking processes and strategies linked to performance in CT? (2) Are the thinking processes and strategies equivalent in both countries?
2. Theoretical Framework
The link between thinking processes and strategies through think-alouds and performance has previously been studied in CT (Ebright-Jones, 2024), and in the processing of multiple documents (Anmarkrud et al., 2014; Cho, 2014).
In the research on strategic approaches to multiple documents, the document model serves as a basis to explore how students make sense of multiple documents both considering the taxbase and information about the sources and integrating information from different texts (Anmarkrud et al., 2014). Within this model, the proposed categories of monitoring and evaluation behavior especially resonate with the CT construct, including attitudinal dimensions of CT as well as the construct itself (in the evaluative component).
3. Method
To study the response process and strategies used by students as they approach the PA, we conducted case studies in Switzerland and Colombia. We implemented cognitive laboratories with the think-aloud process (Leighton, 2017) and follow-up interviews. The think-aloud process was converted into flow charts, visually showcasing the step-by-step process of each student as they read through the task instructions, diverse documents, and wrote their answer. We additionally coded and analyzed the think-aloud using the Critical Thinking assessment framework (Braun et al., 2020) as a cognitive model, and considered the monitoring and evaluation categories from the document model (Anmarkrud et al., 2014).
We distributed flowcharts of the 20 cognitive laboratories to three experts and tasked them with categorizing the flowcharts into distinct groups based on their similarities. Additionally, we utilized standard algorithms to create clusters. To approach performance in CT we scored students responses using an analytic rubric. We then contrasted the clusters of flow charts to the performance in CT.
4. Data Source and Analysis
The data sources include the 20 cognitive laboratories implemented in Switzerland and Colombia, as well as both the think-aloud protocols and the students’ produced essays.
5. Results
Overall, we find that students who monitor their performance more often and approach texts more evaluatively have a higher performance in CT. However, it’s interesting to find that sometimes an over-evaluative approach, especially if directed towards mistaken aspects, can also be linked to generalized skepticism, even of trustworthy sources, and thus to lower CT performance. This is coherent with previous research findings that highlight how interventions to combat misinformation can increase skepticism also for trustworthy information (Hoes et al., 2024).
6. Scholarly Significance
This work contributes to understanding process-related drivers of performance in CT that could be used to inform instruction.

Authors

• Natalia Ronderos, University of Zurich

• Richard J. Shavelson, Stanford University

• Henry I. Braun, Boston College

• Julian Patricio Mariño von Hildebrand, Universidad de los Andes

• Doreen Flick-Holtsch, University of Zurich

• Guillermo Solano-Flores, Stanford University

• Roza Nalbandyan, Stanford University