From Domain-General to Domain-Specific Assessment of Critical Thinking: The Case of Electricity and Magnetism

• In Event: Teaching Critical Thinking: Assessing and Improving Students’ and Teachers’ Reasoning Skills

Abstract

Introduction:
Critical thinking (CT) is one of the most desirable objectives of most, if not all, secondary and tertiary education curricula. The question then arises: how can CT be assessed? In this work we try to, in part, address this question. CT-tests are, at this moment, predominantly domain-general. However, some authors stress the importance of domain-specific knowledge for CT or stress the domain-specificity of CT (McPeck, 1981; Moore, 2004). Hence, there is a need for domain-specific tests to advance the field.
Confronted with educational goals that focus on CT on the one hand and the lack of validated tests to test domain-specific CT on the other hand, this study aims at constructing and validating a domain-specific test for physics (more specifically for electricity and magnetism – E&M). The test is aimed at secondary school students that have received instruction on E&M.

Method:
The development of the test followed design guidelines (Adams, & Wieman, 2010) in which several distinct phases in the development can be distinguished:
1) Construction of the theoretical framework based on Halpern’s construction of CT.
2) Construction of the test-items reflecting the structure of the Halpern critical thinking assessment, HCTA (Halpern, 2014).
3) Focus-groups discussion with in-service teachers regarding the understandability and answerability of the test-items for students of the targeted population.
4) Cognitive interviews (Willis, 2004) with students of the targeted population.
5) Large group administration (convenience sampling, N=162).

Results:
Internal consistency of the test was found to be . This value is relatively low, a value of 0.7 is often set as minimum (Sijstsma, 2009). However, for a test that measures complex cognitive abilities, as certainly CT is, a lower internal consistency might not only be expected but also wanted (Peters, 2014).
Given the time constraints only two thirds of the students (N=105) were able to complete the entire test. On average the students scored 9 ± 4.5 out of 46 points.

Discussion and Conclusion:
Even though the sample in this study cannot be said to be representative for all students in the targeted population, the low average score obtained by the participating students raises multiple questions:
1) Was the test was too difficult for the students? This seems unlikely as in stage 3 of the test-development the in-service teachers confirmed that all questions should be feasible for their students.
2) Are students not adequately prepared to think critically in their school career? A positive answer would be quite worrisome as this means that a major educational goal was not reached.
3) Is the test not valid as it is based on the translation of a domain-general framework to a specific domain?
Future research should try to answer the questions asked above, for example by comparing the domain specific CT skills probed in this test with domain general CT skills probed in the HCTA. Future research is also needed to design an educational methodology to increase the CT skills and disposition among secondary education students. An infusion approach seems promising (Davies, 2006; Tiruneh et al., 2016).

Authors

• Jan Sermeus, Odisee University College

• Mieke De Cock, KU Leuven, Belgium

• Jan M. Elen, KU Leuven