Evidence-Centered Concept Map in Assessing Critical Thinking Skills

• In Event: Technology Tools for Real-World Skill Assessments in Schools

Abstract

Using concept map engages students in a variety of critical and complex thinking, such as evaluating, analyzing, and decision making. Although the potential use of concept maps to assess students’ knowledge has been recognized, concept maps are traditionally used as instructional tools. An interactive Evidence-Centered Concept Map (ECCM) is one of the promising technology tools in making student thinking visible in critical thinking assessments (Rosen, 2014; Rosen, & Tager, 2014). When creating ECCM, students perform a task that no ordinary collection of notes may encompass. ECCM represents a personal visualization of claims and supporting evidence on a topic, as well as relationships between the claims. The tool also represents possible gaps in student’s analysis of the topic and the ability to make a valid and reliable evidence-based conclusion. In this way concept maps can be used as a formative tool to enhance teaching and learning. The stages of student work with ECCM on an assessment task include: (a) gathering various claims and evidence from the resources provided (some claims and evidence contradict one another); (b) organizing the claims with supporting evidence gathered in the previous phase on ECCM without hierarchical relationships; and (c) linking claims and specifying the kind of a relationship between claims.
A critical thinking assessment task ‘Organic Milk’ has been developed and administered both in ECCM and Notepad settings to one hundred ninety 14-year-old students from the United States, United Kingdom, Singapore, and South Africa. In all, 102 students participated in ECCM mode, and 88 participated in notepad mode.
In this critical thinking computer-based assessment task, the student was asked to analyse various pros and cons of whether or not to buy organic milk for the school cafeteria and write a recommendation to a school principal. Students who participated in ECCM mode were required to use a concept map during the analysis of web-based pre-determined resources, while students who participated in notepad mode were able to take notes by using an embedded free text notepad, but were not provided any kind of thinking tool.
Findings showed that students assessed in ECCM mode outperformed their peers in notepad mode in their critical thinking (M=69.9, SD=27.2 in ECCM mode, compared to M=54.5, SD=19.0 in notepad mode; ES=.7, t(df=188)=4.7, p<.01). Overall, decision making with a concept map involved significantly higher levels of analysis and evaluation of evidence, claims, and alternative points of view, as well as synthesis, making connections between information and arguments, interpreting information, and making inferences by using reasoning appropriate to the situation. Moreover, it was found that student ability to construct ECCM and the ability to create relationships within ECCM are positively linked to student performance in critical thinking (r=.62, p < .01 and r=.59, p < .01, respectively).
Future studies need to address the key psychometric issues related to the potential use of ECCM in large-scale accountability assessments.

Author

• Yigal Rosen, Harvard University