What Are Mental Models of Electronic Circuits? Basing an Assessment on Computational Simulations of Experts

• In Event: Cognitive Models for Assessment

Abstract

We are developing a tutoring system for a US Navy technician training school. The stated objective is that the tutoring system should teach students a “good mental model of common analog circuits, such as power supplies and amplifiers.” However, exactly what was meant by “good mental model” was not explained. This talk describes how we used cognitive models to help define this construct precisely enough to measure it.
In order to assess the students’ understanding of circuits, the tutoring system asks questions similar to those already used in the Navy school, such as “what happens when this switch is closed” or “If the load on the power supply drops slightly, what happens?” or “What is the function of the filter of a power supply?” Troubleshooting tasks play little role in the instruction, which is unfortunate because most cognitive modeling of electronics has focused on troubleshooting (Acchione-Noel & Psotka, 1993; Allen, Teague, & Carter, 1996; Besnard & Bastien-Toniazzo, 1999; de Croock, van Merriënboer, & Paas, 1998; Gitomer, 1988; Gugerty, 2007; S. D. Johnson, Flesher, Ferej, & Jehn, 1992; S. D. Johnson, Flesher, Jehng, & Ferej, 1993; W. B. Johnson & Norton, 1992; Jonassen & Hung, 2006; Lesgold, Lajoie, Bunzo, & Eggan, 1992; Lesgold, Lajoie, Bunzo, & Eggan, 1988; Morris & Rouse, 1985).
Fortunately, two cognitive models (de Kleer, 1984; White & Frederiksen, 1990) focused on modeling just the behavior and functions of circuits. They are computationally sufficient in that they were able to generate causal explanations and answers to questions similar to the ones asked by our tutoring system and the existing Naval training material.
However, the pieces of knowledge they used were far too specific and small for our purposes. Answering a simple question might involve a hundred pieces of knowledge, according to the models. Our solution was to define knowledge components that honored the ontology of the cognitive models and preserved a precise mapping to the fine-grained pieces of knowledge used in the models. This turned out to be much more subtle and complex that we first imagined it would be.
At this writing, the assessment system and tutoring system are operating with a simple counting-based method for interpreting the evidence. A method based on Bayesian Knowledge Tracing (BKT) is being developed.
These results may extend beyond electronics. Two fields, Qualitative Process Reasoning and Model-based Diagnosis, have shown that the types of qualitative models used by de Kleer, White and Frederiksen can be developed for a large variety of engineered and natural systems. Using our definitions of “mental model” and these computational qualitative models, future work may be able to precisely define and measure students’ mental models of global warming, ecosystems, biological homeostasis, and many other important systems in high school and university science and engineering curricula.

Author

• Kurt VanLehn, Arizona State University - Tempe