Using Bayes Nets to Measure Learning in E-Learning Environments

• In Event: New Measurement Paradigms: Psychometric Methods for Technology-Based Assessments

Abstract

This paper describes how Bayes Nets are being used to measure learning in e-learning environments that have been developed and tested by several projects that are part of the CADRE group sponsored by the National Science Foundation. The purpose of the presentation is to explain the principles of Bayes Nets and illustrate how they are useful in complex assessment situations.
Bayesian networks, or Bayes Nets (BNs) for short, have a long history of use in electronic environments, dating back to Judea Pearl who first developed techniques to build BNs (Pearl, 1988). A Bayes Net is a probabilistic graphical model that represents a set of random variables and their conditional independencies via a directed acyclic graph. In the BN nodes represent random variables and the edges (links between the nodes) encode the conditional dependencies between the variables. Across a series of nodes and edges a joint probability distribution can be specified over a set of discrete random variables. This paper is not intended to present a comprehensive discussion of the origins of BNs and the wide variety of ways that they have been applied in fields as diverse as astronomy and speech recognition, but rather to describe the ways that they are used in a current sample of e-learning and e-assessment projects to measuring complex learning and to discuss their advantages and limitations. Bayes Nets, in this setting, form a class of Diagnostic Classification Models (Rupp, Templin, & Henson, 2010) that have been widely used in intelligent tutoring systems to predict student behavior and make tutoring decisions (Woolf, 2009), and their use in systems for assessment is growing. Martin and VanLehn (1995) and Mislevy and Gitomer (1996) studied the applications of BNs for student assessment. Mislevy has continued this work with Behrens in the NetPass program which assesses examinees ability to design and troubleshoot computer networks (Behrens, Frezzo, Mislevy, Kroopnick, & Wise, 2008). Conati et al. (2002) applied BNs to both assessing students’ competence and recognizing students’ intentions.
Within the CADRE projects that have been funded by the National Science Foundation, BNs are currently being used in various ways. Researchers at WestEd is using a BN system (among other methods) in the SimScientists simulation-based science assessment (www.simscientists.org). The Crystal Island, an intelligent game-based learning environment, that was developed by the IntelliMedia group at North Carolina State University (http://www.intellimedia.ncsu.edu/projects.html) uses Dynamic Bayes Nets to model learning and provide hints to eighth grade science students. The ASSISTment system developed by researchers at Worcester Polytechnic Institute use BNs to estimate probability that students possess particular math skills based upon their responses to multiple choice questions from Massachusetts’ state test and other scaffolding questions. The JavaTutor project, also part of the IntelliMedia group at NCSU, is using a related method to BN that is attempting to learn hidden Markov models (HMMs) to discover the structure of task-oriented tutorial dialogue in order to develop a dialogue based tutor to teach Java programming.

Author

• Michael Timms, Australian Council for Educational Research