Physics Playground Scoring and Activity Selection Algorithms

• In Event: 18.057 - Physics Playground, a Game for Learning: Demonstration and Posters

Abstract

The evidence accumulation (EA) process for Physics Playground (PP) is implemented using Bayesian networks based on an evidenced-centered assessment design framework (Almond, Mislevy, Steinberg, Yan & Williamson, 2015). At the hub is a central Bayesian network representing the competency model, and the spokes are evidence models—Bayesian network fragments associated with each game level. The scoring engine works as follows: (1) Each user is given a student model—an individual copy of the competency model in which information about that student is stored. (2) When information comes from the unity engine that a student has completed a particular level, the evidence identification (EI) engine is run to fetch the records of the student processing that level from the LearningLocker database and extract the key observable features for that student performance according to the specifications in the evidence model. (3) The Bayesian network fragment for the evidence model is attached to the student model, the variables representing the observables are instantiated to their observed values and the competency estimates for that student is updated.

If PP is being run in an adaptive mode, the activity selection (AS) process runs on top of the EA code. The AS algorithm consists of an outer loop and an inner loop. The outer loop follows the critiquing strategy (Barr & Feigenbaum, 1982) where a series of target competencies are chosen in sequence. In each case tasks and activities related to the target competency are presented to the student until the probability of mastery exceeds either a high or low threshold. If the student passes the high threshold, they are passed on to the next competency in sequence. If they fall below the low threshold, they are routed to remedial activities. In the inner loop, the expected weight of evidence (EWOE; Madigan & Almond, 1996) algorithm is used to select the task which provides the highest EWOE for the currently targeted proficiency. Shute, Hansen & Almond (2008) speculate that tasks with high EWOE are in the center of the zone of proximal development (1978) of the student.

Because the Bayesian network scoring model is closely tied to the evidence-centered design (ECD) models, it is necessary to be able to quickly update the Bayesian networks when the ECD models change and visa versa. The R package Peanut (Almond, 2018) automates this translation process, providing both network and tabular views of the assessment. It also uses the DiBello parameterization for the Bayesian networks, allowing cognitive scientists to specify relationships in terms of difficulties and discriminations instead of conditional probabilities.

Although the EI engine is conceptually a separate system, it is implemented in the same code base. It replays the events from the LL record in a finite state machine. As this machine runs, various rules run to note when observable outcome variables should change value. The resulting collection of outcome variables is logged to a database, and then sent to the EA engine to initiate the rest of the evidence accumulation and activity selection processes.

Authors

• Russell Almond, Florida State University

• Seyfullah Tingir, Florida State University

• Zhichun Liu, Florida State University