Expert Evaluation of Student Presentations to Assess Learning in an Interactive Digital Dome: Opening the Gates of Horus

• In Event: Assessing Learning in Embodied Mixed-Reality Environments

Abstract

Ideally, Mixed Reality provides a meaningful, interactive, embodied experience, which integrates the personal physical presence of the user. Winn (2003) describes learning activity, as a mutual adaptation between the student and the virtual environment. Unfortunately, standardized written tests are so greatly MR experience, that there is much their fail to measure. Furthermore, some students maybe capable with visual spatial tasks even spoken English, but be very poor quiz takers. Furthermore, it is not enough to demonstrate that an augmented or virtual reality application is an adequate instructional tool. We must understand when and how it works to situate it in a larger curriculum.

Relevant empirical research is remarkably rare, but experiments by Salzman, Dede, and Loftin (1998), Bailenson (2008), and Jacobson (2011) show that an egocentric view of something (a magnetic field, virtual classroom, Egyptian Temple) assists learning some types of material. Also, Winn (1997) showed that “low ability” students gained more from building virtual environments.

We developed an instructional game that can run equally well on the desktop computer or an all-digital dome display, Gates of Horus (Jacobson, 2009). The student gains access to progressively more hidden areas of the Temple by answering questions in a dialogue with the virtual Egyptian priest. Some students played Gates of Horus on a standard desktop computer, while others used and all-digital dome. Students employing the dome learned more (Jacobson, 2011).

After they played the game, we tested the students in a number of ways (maps, questionnaire, etc.) but only one was fruitful. Each student produced had recorded a guided tour of the Temple, which was independently scored by several independent evaluators. Three educators used a long checklist, assigning points for each fact or idea. Students who used the dome conveyed more facts in their presentations (p < 0.05). Interestingly, graders’ responses to a single subjective question for each student, “Is the student doing a good job of reciting the facts of the temple?” showed an even stronger sensitivity to the experimental treatment (p < 0.01).

We also had an Egyptologist score the student videos in a similar way, answering somewhat more complex questions, with much the same result. Individual questions yielding an aggregate difference of p < 0.05, while his answer to a single general assessment question yielded p < 0.001. Much later, we had a more senior Egyptologist evaluate the student videos again producing a strong result, p < 0.016 for the general question

We believe that having the students make videos was successful, because it was just as visual, spatial, and verbal as the experimental treatment. It also had the advantage of by passing literacy test taking skills, which were not the objectives of the learning experience. Finally, posing very general questions to expert evaluators may be the most effective way to measure overall student performance, because it allows for more synthesis and judgment than a long list of atomized questions.

Author

• Jeffrey Jacobson, PublicVR