Iconic Gestures Enhance Vocabulary Acquisition: A Virtual Pedagogical Agent as Foreign Language Trainer

• In Event: 33.078 - Roundtable Session 11
  In Roundtable Session: 33.078-16 - The Learning at Hand: Gesture Production in Virtual Pedagogical Agents

Abstract

Objectives
A growing body of scientific evidence has demonstrated that performing an iconic gesture together with a novel word in a foreign language, enables faster learning and makes those words more resistant against forgetting (Kelly et al. 2009, Macedonia et al. 2010, Macedonia & Knösche 2011, Quinn-Allen 1995, Tellier 2008). At this point, virtual agents provide a flexible way to support foreign language learning. The work reported here aims to compare gesture-enriched vocabulary acquisition achieved with the help of a virtual agent vs. the help of a human trainer.

Perspective(s) and theoretical framework
The supporting function of gesture performance in foreign language acquisition is based on the so-called enactment effect. This effect has been documented for various verbal materials (verbs, phrases, actions with real objects, common, and bizarre actions), for diverse populations (children, students, elderly subjects, people with memory impairments) and with different tests (i.e., recognition, free, and cued recall (for a comprehensive overview see Zimmer, 2001).

Method
The study employed a within-subject design manipulating the type of training in terms of (1) gesture-based training with human stimuli, (2) gesture-based training with virtual pedagogical agent stimuli, and (3) a control condition without any gestures. A total of 32 participants learned 45 vocabulary items on three consecutive days. Short-term learning performance was measured the next day prior to the training session, respectively. Long-term effect of information decay was measured additionally four weeks after training was finished. In every test session, participants conducted a free and thereafter a cued recall test.

Materials
Training materials comprised 45 nouns in German and Vimmi, an artificial corpus created for experimental purposes to avoid prior associations and to control for different factors that might constrain the memorization of particular vocabulary items (Macedonia et al. 2010). Human gesture stimuli were video-taped with a human actress. The agent's gestures were modeled to match the human gestures as much as possible. For examples of both human and agent gesture stimuli see Figure 7.

Results
For both types of long-term measures (free and cued recall) memory performance for items learned in the agent condition outperformed memory performance for items learned in the control condition. The same effect was present for short-term measures of free recall. Notably, in all tests performed, there was a trend for the virtual agent leading to better memory performance than training with a human (for details see Bergmann & Macedonia 2013).

Scientific and scholarly significance of the study
The present study provides first evidence for a virtual agent to support training. These findings doubtlessly need follow-up studies to elucidate which factors and constraints constitute the beneficial effect of the virtual character. Nevertheless, together with further evidence from gesture-supported language learning beyond vocabulary acquisition (Macedonia & Knösche 2011), they clearly substantiate the view that virtual pedagogical agents might play a crucial role in future language learning.

Author

• Kirsten Bergmann, Bielefeld University