Interacting with a Conversational Agent promotes children’s science learning

• In Event: Young Children's Learning from Educational Media

Abstract

Children spend an increasing amount of time consuming screen media. Thus, it is important to understand how to better design educational media so that children’s learning can be maximized. Studies have suggested that incorporating opportunities allowing children to contingently interact with the media can improve learning (Calvert et al., 2019), sometimes comparable to learning from face-to-face interactions (Gaudreau et al., 2020; Roseberry et al., 2014; Troseth et al., 2006). However, while studies found the learning benefits increased with adding the contingent interaction element to media, a few studies have examined the impacts of using an animated character in the video instead of a person on screen.

In our study, we partnered with PBS kids to develop science focused educational videos in which the main character, powered by artificial intelligence, engages in contingent conversation with children. The goal is to compare children’s science learning outcomes under three different video watch formats.

We conducted a randomized controlled experiment recruiting 240 children between the ages of 3 ½ and 7 ½, most of whom were from low-income Latino families. Children were randomly assigned to one of the three conditions: 1) watching the conversational videos, and having contingent interactions with the main character, 2) watching the same video with pseudo-interactions where the character will ask the same questions as the conversational video condition, and then provide children generic response after pausing a fixed amount of time, 3) watching the broadcast version of the video. After watching two videos in each session in the assigned condition, children will complete post-test science assessments. To compare children’s science learning, we performed linear regression analysis to examine the effects on children’s science learning and control for children’s age, language, and science knowledge.

Our result showed that children having contingent interaction with the conversation agent performed the best in the post-test (M= 12.55, SD= 4.24) than those in the pseudo-interaction (M=11.3, SD=4.27) and those in the broadcast version (M=10.67, SD=4.54)(Table 1). The regression analysis showed that children in the broadcast version scored .26 SD lower than those in the interactive condition (p<0.05), holding children’s age, science, and language scores constant. We did not find a significant effect between interaction and pseudo-interaction. After adding three interaction terms to the models, we found no moderating effects between conditions and covariates (Table 2). By calculating the frequency of scientific terms (e.g., molted) taught in videos in children’s responses to open-ended questions, we found that children in the contingent interaction and pseudo-interaction reported a comparable number of terms. In contrast, children in the non-interaction condition reported much less. Our finding provided evidence that having contingent conversations with a media character during video watching can promote children’s science learning, consistent with the previous study (Xu et al., 2022), and highlighted the effectiveness and benefits of a conversational agent to reform Educational video designers to create more interactive tools that would elite rich conversations and learning opportunities for children during screen time.

Authors

• Shan Zhang

• Valery Vigil, University of California, Irvine

• Daniel Ritchie, University of California - Irvine

• Julian Levine, University of California - Irvine

• Ying Xu, University of Michigan - Ann Arbor
  Presenting Author