Computer Vision Affordances for Child-Robot Interaction

• In Event: 28.026 - Designing Robots for Children: Multisensory Interactions and Multimodal Assessments

Abstract

Recent advancements in computer vision are providing exciting new opportunities to develop child/robot learning experiences that both respond to participant involvement and engagement interactively, and measure learning intervention efficacy. Addressing the question Q5, this presentation will discuss an array of these computer vision techniques and their applications. The presentation will also introduce a case study of an interactive experience for reinforcing emotion recognition for children with autism (Washington, 2017). Addressing the question Q3, we will also discuss key challenges in the field, particularly the impact of diversity on facilitating equitable learning contexts.

Face tracking, emotion recognition and pose estimation are areas of computer vision that can provide educators with important insight into the reactions, body language and overall engagement of the participants in a learning experience. Recent advancements allow for these techniques to be employed in real-time via increasingly affordable technology. One example of this, “Superpower Glass” (Washington, 2017) leverages these techniques and wearable and mobile technologies to reinforce emotion recognition for children with autism. While a child socially interacts with others, the system tracks the emotional responses of the participants and provides reinforcing cues for the child as to the emotions they may be seeing. A session review app allows the child and caretakers/educators to replay the video of an interaction and the emotions that were detected, fostering further discussion. Initial studies have found that children with autism strongly connect with the technology, show increased socialization skills, and exhibit them in contexts outside of the learning sessions themselves (Washington, 2016; Voss, 2016; Washington, 2017). The session videos are analyzed for engagement and efficacy using the same face tracking and emotion recognition capabilities used for the sessions themselves – demonstrating how such techniques can serve to both create interactive learning experiences and assess their impact.

Other recent advancements in computer vision extend these capabilities beyond learners to the spaces and objects around them. Object detection and semantic segmentation focus on providing insight into the overall understanding of a scene – not just tracking the people and objects within view, but also determining the relationships between them. Equipping child/robot interactions with this level of contextual awareness opens exciting opportunities to foster collaboration and track the progression of complex multi-step learning processes.

However, in order for these capabilities to translate into experiences that are equitable for all learners, diversity in the field of artificial intelligence, as with STEM more broadly, still remains a key challenge. Ethnic and gender underrepresentation seen in academic programs and in the workplace are also reflected in the massive datasets used to drive these technological advancements. As a result, we risk providing educators focusing on increasing engagement of underrepresented groups with tools that do not perform as effectively for those very learners. Thus, bridging these diversity gaps is critical for the creation of equitable learning contexts. Addressing these challenges will then allow these advancements in computer vision, along with speech recognition and similar technologies, to fully enable exciting new multimodal opportunities to create interactive, collaborative child/robot experiences for all learners.

Author

• Aaron Kline, Stanford University