Implementing a Co-designed Social Bot Using AI to Support STEM Learning in VR

• In Event: Co-Design as a Means of Creating Inclusive and Effective Technology-Enhanced Learning Experiences

Abstract

This presentation will share how co-designed ideas about a Companion Bot character and recent advancements with generative AI have come together within virtual reality (VR) to potentially enhance an informal STEM-learning experience.

As part of the NSF-funded project [NAME], a neurodiverse team co-designed, researched, and developed a virtual reality (VR) experience for informal STEM-based learning (BLINDED CITATION). The team included eleven neurodivergent members, ten of whom were recruited from [NAME] College, a college exclusively for students who learn differently, including students with a learning disability (such as dyslexia), ADHD, autism, or executive function challenges. And for two years, this team co-designed many aspects of the VR experience to ensure it meets the interests and needs of a broad range of learners.

The original project design envisioned a robot character to provide guidance to players in the VR experience. This robot was intended to be a purely utilitarian attention and guidance tool for players. However, co-designers increasingly wanted to have a more social, friendly robot that they could interact with and relate to. Until recently, such a Companion Bot was not practical for the project to implement. Some co-designed elements, such as a highly emotive face, were integrated, but until recently, the deeper intent expressed by the neurodivergent co-designers was beyond reach. With the recent proliferation of generative LLM foundation models and generative AI, it became possible to implement and research an interactive social bot within the VR experience (Sonderegger, 2022).

In this session, we will describe how co-design helped inform the design and development of this social Companion Bot, as well as the research we conducted on its use by neurodivergent learners. In one VR-based study, we collected data on neurodivergent learners’ interactions with an AI bot prompted to only respond with short, factual information and another AI bot prompted to not only respond with short, factual information, but also to engage socially, with the definition of social based on what the co-design team determined would “make the bot feel friendly”. A comparison of these interactions, player impressions, AI bot limitations, and implications for future design are discussed.

The presentation will conclude with an overview of future directions for the project and team, as well as recommendations on how best to engage and support co-designers in shaping VR, AI, and other experiences for a broad range of learners.

Authors

• Teon Edwards, TERC

• Ibrahim Dahlstrom-Hakki, TERC

• Zachary Alstad, TERC