Virtual-Reality-Based Training of Representational Flexibility for Adolescents With Autism

• In Event: Emerging Technologies for Supporting Neurodiverse Learners

Abstract

This design-based research project studies individualized training of representational flexibility for computational and scientific problem solving among adolescents with autism spectrum disorder (ASD) in a virtual-reality (VR) based adaptive learning program called Force and Motion – Adaptive Representation (FM-AR). The facility to use multiple external representations and the ability to switch flexibly between them are critical indices for the practices of computational thinking and problem solving in the STEM domains in general. However, the development and enactment of representational flexibility in cognitive processes is effortful and demanding for most learners, but particularly so for students with ASD. Prior research observed that compared to non-autistic peers, students with autism and a high heterogeneity experienced difficulty in holistic processing, cognitive shifting, and categorical induction (e.g., encoding general similarity and thematic relations). Yet cognitive training that aims to promote representational flexibility among heterogeneous learners with special needs is limited.

Utilizing personalized learning and constructionism, FM-AR trains participants how to design and program motion-based simulations, ranging from interactive objects to computerized characters, in an open-source, 3D collaborative virtual world. FM-AR is aimed to facilitate participants’ ability to select, connect, convert, and construct multimodal representations of physics problems while practicing computational concepts (e.g., sequencing, iteration, events, and conditionals).

Via design-based, mixed-method research of the computational behaviors of autistic adolescents and their learning outcomes in the FM-AR program, we examined whether and how adolescents with ASD would practice and improve representational flexibility during FM-AR-supported simulation design and program. We adopted multimodal data mining with participants’ logged computational behaviors and verbal protocols (voice and text chats), for tracking and assessing the development and practice of representational flexibility in the FM-AR program.

The study results indicated a significant improvement in three key facets of representational flexibility of the program participants from the baseline to the intervention phase. Specifically, the paired samples t-tests indicated a significant improvement with a large effect size from the baseline to the intervention phase in participants’ performance of attention or mental set shifting, t(9) = -2.76, p < .05, Cohen’s d=.88; in their performance of using or generating multiple/alternative representations, t(9) = -2.57, p < .05, Cohen’s d=.82; and in their performance of pattern identification/development, t(9) = -3.31, p < .01, Cohen’s d=1.05. There was a numerical improvement in participants’ performance of pattern contextualization or application, but the result was not statistically significant, t(9) = -1.66, p = .13, Cohen’s d=.53. The visual analysis with the multiple-baseline across-participant graphs of the manually coded behavior measures of representational flexibility also demonstrated an increasing trend from the baseline to the intervention phase, confirming the data mining findings. There is also a significant and positive correlation between the data mining results and the external measure of cognitive flexibility (Wisconsin Card Sorting Test), Pearson's r=.66, p=.02. This provides an external validation for the multimodal data mining measures. The study findings will shed light on the design and research of a STEM+C learning environment that scaffolds representational flexibility for neurodiverse learners.

Authors

• Fengfeng Ke, Florida State University

• Jewoong Moon, The University of Alabama

• Zlatko Sokolikj, Florida State University

• Kelly Whalon, University of Florida

• Shayok Chakraborty, Florida State University

• Greg Hajcak, Florida State University