Multimodal Miracles: The Kinds of Literacy Experiences Video Games (and Their Developers) Enable and Inhibit

• In Event: From Online to Off-Line: Examining Gaming Environments and Innovative Learning

Abstract

Objectives
Videogames have been the subject of both enthusiasm (Gee, 2007; Prensky & Thiagarajan, 2007) and speculation (Buckingham, 2008) in the educational research community. The ambivalence of researchers, however, is not reflected in the minds of companies, philanthropic groups, and policy makers. If it is through political, commercial, and philanthropic coordination that current reforms gain traction (Picciano and Spring, 2013), we must understand more precisely how the ontology of videogames enable and inhibit perceived benefits of their use, including the learning environments they create (Abrams & Roswell, 2011).

Theoretical Perspectives
Like other software-powered technologies (Lynch, 2013), videogames are by their nature reductive phenomena. Videogames must be tightly pre-programmed to work. This creates a tension that is fundamental to the ontology of video games: players experience a sense of freedom, but that freedom is “illusory” (Manovich, 2001). Players aren’t free; their choices are tightly controlled by development companies. When examining the role of videogames in education, we must critique both the way videogames represent the “official knowledge” (Apple, 2000) of the conservative apparatus seeking to control culture and the degree to which videogames realize a commercial pedagogy that steers teaching and learning toward commercial rather than educational ends (Picciano & Spring, 2013).

Data Sources & Methods
I employ a multimodal social semiotic analysis of Assassin's Creed III in six classrooms over a two-month period. I ask: How does the multimodal nature of video games both enable and inhibit students’ classroom literacy practices as defined through the Common Core, specifically the building of content knowledge through (multimodal) informational texts? My qualitative approach involves the transcription of game-based semiotic data (Kress, 2013; Jewitt, 2009) in the form of “modal ensembles” (Kress, 2013) like screenshots, layout, audio, and actions, as well as Common Core materials used in New York.

Findings
While any single mode employed in the videogame has minimal effects on its own, when multiple modes are coordinated to convey a common curricular goal the potential for learning is deemed greatest. This seems true for visual movement and sonic modes. Additionally, multimodal coordination around discrete curricular goals is challenging to achieve due to the fast-paced nature of the game, the relatively low profitability of educational games as compared to mainstream ones, and the tension between the game’s educative and entertainment narratives.

Significance
My preliminary findings suggest that while the potential for videogames as educational aides is great they must be approached with scrutiny. One way educators and policy makers can scrutinize the educative value of games is to employ multimodal social semiotic techniques to determine the confluence of modes in promoting a clear and valuable educational purpose.

Author

• Tom Liam Lynch, Pace University