Centering Children's Voices and Purposes in Social Semiotics Research

• In Event: Theorizing Multimodality Through Children's and Youths' Perceptions and Experiences

Abstract

Objectives
There is a disconnect between how multimodality is theorized in academic scholarship and how children make sense of multimodal texts and composing contexts. This premise invites us to examine what digital multimodal composing looks like from the points of view of children, and the sorts of theorizing children do as they make sense of the properties and purposes of the semiotic modes they design with.

Theoretical Perspectives
While multimodality is well-established in literacy scholarship, children’s perspectives about multimodal design are considerably less so. This paper draws from the intersection of critical childhoods (Barker & Weller, 2003) and the interwoven branches of multimodal literacies. Multimodalities research roughly follows three parallel tracks: NLS, multiliteracies, and studies with roots in SFL (Jewitt, 2008). Much SFL-informed scholarship attempts to identify communicative affordances/constraints, modal densities, and functional loads, whereas social semiotics scholarship prioritizes the ways modes are socially shaped by users. Neither approach fully captured the ways children talked about multimodal design in our study, as our findings indicate.
Failure to attend to children’s perspectives is an act of epistemic injustice (Fricker, 2009), that deprives them of the right to theorize within spaces and discourses regulated by adults. For children to influence the worlds they traverse, their voices must transcend the cultural boundaries of childhood and negotiate a shared understanding in the adult world (Ghiso, 2015; Murris, 2013; Vasquez, 2005; Yoon & Templeton, 2019). We employ a transliteracies framework (Stornaiuolo, Smith, & Phillips, 2017) to shuttle among theoretical traditions in order to better honor the stated compositional purposes of children.

Methods/Data Sources
We analyze interviews (N=75) with children ages 8-10 about their digital videos as artifacts, and about their multimodal videomaking processes and design choices. The interviews were conducted as part of a design-based, qualitatively oriented four-year study in a dual immersion Spanish-English charter school where children made videos on iPads in their language arts and social studies classes.

Findings
We identified a range of children’s context-specific comments about multimodality, from the communicative and aesthetic effects of modes to children’s reasons for employing or not employing particular design elements in their videos. Across our dataset, children articulated seven purposes for choosing modes in their digital videos. As part of our analysis, we explored how each purpose co-occurs with the use of visual imagery, aural effects, spoken language, and written text. Our goal is not to propose new grammars for analyzing children’s multimodal texts, but to learn from children their reasons for composing as they do, which in many cases do not conform to what formal semiotic grammars can functionally capture.

Significance
How might child-led theorizing align with or complicate academic semiotic theories? And how might children’s perspectives shape the ways educators incorporate digital multimodal composing into our pedagogies? As literacy researchers who train future teachers to work with children on multimodal literacy projects, these questions are of deep significance to us. Examining how children understand semiosis would allow us to ground theory-building in the lived realities and authorial purposes of the young people to whom digital multimodal composing is emergent.

Authors

• David Eric Low, California State University - Fresno

• Jessica Zacher Pandya, California State University, Dominguez Hills