The Multimodal Language of Emotions in Early Adolescent Book Trailers as Digital Media Composition

• In Event: How Writing Works: Multidisciplinary Perspectives on the Processes of Composition

Abstract

Objectives
The aim of this research is to articulate a systematic, multimodal language of emotions applied to book trailers produced by early adolescents, attending to images, bodily meanings, written words, music and their interrelationships.

Theoretical Framework
Our multimodal analysis draws on the appraisal framework for spoken and written language by Martin and White (2005). We extend the original theorisation to take account of visual, gestural, linguistic, and bodily modes. Appraisal framework remains one of the foremost systematic approaches for mapping feelings or attitude in applied linguistics (Mills 2016). It refers to the system of language resources used to negotiate emotions, judgments, and valuation, including resources for amplifying and engaging with these valuations (Unsworth, 2013). Our analysis focuses on the area of the appraisal framework that deals with emotions. The axis of the framework of emotions is called affectlanguage resources to convey positive and negative feelingsacross four broad categories of affectun/happiness, dis/satisfaction, in/security, and dis/inclination. There are a range of discrete emotions and dispositions within these categories (Mills, Bellocchi, Patulny, & Dooley 2017).

Methods and Data sources
We offer a multimodal analysis of affect or emotion within a book trailer created by early adolescent girls to promote the popular novel, Rowan of Rin, by Emily Rodda (2005). The movie can be viewed from this browser link: goo.gl/eqZNJk. This multimodal appraisal analysis attends to positive and negative affect through visual elementscolour, visual rhyme, focalisation, social distance); bodily meaningsfacial expression, gaze, gestures, posture, movement); linguistic meanings; and musical elementskey, tempo, timbral features, dynamics (Authors, in press). Semi-structured interviews were also conducted with the students in the same week as the films were created to clarify the students intended affective meanings in the images and sound.

Results
The research highlights the ability of young students to utilise the multimodal affordances of film for communicating emotions and interpersonal meanings in synchronised and orchestrated ways. Affective meanings were evident in the visual design of children’s filmic narratives, including the students’ use of ambiance through the selection of image colour, visual rhyme (e.g. repeated light), focalisation and social distance. Embodied meanings were used to portray affect and judgement experienced by characters in the book, utilising facial expression, gaze, gesture and other bodily movements to develop interpersonal meanings. Written language was used to inscribe or evoke affect and judgement with varied typographic effects. Musical elements, including key, tempo, dynamics, and melodic devices, evoked particular emotions and feelings when combined with visual images. This diverse repertoire of multimodal resources within the interpersonal semantic systems was encompassing and required significant capabilities with a complex ensemble of modes.

Significance
Composing and sharing videos is an everyday literacy practice for children and young people in the 21st century. Children today need apprenticeship into the multimodal communication of emotions through film and media design to gain sophisticated and developed understanding of the interpersonal meaning systems that are necessary to enact social relationships in digitally dominated worlds.

Authors

• Kathy Ann Mills, Australian Catholic University

• Len Unsworth, Australian Catholic University

• Georgina McFarlane Barton, Griffith University