Music in Digital Storybooks for Young Children: Aid or Jammer?

• In Event: Sing, Play, Listen: Exploring the Role of Music and Sounds in Vocabulary Learning

Abstract

Electronic books afford new mechanisms unavailable in print books that can offload and strengthen the mediating role of the adult in literacy apprenticeship. These “new wave” books have the capacity to automate shared reading in ways that scaffold story comprehension and thus support future reading success. In addition to camera movements like pan and zoom and details of illustrations in motion, digital storybooks include background music and sounds. For instance, the background music when Bear, after several attempts, fails to confess his love to Butterfly collapses heartbroken may help children understand his emotional state and understand this uncommon expression.
So far, we could not assess the effects of the different storytelling techniques separately in literature reviews because there is a lack of such studies. At this moment, however, as part of an international research project, there are several ongoing studies by different research groups, all represented in the list of authors, testing the effect of background music and sound on learning words from reading digital books.
One might hypothesize that background music can be especially helpful for understanding the emotional events and words of stories. Sound effects, like the birds chattering, might provide nonverbal information that helps in understanding the narration. So far, results are less promising. In a Dutch sample of children with severe language impairments, it was found that background music and sound interfered with children’s word learning (Smeets et al., 2014). To explain this finding the researchers hypothesized that children may have problems with the auditory processing of the narration in the presence of music and sounds due to their developmental disorder. First, it might be that music and sound effects interfere with listening to the narration on a sensory level. Simply put, children might struggle with perception of spoken words in the presence of other auditory stimuli. In this account, music and sound effects, even though coherent with the story on a semantic level, might function as generic noise and prevent children from hearing the words. An alternative explanation is that music and sound interfere with children’s comprehension on another level. It might be that music and sound are processed in the same auditory channel as verbal information and thus make processing of the oral narration more difficult and even cause cognitive overload.
In three new, ongoing randomized controlled trials, researchers compare books that include meaningful music and sound effects to books without music and sound effects. All data are collected and researchers in different countries (Turkey, Hungary and Israel) are in the process of analyzing the data. We are currently estimating the average effect size and, in so far results are heterogeneous, we use moderators such as language proficiency and quality of the oral language in the digital books to explain the variation in outcomes. Our preliminary analyses indicate that sound effects and music may negatively impact word learning in children with low language proficiency. Potential implications of using music and sound supports in ebooks for children with various language abilities will be discussed.

Authors

• Adriana G. Bus, Vrije Universiteit Amsterdam

• Dorit Aram, Tel Aviv University

• Burcu Sarı, Uludag University, Bursa, Turkey

• Zsofia K. Takacs, Eötvös Loránd University