A randomized controlled trial to compare two dissemination models for innovative ideas in policy development

• In Event: Literacy programs in Latin America: analyzing early grade reading practices both in and out of school

Proposal

Since 2005, J&A, through various USAID-funded projects, has supported the MoE of Guatemala in the implementation of an accountability system, based on large scale tests in reading and math. As a result, reading ability has been tested nationally in benchmark grades (first, third, sixth, ninth and end of high school), and results have been compared yearly ever since. This is known in the literature as achievement or status. However, recent research found that such reporting model does not show the possibility that a school could be improving its results, even though its students don’t achieve the standard due to variables outside the school’s control (Betebenner, 2009). Consequently, since 2014, USAID Lifelong Learning has supported the MoE in measuring reading ability of the same cohort of students through consecutive grade levels. This is known in the literature as growth modeling. Shifting from the idea that learning to read is a one-grade shot to the idea that learning to read is a series of skills that grow in complexity during various grade levels is an innovative idea in Guatemala.
To communicate this novel concept of growth in Guatemala in reading ability to various stakeholders, in 2015 the project published a four-page policy brief entitled: "Learning and growth models". Such policy brief was delivered through the project’s website and social media. It was also delivered in its printed version in specific meetings with key stakeholders. In 2016, growth results obtained by the project in its focus areas (12 municipalities of the western highlands of Guatemala) were disseminated through face-to-face meetings with teachers and school directors. However, the audience response to such results and dissemination formats was not positive. Teachers and school principals demonstrated lack of understanding of growth results and learning to read as a continue of skills that grow in complexity.
Due to the lack of understanding and rejection of school principals and teachers, as well as technical staff at educational departmental level, the project decided at the end of 2017 to create a new tool for disseminating the growth modeling results. Now the project decides to communicate this new idea through an audiovisual which offers rich elements of communication.
Dissemination of innovations has been approached from different theoretical approaches including those by Gabriel Tarde (1890), Friedrich Ratzel and Leo Frobenius (1926), Pemberton, H. E. (1936), and Rogers (1962).
Rogers defined four elements in the process of diffusion of innovations: 1) Innovation
2) Communication Channels 3) Time 4) A Social System. (Rogers, 1995). Also using previously definitions of Lazarsfeld and Merton, Rogers focused on the terms of homophily and heterophily. He defines homophily as: "the degree to which pairs of individuals who interact are similar in certain attributes, such as beliefs, education, social status, and the like" (Rogers, 1995, p.19). But indicates degrees of heterophily are needed to spread new ideas, because if two individuals are identical on the technical aspects of an innovation, it is hard that diffusion occurs because there is no new information to share (Rogers, 1995, p.19).
For the video, as a new media to communicate the “growth modeling”, the project prepared a script using an analogy between the growth in reading ability and the growth of a corn plant. The harvest of corn in Mesoamerica is basic to the region’s diet, but it is also considered sacred in the Mayan culture. Thus, it was assumed that it would connect to the audience’s culture directly. The script with a with a simple and clear language (legibility) was turned into a video that featured rich communication elements (two voices, motion graphics, color and music). The images and music were produced especially for this project with cultural and creative (talent) characteristics known for the audience to create a "homophily" content.
To guarantee effective communication (Richaudeu, 1984) mentions that 4 elements are essential in a media tool: 1) originality 2) the wish, understood this as the recipient's desire to take the information from the sender 3) legibility and 4) talent or quality of the sender. It was hypothesized that such communication channel, with these characteristics in the way of treating the oral contents, images and music, will facilitate understanding of the growth measuring model. The main communication objective of the video was informative, and it was pursued that the messages communicated the multidimensionality of reading learning process for the initial grades. In this sense, the term "growth modeling" is not mentioned in the video, but rather the multiple elements that it measures over time. Nonetheless, the project decided to implement a workshop that walks audience through the important messages in the video. It was presupposed that such workshop will increase understanding of the message in the video.
The purpose of this study was to evaluate the understanding of the concept of growth of a skill as reading when the message has been presented through a video and when the key messages of the video have been emphasized through a face-to-face workshop. To evaluate the efficiency of the video to facilitate understanding of the innovative idea, a randomized controlled trial was implemented. Three groups were compared as follows:
A = Video  Evaluation  Workshop
B = Video  Workshop  Evaluation
C = Video  Evaluation  Workshop  Evaluation
The research hypothesis was the following:
Video+Evaluation The study was conducted in Totonicapán. Totonicapán is in the Western Highlands of Guatemala and for the most part its population is bilingual (k’iche’- Spanish). The targeted audience of the video were school teachers and school principals. School teachers were invited by the Ministry of Education to a workshop in which the MoE delivered national reading results to the audience as part of their regular dissemination activities. USAID Lifelong Learning project launched the video and conducted the randomized controlled trial during this meeting. A total of 500 teachers were invited to the meeting; however, only 495 participated.
We have presumed that communication of novel ideas is straight forward, but results of this research show that complex ideas imply a in depth communication strategy.

Authors

• Antonio Arreaga, Juarez & Associates

• Leslie Vanessa Rosales de Veliz, Juarez and Associates