Establishing Research/Analysis Procedures to Address Student Application and Learning Within a Multimodal Writing Space

• In Event: Extending and Evaluating a Theoretically Based, Online Intervention to Improve Latino Middle-Schoolers’ Writing

Abstract

The aim of this poster is to describe the data collection and analysis procedures used to determine the efficacy of the Web-based writing environment and the embedded instructional approach to support emergent bilinguals’ academic writing.
Our method is based on a concurrent process of web-based system testing and observation of student use and interaction during a series of instructional sessions led by a STEPS project instructor. The overarching question addressed is “to what extent can a proven instructional approach linked to web-based instructional system support emergent bilinguals in developing their academic writing in English?” Specifically, we ask
• How effective is the developing interface as judged by student reports and system tracking in terms of ease of use and support for their writing?
• To what extent do emergent bilinguals understand and use the instructional anchor and the STEPS+G heuristic to support analysis, note-taking and essay writing in content areas?
• How do students make use of the home language to scaffold their work in English?
In Year One, 10 Latino student-consultants and a classroom cohort of 13 from a Bronx, NY middle school tested the system’s alpha version during instructional sessions on a Social Studies topic. During Year Two, 9 Latino student-consultants from a Harlem middle-school and a classroom cohort of 21 students from the Bronx school tested the beta version of the software. In both years, the consultant groups were recruited first in order to provide feedback about the STEPS writing space interface and the curriculum of the sessions in order to incorporate feedback and improve the STEPS system for the classroom cohorts.
Following the same protocol in Year One and Two, data were collected from multiple sources throughout the instructional sessions. To measure effectiveness of the developing interface, the STEPS team relied on student feedback and data tracking methods regarding the system’s ease of use, design, and presentation. Student feedback was collected through interviews, surveys, and recorded talk during instructional sessions. User keystrokes and mouse clicks were tracked, stored in the STEPS Content Management System, and examined for patterns of navigation around the interface and use of the video anchor, STEPS elements, and selection of multimodal resources.
All instructional sessions were audio- and video-recorded for student use of the social and material supports--home language scaffolding, the instructional anchor and the STEPS+G heuristic. Additionally, during each session, field notes were taken of students’ actions as they used the STEPS system. Recorded data were transcribed and analyzed using ELAN, a professional tool for the creation of complex annotations of digital-recordings, to find patterns of student interactions with the online interface and with each other. Additionally, a back-end database was developed to record students’ notes, essays and messages; these were examined for patterns of language use, references to anchor video material, and use of STEPS+G elements for multimodal document analysis and synthesis.
This poster will provide a detailed timeline of the intervention, depicting the above data collection methods and data analysis procedures, including sample codes and ELAN screen shots.

Authors

• Karen Velasquez, Teachers College, Columbia University

• Briana Ronan, Teachers College, Columbia University