Evaluation of the District of Columbia Public Schools Literacy Initiative

• In Event: Urban K-3 Literacy Innovation? Implementation and Effectiveness of Three Districts’ New Approaches

Abstract

This presentation will focus on the first-year independent evaluation of the early reading initiative implemented in grades K – 3 in six schools in District of Columbia Public Schools (DCPS) in Washington, D.C. The District’s proposal for the initiative includes five components as means to build teacher capacity and raise student achievement:

1) Literacy consultants, serving as coaches to support teachers in planning, implementing instructional practices and integrating technology
2) Maximized time and effective use of the entire 120-minute literacy block
3) Professional development opportunities at the school, as well as in district and national workshops.
4) Educational technology resources, including computers, iPod touches, iPads, and literacy station components.
5) Literacy resources, including leveled readers and digital content

Data for the presentation derive from the following data sources at three time points across the year.

Exhibit 1. Breakdown of 2012-2013 Data Collection


Principal: Fall (6 interviews), Winter (6 interviews), Spring (6 interviews)
Project Managers: Fall (2 interviews)
Literacy Consultants: Fall (6 interviews), Winter (2 focus groups), Spring (6 interviews)
Classroom observations: Fall (38 classrooms), Spring (36 classrooms)
Teachers: Fall (56 surveys), Winter (10 focus groups), Spring (55 surveys)
Teacher technology use logs: Winter (192 hours)
Coach logs: Fall (1625 hours), Winter (2866 hours), Spring (2040 hours)
DIBELS achievement data: Fall, Winter, Spring (all K-3 students)
DC Comprehensive Assessment System achievement data: Fall, Winter, Spring (all K-3 students)

Interview, focus group and survey data reflect stakeholders’ perceptions about the initiative and the extent to which they are knowledgeable about its components at the beginning of the year. These data collections in the winter and spring also captured changes in perceptions of the initiative and in implementation of the initiative itself throughout the year. Frequency data from logs will be used to describe the dosage and focus of the coach and technology activities. Results from the classroom observation data will be used to describe changes in instruction. We will also discuss achievement results after the first year, using the DIBELS achievement data and DC CAS data for grades 2 and 3. The DC CAS data for study sites were compared to those from a matched sample of other DCPS schools using an interrupted time series design . Year 1 findings are unavailable as of July 2013 but will be included in the presentation.

Preliminary implementation findings point to several factors as important for success: (1) principals were decision makers in the hiring of literacy consultants and use of funds for technology and resources. The project was managed by an external firm rather than the district office, thereby increasing the layers of communication between the various stakeholders; (2) early and effective communication is necessary to ensure stakeholder buy-in; and (3) integration of the literacy consultants into the management framework of the schools, including as key members of the school leadership team. These and other lessons learned could potentially expand upon the literature on best practices for the successful implementation of school-based literacy initiatives in urban districts.

Authors

• Zodie Makonnen, American Institutes for Research

• Michele Toplitz, American Institutes for Research

• Jonathan Farber, American Institutes for Research

• Myra Thomas, American Institutes for Research

• Julia Parkinson, American Institutes for Research