Implementation Fidelity in a Multiyear Professional Development Intervention for Preschool to Third-Grade Teachers: Decisions, Decisions, Decisions

• In Event: Roundtable Session 25
  In Roundtable Session: Using Fidelity Data to Improve Initiatives for Early Childhood Teachers

Abstract

This presentation describes our measurement of implementation fidelity for a four year PD intervention delivered to 8 schools in a large urban area. The intervention is implemented within the context of preschool to third grade school-wide reform efforts aimed at improving academic achievement outcomes and teacher quality across the grade continuum. Implementation involved four components: learning labs, individual coaching, grade-level meetings, and leadership academies.
Fidelity of Implementation (FOI) is defined here as the extent to which key components of the intervention are delivered as originally intended by the developer. Implementation research suggests multiple steps to fidelity measurement (Cordray & Hulleman, 2009). First, we identified the key implementation components which in this case were activities and inputs that were under the direct control of the developer responsible for program implementation and secondly, we identified an empirically- or theoretically-driven threshold for each component that defines adequate or high fidelity. These two steps were then repeated for each year of the four year intervention. All decisions were made in collaboration with the developer. Table 1 shows the results of these discussions and decisions across the 3 key components for teachers and 4 key components for schools.

Fidelity of implementation was measured at both the teacher- and school-level. Data from attendance rosters and coaching logs were used to document the dose (frequency and length) of teachers’ participation in each of the 3 key intervention components (learning labs, individualized coaching, grade-level meetings). For example, for teachers to have met the fidelity criterion for learning labs, they had to attend 15 or more hours of learning labs. In one school, 70% of teachers met the criterion and at another school, 88% met the criterion. At the school-level, both schools met the fidelity criterion with 70% or more of teachers attending learning labs for 15 or more hours. We will report fidelity data for years 1 and 2 of the 4-year intervention for each component at the teacher- and school-level during the presentation.

Measuring fidelity is important for explaining the mechanisms of any impacts on schools, teachers, or students and can also inform replication and scale-up of the model intervention (Clements & Sarama, 2011). There are challenges inherent in developing reliable and valid assessments of FOI. As developers implement the intervention in different settings and learn what works best in which settings, the implementation components are refined and improved. Often, a fidelity criterion may need to change as more is learned. Other challenges are inherent in a multiyear intervention where supports and key components may necessarily change as teachers’ attitude, knowledge, and practices change. Finally, there are other aspects of fidelity that are not captured by dosage measures and how to capture those in this dynamic system can be a challenge.

Authors

• Erika Gaylor, SRI International

• Ximena Dominguez, SRI International

• Donna Spiker, SRI International