Implementation Fidelity in Pre-Kindergarten and Kindergarten

• In Event: 35.074 - Midwest Child–Parent Centers, Pre-Kindergarten to Third Grade: Impacts of a School Reform Model

Abstract

Objective
Since fidelity of implementation is fundamental to the effectiveness of the MCPC, we evaluated the extent to which the sites implemented the program requirements in Years 1 and 2. The sources of data included site visits, document reviews, observations, and staff interviews.

Conceptual Perspectives
The fidelity study is structured around the six MCPC elements. The strategies and measurements include: 1) effective learning experiences from PreK-3rd via enrollment forms; 2) aligned curriculum via administrative data on classroom practices; 3) comprehensive family services via family engagement plans and parent involvement logs; 4) collaborative leadership team efforts via staff enrollment forms and meeting notes; 5) continuity of student experiences via site location and teacher collaboration across grades; and 6) ongoing professional development to support teachers and principals in implementation via participation data and teacher surveys. Implementation fidelity is measured in the fall and the spring.

Methods and Data Sources
Implementation began in fall 2012. The 26 sites varied in funding agency, co-location, and size. Implementation fidelity is evaluated using a multi-method approach that includes observations, structured interviews, surveys, and administrative data. Data collection began in fall 2012 and will continue through the end of the 3rd grade year. In Year 1, the implementation team rated each school based on the six key elements using a five point scale. In Year 2, to increase transparency and robustness, a rubric was developed based on the thirty program requirements with a five-point scale for each item.

Results
Average ratings of implementation fidelity for Year 1 (preschool) across the six elements was 3.9. This is a moderately high score (min. score = 1, max. score = 5). Across all districts, the highest rated element was Effective Learning Experiences (4.0). The lowest rated element was Parent Involvement (3.5). When averaging across the six elements, 96 percent of sites met the moderate-to-high fidelity standard defined as an average rating of greater than 3.

The rubric used to assess fidelity in Year 2 (kindergarten) consisted of multiple measures for each key element. Ratings from fall 2013 indicate that across all project sites, the overall fidelity rating increased. The overall average across the six key elements was 4.0, with 97 percent of sites rated 3 or higher. The highest rated element was parent involvement (4.3) with 97 percent of sites rated 3 or higher on this element. The lowest rated element was continuity and stability, with only 67 percent of sites rated 3 or higher on this element. In further analysis, fidelity ratings will be correlated with child outcomes and parent involvement. Linear and threshold metrics will be established that best fit the data.

Significance
Implementation fidelity of MCPC provides essential information about the quality of the program in expansion and scale-up efforts. Findings indicate the model is being implemented well. Fidelity also informs the generalizability of the CPC model across districts. The links between fidelity ratings of each of the program elements and school achievement will contribute to program improvement and tailoring of services to optimize effects.

Authors

• Mallory Warner-Richter, University of Minnesota - Twin Cities

• Erin Lease, University of Minnesota - Twin Cities

• Molly Sullivan, University of Minnesota - Twin Cities

• Michelle M. Englund, University of Minnesota - Twin Cities