Afterschool Program Practices Tool (APT) Validity and Reliability: Identifying and Minimizing Measurement Error of Youth Program Observation Ratings

• In Event: Roundtable Session 31
  In Roundtable Session: Supporting Valid Decisions at the Individual and Program Levels

Abstract

Based on observations of 25 afterschool programs serving grades K–8 in Massachusetts, this study provides evidence that the Afterschool Program Practices Tool (APT) has strong technical properties, particularly short-term stability and factor structure. Although results for interrater reliability are somewhat mixed, APT scores are related to positive youth experiences and outcomes. These findings suggest that the APT instrument is working well as a measure of key quality constructs and is suitable for a number of lower-stakes purposes, such as self-assessment and program support. APT ratings by a single observer are stable over time, which suggests that individuals can use the tool consistently to capture aspects of quality that are not overly sensitive to day-day fluctuations in practices.

Authors

• Amanda Richer

• Allison Tracy, Wellesley Centers for Women

• Linda Charmaraman, Wellesley Centers for Women