Poster #72 - Validating Self-assessment Measures for Quality of Center-based Childcare: A Meta-analysis

• In Event: Poster Session 08

Abstract

A growing number of stakeholders rely on self-assessment to assess and improve the quality of early childhood education and care (ECEC), but its validity is as yet uncertain. Since there are benefits of self-assessment over assessment by others, a clear view of the validity of self-assessment procedures is needed. Meta-analyses were therefore conducted in the current study to investigate the convergent and predictive validity of self-assessment in ECEC. This systematic review and meta-analyses report findings from 20 publications involving a total of 969 groups and 5,888 children aged 0-72 months in center-based childcare spanning the period between 2010 and 2021. A three-level meta-analysis over seven studies including a total of 32 effect sizes, revealed a positive association (r = .43) between self-assessment ratings and ratings with other validated measures of ECEC quality, indicating a moderate convergent validity. Effect sizes for the individual studies are shown in Figure 1.The association was moderated by study quality, showing that studies which were rated as ‘poor’ reported somewhat higher correlations between self-assessment ratings and ratings with other validated measures of ECEC quality. We assessed the quality of each publication by applying an overall rating of ‘good’, ‘fair’ or ‘poor’ using the ‘Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies’ (National Heart Lung and Blood Institute, 2013). After removal of the studies that were rated as poor, the relationship between self-assessment and other validated measures was still significant with a medium positive effect size ( r = .36). The results of the current review complement earlier research in the United States that reported higher self-assessment ratings for centers with higher scores on validated measures of child care quality, as measured with the Environment Rating Scales (ERSs) (Karoly, 2014). A second three-level meta-analysis over nine studies including a total of 31 effect sizes, focusing on predictive validity, showed a weak significant relationship between self-assessed quality and child outcomes (r = .11). Effect sizes for the individual studies are shown in Figure 2. No moderating effects were found. Our results are in line with a previous meta-analysis also revealing comparably small, associations between the CLASS and children’s outcomes (Perlman et al., 2016). A meta-analysis by Ulferts (2019) regarding longitudinal relations between ECEC quality and academic outcomes using the validated measure CLASS also reported positive but small association. Self-assessment measurements can be a valuable addition to quality ratings conducted with validated measures. Providing professionals with valid self-assessment quality rating may increase further awareness of their strengths and weaknesses regarding ECEC quality. Furthermore, providing ECEC practice with validated self-assessment instruments will facilitate childcare providers to bring their own perspective into the ongoing dialogue about ECEC quality with scientists, parents, policymakers and ECEC authorities.

Author

• Simon Mathieson Hay, Universiteit Leiden
  Presenting Author