Poster #46 - Concordance between Direct Assessment and Adult Report in Child Assessment: A Systematic Review and Meta-analysis

• In Event: Poster Session 12

Abstract

The significance of the early years for human and national development has been recognized, and numerous interventions have been implemented to promote early childhood development (ECD). Valid measures of ECD are necessary for scaling up programs and best practices. However, there is no consensus as to the most appropriate method of measuring ECD at the population level. On the one hand, adult report (AR) is quick and cost-effective (Johnson & Marlow, 2006), and may be the most appropriate choice in large-scale assessments of ECD. However, AR can be susceptible to bias (Sachse & von Suchodoletz, 2008). Furthermore, there are issues with the comparability of data collected by AR in multi-country studies (Raikes, 2017; Rao et al., 2020). Direct assessment (DA), on the other hand, is considered to be a “gold standard” in early childhood assessment (Miller et al., 2017), and is widely used to validate AR.

Previous studies that compared DA and AR in early childhood assessment have been typically conducted in only one context. Few of them examined how the concordance between the two methods differs across variables. Furthermore, studies have rarely explored correlations between DA and parent report (PR), as well as teacher report (TR) in measuring ECD. Against this background, this study aimed to determine the general concordance between DA and AR in measuring young children’s learning and development, and to consider the potential moderators (e.g., age, domains, respondent, country income levels) of the relationships between them.

Relevant keywords (two-to five-years, learning and developmental competencies, direct assessment, adult report) were used to search for evidence in six electronic databases. Grey literature was also identified. Studies published after 2000 were considered. Following a rigorous screening procedure, 48 studies conducted in 21 countries or regions were included in the systematic review. Study characteristics (e.g., publication year, study quality, sample size etc.) were coded. Findings from 43 of these 48 studies were statistically combined using meta-analysis.

Meta-analysis showed a moderate effect size of 0.48 regarding the correlation between DA and AR. The effect size was 0.50 for relations between DA and PR and 0.43 for relations between DA and TR, but these effect sizes did not differ significantly from each other. Moderator analysis of child age showed AR may be more accurate when evaluating younger than older children. Furthermore, correlations between AR and DR were highest for Language, Literacy, and Math domains compared to other domains such as Cognition and Self-regulation. Analyses of the interactions of effects found higher concordance between the two methods when assessing younger children’s Language and Math, and older children’s Literacy. Moreover, teachers were more accurate when evaluating Executive Functions, Literacy, and Math of children, while parents were more reliable in reporting children’s language development. Finally, neither study quality nor country income level moderated the concordance between DA and AR. Implications of findings are discussed.

Authors

• Zeyi Li, University of Hong Kong (HKU)
  Presenting Author

• Nirmala Rao, The University of Hong Kong (HKU)

• Jing Gong, University of Hong Kong (HKU)