Validity Study of the Child Functioning Module-Teacher Version in Nepal

• In Event: Highlighted Session: Evaluating a tool for school-based identification of children with functional difficulties: the Child Functioning Module – Teacher Version

Proposal

Background: The CFM-TV Validity Study contributes to global evidence on the usefulness of the CFM-TV for providing data on learners with disabilities in school settings in three ways:
• It contributes to an understanding of whether, in which circumstances, with what types of teachers, and for which domains of functioning the CFM-TV can provide adequate information about a learner’s functional difficulties in Nepal for disaggregation.
• It advances the overall body of evidence related to identifying and disaggregating early grade reading outcomes of children with disabilities in schools in Nepal.
• It provides insights on how to conduct similar validation efforts in contexts that share similar goals.
Methods: The CFM-TV Validity Study used a non-experimental cross-sectional mixed-methods approach combining descriptive research with elements of diagnostic accuracy studies to understand factors that influencing teachers’ assessment of learners’ functional difficulties. The study also examines the consistency of the CFM-TV tool with the WG Child Functioning Module (CFM) and medical screenings. The study collected data from teachers, primary caregivers, and medical screenings using both quantitative and qualitative tools at two time points during the Nepali school years 2022-2023 and 2023-2024. The sample included four school types (mainstream, mainstream with resource classes, special schools, and madrasas) from four provinces in Nepal to understand how the tool functioned in different settings.
Analysts calculated prevalence rates according to CFM-TV responses and explored their relationships with other variables, first through chi-square tests to look for general associations, then through multi-level logistic regression models. A similar approach was implemented to understand factors influencing consistency between teachers’ and PCGs’ responses.
To understand consistency between the CFM-TV and medical screenings, analysts compared results from the CFM-TV and medical screenings for a sample of learners who were assessed separately by both medical professionals and teachers. Inter-rater reliability of CFM-TV responses compared with primary caregiver or medical results was calculated using Cohen’s kappa.
Results: The study collected a total of 2,222 CFM-TV records from 58 schools and 157 teachers. A subset of CFM-TV records for learners (629) were paired with responses from primary caregivers who used the CFM, and a subset (408) were paired with results from medical screenings in vision, hearing, and/or mobility. Overall, findings indicate that teachers’ and primary caregivers’ responses showed sufficient to moderate reliability for the purpose of disaggregating population-level data by overall functional difficulty. However, the tool is less reliable in the cognitive and psycho-social domains when measured against the CFM. Data from medical assessments further indicated that there might be some doubt to teachers’ ability to provide reliable identification of impairment (rather than functional difficulty) in hearing and vision.
Implications: Results from the study indicate that, in Nepal, the CFM-TV may be a valid tool for providing estimates of functional difficulty prevalence. Validity is promising if estimating prevalence only in the functional difficulty domains of seeing, hearing, and walking. Findings from this study indicate that the CFM-TV is an inappropriate tool for individual pre-screening for medical diagnoses.

Authors

• Anne Laesecke, Zemitek

• Aimee Reeves, School-To-School International

• Eileen Dombrowski, School-to-School International (STS)