Findings from feasibility test of Child Functioning Module – Teacher Version in Sierra Leone

• In Event: Highlighted Session: Evaluating a tool for school-based identification of children with functional difficulties: the Child Functioning Module – Teacher Version

Proposal

Background: A disability-inclusive Education Management Information System (EMIS) is a crucial tool in ensuring that children with disabilities are made visible within educational systems, and that schools and educational authorities can appropriately plan and secure resources to meet children’s educational needs. However, the absence of an internationally-recognised disability data collection tool validated for use by teachers in schools poses a challenge for the routine collection and reporting of school-level disability data. This mixed-methods, multi-stage study assesses the feasibility and acceptability of teachers using the Child Functioning Module Teacher Version (CFM-TV) and the Washington Group Short Set (WG-SS) to generate disability data in schools in Bombali and Karene Districts of Sierra Leone.
Methods: Both tools were initially piloted with teachers from two schools (WG-SS: n=6; CFM-TV: n=12). Teachers each assessed 30 learners from their registers, with each child assessed by two different teachers; prevalence of functional difficulty (FD) using standard cutoffs and inter-rater agreement were calculated. In depth interviews and focus group discussions explored assessment experiences and item interpretation. Subsequently, 30 class teachers from 4 schools assessed all children on their registers using WG-SS (n=1476), while 35 teachers from 4 other schools used the CFM-TV (n=1830). Prevalence and distribution of FD was explored for each tool. Feedback from teachers and schools was collected through interviews and focus group discussions.
Results: During the pilot, prevalence of FD was much lower among children assessed using WG-SS (2%) than CFM-TV (18.3%). Across both tools, inter-rater agreement was similar and high (>98%) for the domains of seeing, hearing and walking. Overall inter-rater agreement on FD status was 98% for WG-SS and 85% for CFM-TV. Teachers struggled most with the communication and cognition items in WG-SS, and with anxiety, depression, remembering, controlling behaviour and accepting change in the CFM-TV. Across both tools teachers initially struggled with aspects of the rating scale; the length of the CFM-TV was a concern. School-based data collection generated FD prevalence of 2% using WG-SS and 15% using CFM-TV. For both tools, prevalence did not vary for girls and boys, but was higher among older children. Prevalence of FD was below 1% for all WG-SS domains. CFM-TV domains with highest prevalence were anxiety (5%), depression (4%) and remembering, accepting change and learning (all 3%). FD prevalence estimates ranged notably at school (WG-SS 0-4%; CFM-TV: 3-32%) and teacher (WG-SS: 0-25%; CFM-TV: 0-85%) levels. Data completeness across both tools was good.
Implications: Both tools appeared to broadly acceptable, and feasible in the context of a study with incentives provided to teachers. While CFM-TV generates information for additional domains relative to the WG-SS, many of these domains were more challenging for teachers to assess and generated higher levels of disagreement between assessors. High levels of variability in FD prevalence across schools and teachers require further exploration. Clear mechanisms for data use by teachers and schools, or for aggregation and integration into EMIS, were not immediately identifiable, and would require contextually-appropriate development. Additional research into teacher training requirements and frequency of assessment is also advised.

Authors

• Julia de Kadt, Sightsavers

• Steven Kaindaneh, Sightsavers

• Elena Schmidt, Sightsavers