Child Functioning Module – Teacher Version: Pilot test in Somalia

• In Event: Highlighted Session: Evaluating a tool for school-based identification of children with functional difficulties: the Child Functioning Module – Teacher Version

Proposal

Background: An International Non-Governmental Organisation implemented a five-year (2019-2023) development project that had education, child protection and child rights components in Somalia. The project was funded by Norwegian Agency for Development Cooperation. Data for the pilot test was collected in February 2022 from schools supported by the project.
Purpose: The pilot test was conducted to assess reliability of Child Functioning Module-Teacher Version (CFM-TV) using inter-rater reliability analysis, to assess feasibility of administering CFM-TV set of questions, and to document lessons learned.
Method: A total of 328 (61% boys, 39% girls) primary school learners were sampled from four schools in Garowe district. 11 classes of learners from grades 3, 5 and 7 were sampled. Their average age was 12 years. Each learner in a specific class was rated on their disability status by two different teachers. The 22 teachers (36% female) who participated in the pilot test taught 4.4 days per week in the class and used two months to observe and understand the functional difficulty each learner may have in any of the 12 health-related functional difficulty domains included in the CFM-TV set of questions. A descriptive percentage agreement and Cohen’s Kappa analyses were conducted to examine inter-rater reliability.
Findings: The study found that both Teacher 1 and Teacher 2 showed agreement in 89-96% (N=328) of the learners regarding whether or not a learner had a disability in the same functional difficulty domain. While the highest level of agreement (96%) was observed when rating learners on “controlling behaviour” domain, the lowest agreement (89%) was documented on “anxiety”. Both teachers agreed in 5% of the learners (N=328) as having disability (“a lot of difficulty” or “cannot do at all” level of difficulty) in the same functional domain. In terms of functional-domain-specific Cohen’s Kappa results, there was a slight agreement (Kappa value 0.0-0.20) in most of the functional domains, while there was a fair agreement in “remembering” and “controlling behaviour” functions. In rating a learner as a “learner with disability” or “learner without disability” regardless of agreement in type of disability, there was indication that teacher’s gender was related to proportion of learners identified as children with disability. When both teacher raters were male, they agreed in 80% of their ratings, compared to agreement only in 69% of ratings when both raters were female. When both teachers were male, they were likely to agree on smaller proportion (8%) of learners rated as a child with disability, compared to proportion (18%) agreed when both raters were female. Furthermore, teachers of different gender were likely to agree on smaller proportion (6%, Kappa value = 0.14) of learners rated as a child with disability, compared to proportion (10%, Kappa value = 0.31) when both teachers were of the same gender.
Recommendations: Teachers recommended to increase the observation and interaction duration to help teachers improve their familiarity with health-related functional difficulties of their learners. Practical training for teachers on determining the type and level of functional difficulty was recommended.

Author

• Henok Tesfay Zeratsion, Save the Children Norway