Evaluating the ability of AI to improve Kenyan Sign Language language acquisition assessment

• In Event: Roundtable Session 6
  In Refereed Roundtable Session: Inclusive Education for Belonging: Equity, Identity, and Innovation

Proposal

Our team is evaluating the efficacy and cost effectiveness of an AI-enhanced Kenyan Sign Language (KSL) assessment tool to support foundational language learning. In our proposed Round Table discussion, we will compare initial results from the AI-enhanced assessment vs. a data set of 1,000+ assessments performed with our current, non AI-enhanced tool in Kenyan schools.

Questions for discussion include:
Can AI-enhanced sign language assessment lower barriers to equitable assessment?
What are the risks of AI enhanced assessments? How or can they be mitigated?
How can the development and deployment of AI-enhanced sign language assessment tools be made more inclusive of deaf communities and educators?
What are the cost implications of developing and implementing AI-enhanced sign language assessment tools compared to traditional methods?
What lessons can be shared from similar attempts globally?

Our current, non AI-enhanced tool supports foundational language acquisition for deaf learners in their first language: sign language. It assesses both receptive and expressive skills. In the receptive section, the assessor signs, then the learner selects the appropriate picture. In the expressive section, the learner signs words, sentences, and stories prompted by flashcards. An assessor fluent in KSL is required to score the responses. Progress is recorded against 4 levels, corresponding to competencies expected in Pre-primary 1, Pre-primary 2, Grade 1, and Grade 2—years that cover most Kenyan deaf learners' first exposure to sign language.

The tool is used by teachers in schools and has both formative and summative versions, as well as baseline and endline assessments to support grade progression and research. Since 2023, teachers have used the tool to assess 835 deaf learners in 1,740 sessions across 10 Kenyan public schools. In 2025, the program is scaling to 30 schools, and pilot projects will be launched in Rwanda and Malawi, where we have staff fluent in local sign languages. This scaling, along with the supporting research, is funded by iGravity, GPE Knowledge Innovation Exchange (KIX), and eKitabu. To our knowledge, this is the only tool of its kind currently used in African schools.

In our first year in Kenya, 78% of deaf learners improved by 1-2 grade levels, and enrollment in treatment schools increased by 220%.

We are evaluating the AI-enhanced assessment tool’s ability to reduce the time required for administration and improve ease of use for teachers. Deaf teachers, particularly those fluent in sign language, are rare even in schools for the deaf. Teachers have indicated that adding video prompts by fluent deaf signers from our team, and developing an app for data collection, would make the tool more efficient and accurate. Currently, we use paper booklets alongside Google Forms.

In June 2025, with support from the Tools Competition, we began enhancing our Kenyan Sign Language Assessment Tool by: 1) digitizing it in an app for more efficient data collection; and 2) testing AI’s ability to transcribe signed responses in the assessment’s expressive section.

Authors

• Matthew Utterback

• GEORGINE AUMA OBURA, eKitabu