Bridging Digital Divides: Enhancing Educational Assessment in Resource-Constrained Classrooms

• In Event: Stakeholders, technology and inclusive education

Proposal

Introduction
In typical Kenyan classrooms, assessment methods primarily rely on classical test theory (CTT), which emphasizes overall test scores and often fails to capture the nuanced competencies of students. This conventional approach tends to focus on content coverage and numerical rankings, yielding grades that do not necessarily reflect a student’s true proficiency. With the introduction of the Competency-Based Curriculum (CBC) in Kenya, there is an increasing demand for assessment strategies that prioritize student competencies over mere content knowledge. To address these limitations and enhance educational assessment, there is growing interest in implementing more sophisticated psychometric approaches, particularly Item Response Theory (IRT).
IRT is a modern psychometric approach used for designing, analyzing, and scoring tests and assessments. Of particular interest is the 3-Parameter Logistic Model (3PL), which evaluates three parameters item difficulty (how challenging each question is), item discrimination (the effectiveness of an item in distinguishing between different levels of student ability), and a guessing parameter (the likelihood of a student guessing the correct answer). This method produces ability estimates representing the latent trait level for each respondent, which in educational testing would be the student's ability level.
While psychometric tests like IRT offer significant advantages in creating more reliable and valid assessments, its implementation in traditional classrooms faces challenges. The method requires advanced statistical and programming skills, and its outputs, such as Item Characteristic Curves (ICCs), Test Information Functions, Fit Statistics, and standard errors, can be difficult for teachers to interpret without extensive training.
Recognizing these challenges, the Assessing Student Competencies (ASC) program has created a user-friendly web application to help teachers apply IRT testing in their classroom assessments. This tool simplifies complex outputs into easy-to-understand statistics, making advanced psychometric methods accessible for everyday teaching.
However, the accuracy and reliability of item parameter estimation crucially depend on sample size. Research indicates that accurate item-parameter estimates require at least 1,000 learners, with shorter tests needing a minimum of 500. Even for tests with 10 to 30 items, a sample of 150-750 learners is necessary for reliable results. These numbers far exceed typical classroom sizes in most African contexts, potentially compromising the precision and reliability of IRT-based applications in these contexts.
Objectives
The specific objectives of this study are:
1. Enhance in-class decision-making via rapid processing of learning microdata with IRT analysis.
2. Help teachers develop competence-based assessments, particularly around new curricula.
3. Develop, test, and streamline statistical simulation techniques for reliable IRT analysis in African classrooms with small sample sizes.
4. Demonstrate the usability and utility of the ASC web application in resource-constrained educational environments.
Analytical Strategies
To tackle the sample size limitations inherent in IRT analysis, we employ a mixed-methods approach. This includes:
1. Establishing a theoretical foundation of IRT and reviewing literature on the impact of sample size on educational assessment characteristics.
2. Analyzing data from typical African classrooms to ascertain the extent of sample size challenges.
3. Developing and testing a simulation model using Bootstrapping and Markov Chain Monte Carlo (MCMC) techniques combined with Bayesian modeling to create synthetic datasets that reflect the statistical properties of original small sample data.
4. Conducting comparative analyses to validate our approach by comparing IRT results from simulated datasets with those derived from larger, real-world datasets.
5. Engaging in qualitative research through interviews and focus groups with teachers and educational administrators to identify challenges in implementing digital assessment tools like the ASC application.
6. Iteratively refining the ASC web application based on insights gained to enhance its effectiveness and user-friendliness in small sample contexts.


Results and Implications
Preliminary results from the simulation methods show promise in producing valid synthetic data and enhancing IRT analyses for limited sample sizes. Four different resampling models were created, with varying levels of accuracy:
• Bootstrapping: 100% accuracy
• Gibbs sampling: 99.33% accuracy
• Jackknife method: 47.63% accuracy
• MCMC-Metropolis Hastings variation: 98.67% accuracy
While the high accuracy percentages are encouraging, they may also indicate potential overfitting. Efforts are underway to address this issue, after which the best model will be selected for implementation in the ASC platform to achieve high, reliable, and representative samples from the original data.
Key implications of this research include:
• Improved assessment: Enabling competent student evaluation with typical classroom assignments.
• Support for educational reform: Facilitating the implementation of competence-based curricula.
• Bridging digital divides: Making powerful assessment tools accessible in resource-constrained settings.
Relevance to the CIES theme
This study aligns with the CIES 2025 theme "Envisioning Education in a Digital Society" In summary, this study aligns with the theme of CIES 2025, "Envisioning Education in a Digital Society," by demonstrating how digitally enabled educational assessments can be effectively implemented in contexts with limited resources. By leveraging statistical simulations to overcome practical limitations, we illustrate new ways to integrate technology into education, thereby enhancing learning and assessment in a rapidly evolving digital landscape.

Author

• Kennedy Marigi Kamau, African Population & Health Research Centre