Methodological Challenges and Solutions for Administering Large-Scale Assessments in India

Session Type: Coordinated Paper Session

Abstract

This symposium presents our collaborative efforts with India's assessment governmental agency to improve assessment methodologies to promote evidence-based policies for equitable opportunities across genders and social classes. The first presentation introduces the National Achievement Survey (NAS), which is one of the key measures of student achievement in India. In the second study, we investigate differences between test designs that administer all subjects of interest to all students and test designs that administer a subset of these subjects. The goal here is to maximize the precision of group-level statistics while taking into account the constraints related to sample size and testing time. In the third study, proficiency estimates of subgroups derived from plausible values (PVs) are compared with those derived with weighted maximum likelihood estimates (WLEs). In the fourth study, principal components analysis is compared to principal components regression and partial least squares in reducing the large number of background variables often seen in IRT latent regression models used in LSA. The final study compares currently used operational ad-hoc procedures for treating missing responses with modern imputation-based approaches.

Sub Unit

• Individual Papers, Innovation Demonstrations, Coordinated Sessions, or Organized Discussions

Session Organizer

• Jonas Bertling, EDUCATIONAL TESTING SERVICE

Individual Presentations

• The National Achievement Survey in India - Indrani Bhaduri, National Council of Educational Research and Training, India; Dinesh Prasad Saklani, National Council of Educational Research and Training, India; Jonas Bertling, EDUCATIONAL TESTING SERVICE

• Methodological Considerations in Measuring Single or Multiple Subjects in Large-Scale Assessment Design - Han-Hui Por, Educational Testing Service ETS; Daniel McCaffrey, EDUCATIONAL TESTING SERVICE; Indrani Bhaduri, National Council of Educational Research and Training, India

• Comparing Group Scores from Plausible Values and Weighted Maximum Likelihood Estimates - Parth Tusharbhai Soni, Indian Institute of Management Ahmedabad; Daniel McCaffrey, EDUCATIONAL TESTING SERVICE; Indrani Bhaduri, National Council of Educational Research and Training, India

• A Comparison of Data-reduction Techniques for IRT Latent Regression - Peter van Rijn, ETS GLOBAL; Jonas Bertling, EDUCATIONAL TESTING SERVICE; Indrani Bhaduri, National Council of Educational Research and Training, India

• Modern Techniques for Treatment of Missing Item Responses in Large-Scale Survey Assessments - KAMAL CHAWLA, UNIVERSITY OF DELAWARE; Usama Ali, EDUCATIONAL TESTING SERVICE; Peter van Rijn, ETS GLOBAL