Using Subsets of Items to Estimate Cut Sores on a Test of English Language Proficiency

• In Event: 49.088 - Roundtable Session 17
  In Roundtable Session: 49.088-18 - Standard Setting: Issues and Applications

Abstract

The Angoff method for setting standards is time consuming with large numbers of test items. Building on previous research, a G-theory framework was used to determine if a subset of items can produce generalizable cut-scores in a language testing context. Data from previous standard setting studies for general language ability test were used in a re-sampling study. Proportionally stratified subsets of items were extracted under various conditions. G-theory statistics, expected standard error (SE) around the mean, and RMSD were estimated at each condition. The expected SE and RMSD decreased as the number of items increased, but this reduction diminished after 45 items (out of 100). G-study results indicate between 30 and 45 items are sufficient to make generalizable recommendations.

Authors

• Adrienne Sgammato, ETS

• Priya Kannan, Educational Testing Service