What Might Changes in Psychometric Approaches to Statewide Testing Mean for NAEP?

• In Event: NAEP and the Challenges Presented by a Move to Common Core State Standards

Abstract

Development of the Common Core State Standards (CCSS), and the creation of the Smarter Balanced Assessment Consortium (SBAC) and the Partnership for Assessment of Readiness for College and Careers (PARCC), changes the pattern of accountability testing. These changes raise the question: “How should NAEP’s validity and utility be maintained?” The assessments planned by the consortia may be different enough from current state assessments to raise questions as to whether NAEP can continue to play its historic role as an independent monitor or “check” on the validity of state assessments. It is also clear is that computer-based assessment is coming to K-12, and both consortia plan to include more varied item types than have been common.
Assessments developed by SBAC and PARCC may reduce the number of statewide tests to not many more than two; linkage may become feasible. Relations between the results of disparate educational assessments tend to change over time, so any linkage between the NAEP scale and the consortium statewide tests will need to be maintained regularly. A singular opportunity exists in a short window of time—essentially right now—to design data collection for linkage between the NAEP scale and the consortium assessments while the latter are under development.
Computerization of NAEP is inevitable, and already planned by NAGB. Computerized NAEP assessments may appear more similar to future statewide assessments. Comparability of results can usually be maintained as a test makes the transition from paper-and-pencil to computerized administration, but computerization may have an effect on results for some subgroups of the population. Computerization of NAEP is best approached in the same way as other changes to NAEP assessments have been approached: A bridge study should ensure comparability of results across the transition, unless an a priori decision is made to “break trend” regardless.
NAEP has a long history of implementing gradual change so that results remain comparable from year to year, while, at the same time, the assessments remain relevant in the presence of continuing educational and curricular change. We expect that spirit of gradual incremental change will continue to guide NAEP in its adaptation to the introduction of the common core assessments.

Authors

• David Thissen, University of North Carolina - Chapel Hill

• Scott Norton, Louisiana Department of Education