(Un)intended Uses of the SAT

• In Event: Exploring Unintended or Inappropriate Interpretations, Uses, and Consequences of Test Scores

Abstract

The SAT was developed for use in college admissions. Due to the variability in the courses students take in high school and in the rigor of the curriculum offered at various high schools, there is a need to assess students’ level of academic preparation on a standardized scale. The SAT serves this need. Since its inception, hundreds of studies have been conducted validating the SAT for use in college admissions by examining its relationship with measures of college performance. However, SAT scores have been used in ways that were not intended by the test developer. This talk will focus on (a) ways in which SAT scores are used that were unintended by the test developer, (b) whether this unintended usage may be inappropriate, and (c) solutions to minimize inappropriate usage.
Test developers are responsible for defining the intended interpretations and uses of test scores as well as for collecting evidence supporting such uses (AERA/APA/NCME, 1999; Kane, 2006). The proposed use of the SAT is for college admission; therefore, the College Board has focused most of its validation efforts on examining the relationship of the SAT with college performance. If tests are used for purposes that were unintended or for which no validity evidence exists, the user bears responsibility for furnishing evidence supporting the alternative use (AERA/APA/NCME, 1999; Kane, 2006). For example, state ranking based on mean SAT scores have been reported in the media and used to draw inferences about the quality of a state’s educational system; however, evidence supporting this use does not exist.
The current study will review a variety of ways the SAT has been used that were not intended by the test developer. Whether these alternative uses are appropriate, the authors will focus on two disparate uses: one that appears appropriate – college course placement – and one that does not – state ranking. Employing a variety of educational databases, including College Board records, grades provided by US colleges, and other national databases (e.g., WICHE, IPEDS), the relationship between SAT scores and various outcomes was examined. For course placement, a moderate relationship between SAT scores and course grades was found, providing evidence for this use. As for ranking; however, results revealed a large, negative correlation between the percentage of students in the state who took the SAT and the state’s mean SAT score, suggesting this practice may not be valid, particularly when the sample is not reflective of the population.
It is important that the general public, and test users in particular, be informed about the intended and appropriate uses of tests. When alternative uses of tests are employed, research should be conducted to validate this use. Test users will find creative ways to use scores that test developers could never imagine; therefore, it is unrealistic to assume that test developers will collect evidence in support or against every potential use. Rather, it is important that test users are aware of their responsibility to provide evidence for their particular use if it does not already exist.

Authors

• Krista D. Mattern, The College Board

• Jeffrey Nagle Wyatt, The College Board