Cheating on Tests: A Threat to Response Validity

• In Event: Practical Issues in Testing and Assessment

Abstract

Cheating on standardized tests, how it is detected and what is done about it are confidential and methods of detection are often treated as proprietary. In this paper, we propose that cheating be treated as a threat to response validity. In this conceptualization, the validity argument approach (Kane, Crooks and Cohen, 1999) would be used to support taking four levels of action: 1. purge the responses from the test data-base; 2. launch a “forensic” search of the evidence; 3. withhold reporting scores/require retaking the exam; 4. pursue charges of fraud, or other serious allegations.

Treating cheating as a response validity concern changes the nature of the process from being an action taken against individuals and the potential backlash that such action can yield to developing an argument for whether the responses should be pulled from the data-base. This is a low stakes decision that merits the development of tools to detect low levels of cheating and their impact on test analysis data and other aspects of the test development process. In the process of developing that argument, it may come to light that a higher level action is merited, but that would not be the focal point. Processes for confirming response validity could become standard practices incorporated into the data that would be a by-product of normal test analyses, much like difficulty and discrimination-type data are done currently.

The other part of considering cheating as a threat to response validity is that it enables the validity argument to be crafted to the situation. As technology capability increases, examinees can find ever more creative ways to fraudulently obtain an item answer. Test developers need to have tools to keep ahead of the game. The ability to have a validity argument in their arsenal will help to level the playing field.

Authors

• Mark A. Albanese, National Conference of Bar Examiners

• James A. Wollack, University of Wisconsin - Madison