Toward More Equitable Measurement and Assessment: Implications for the Test Standards

Session Type: Invited Speaker Session

Abstract

Abstract:
The Standards for Educational and Psychological Testing is the gold standard guiding the “sound and ethical use of tests” (Plake & Wise, 2014, p. 2). While fairness is a core principle, is the Standards’ current conception sufficient to meet today’s needs? The COVID pandemic and Black Lives Matter have underscored longstanding disparities and opportunity gaps. Demands for equity are pervasive; growing research documents a range of cognitive, psychological, social and cultural factors that mediate performance; and backlash against testing and its role in maintaining the status quo is growing.

This interactive symposium explores how the Standards may need to change to address current demands for equity and fairness. Feedback from the symposium will contribute to the Standards’ upcoming revision.

Summary
According to the 2014 Standards, a fair test “reflects the same construct(s) for all test takers, and scores from it have the same meaning for all individuals in the intended population; a fair test does not advantage of disadvantage some individuals because of characteristics irrelevant to the intended construct (p.50).” This idea of equivalence of construct meaning– what is being measured – permeates the overall fairness standard and the 20 specific standards that comprise the chapter. The overall standard presents the guiding principle of the chapter and infuses all operational chapters.
Standard 3.0: All steps in the testing process, including test design, validation development, administration, and scoring procedures, should be designed in such a manner as to minimize construct-irrelevant variance and to promote valid score interpretations for the intended uses for all examinees in the intended population.

The specific standards are organized into four clusters dealing with test design, development, administration and scoring; validity of test score interpretations; accommodations to support valid score interpretations; and safeguards again inappropriate score interpretations for intended uses.
The Standards make clear that fairness cannot be an afterthought, but rather is a fundamental principle that must guide all stages of testing, beginning with test design and ending with score interpretations and use. Still, the Standards define fairness in terms for which measurement professionals can reasonably be accountable. They focus on measurement bias as a central theme, but also include attention to accessibility, the concept that “…all test takers should have an unobstructed opportunity to demonstrate their standing on the construct(s) being measured.” (AERA, APA, NCME, 2014, p. 49); universal design, meaning design that takes into account the characteristics of all members of the intended population (see Thompson, Johnstone, & Thurlow, 2002), and the reality that test adaptations may be needed to assure fairness for some individuals.

The measurement orientation of the fairness standards has been criticized as too narrow, as has the Standards’ indeterminant stance on the role of consequences in test development and use. Recent research on socio-cultural learning and the role of cultural relevance also give pause. What, if any, changes might be needed for the upcoming revision of the Standards to strengthen its approach to equity and fairness? This is the central question addressed by the symposium.

The symposium will start with an overview of the Standards revision plan. Then three prominent researchers specializing in equity will present brief opening remarks (6-8 minutes) on what changes are needed in the Standards approach to fairness and why. Symposium participants will then have the opportunity to raise questions and provide feedback. The session chairperson will also serve as discussant to conclude the session

Sub Unit

• AERA Research and Science Policy Series

Chair

• Frank C. Worrell, University of California - Berkeley

Discussant

• Frank C. Worrell, University of California - Berkeley

Panelists

• Jennifer Randall, University of Michigan

• Michael C. Rodriguez, University of Minnesota

• Guillermo Solano-Flores, Stanford University