Practical Guidance and Best Practices for Gathering Validity Evidence Based on Assessment Type

Session Type: Working Group Roundtable

Abstract

In the last twenty-five years, the discussion surrounding validity evidence has shifted both in language and scope, from the work of Messick and Kane to the updated Standards for Educational and Psychological Testing. However, these discussions haven’t necessarily focused on best practices for different types of instruments or assessments, taking into account specific concerns for each instrument type. The purpose of this working group roundtable is to (1) illustrate validation activities and evidence for 4 distinct types of measures (forced-choice and mixed format knowledge assessments, Likert/Rating Scale instruments, performance assessments, and observation protocols), (2) facilitate a discussion of best practices for each of the measurement contexts, and (3) foster possible collaborations for dissemination of best practices for these contexts.

Sub Unit

• Division D - Measurement and Research Methodologies / Division D - Section 1: Educational Measurement, Psychometrics, and Assessment

Chair

• Andrew Maul, University of California - Santa Barbara

Papers

• Validation Evidence for Forced-Choice and Mixed-Format Knowledge Assessments - Cari F. Herrmann-Abell, BSCS Science Learning

• Validity Evidence for Likert/Rating Scale Instruments - Leigh M. Harrell-Williams, The University of Memphis

• Validity Evidence for Rater-Mediated Performance Assessments - Stefanie A. Wind, The University of Alabama - Tuscaloosa

• Validation Evidence for Observation Protocols - Eli Andrew Jones, The University of Memphis