BackHome
Refresh: Off
Paper Summary
Share...

Direct link:

Examining Fairness in Educational Assessment: A Scoping Review of Current Practices

Fri, April 12, 3:05 to 4:35pm, Philadelphia Marriott Downtown, Floor: Level 5, Salon C

Abstract

Background
By fostering fairness, assessment can uphold principles of social justice including countering systematic inequities for certain groups of learners, enhancing learning outcomes, and cultivating a more inclusive and diverse learning environment. Expansion from previous versions, the latest version of Standards in Educational and Psychological Testing (Standards) provides more expansive guidelines for considering fairness at all stages of assessment including test design, development, administration, scoring, and use. Given the lack of a clear definition and approach to implementation of these current standards around fairness, various stakeholders, including test developers, administrators, educators, and policymakers, may interpret and implement the standards differently. The goal of the present study is to characterize the current landscape of how fairness is conceptualized in assessment practice across various educational contexts in the literature and to provide future directions for how to promote fairness in the assessment system and practice.

Method
The authors conducted a scoping review to investigate how the literature across the educational continuum from elementary school to health professions in education defines fairness in assessment and how the Standards are operationalized. A systematic search was conducted in August-September, 2022 using four databases: PubMed, Embase, Web of Science, and ERIC. Of these, 769 records met the inclusion criteria. These keyword searches generated a total of 316 references with resulting in 74 articles after abstract and title reviews. We conducted content analysis for the definitions extracted directly from the articles and generated descriptive codes (e.g., social justice definition, equity, rater bias, psychometric issues, procedural standardization, test security, and validity).

Results
About 31% (n=23) of the articles specifically mentioned the Standards with regards to fairness. Of those, the concept of fairness described aligned most with the following categories of the standards: 1. Test development and administration (n = 22, 30%), 2. Test score interpretation (e.g., construct irrelevant variance, association to other variables, item analysis/bias, reliability) (n = 38, 51%), 3. Test accommodations (e.g., accessibility and inclusion of all students, adaptation to different languages) (n = 4, 5%), and 4. Test consequences for subgroups (e.g., judgments about placement, standard setting) (n = 8, 11%). More than half (59%) targeted learners from K-12 or higher education. Medical students and residents/fellows were the focus in 16% of the articles. Content analysis revealed that conceptualization and operationalization of fairness varied in terms of scope from narrowly specific to one area to description encompassing multidimensional and complexity of the fairness concept.

Conclusions
This scoping review adds to the literature on fairness in assessment by characterizing how the most recent fairness-related Standards were conceptualized and operationalized in the education literature. Across education contexts, fairness was conceptualized both broadly and narrowly and was positioned as subject to threats are multi-factorial including macro-societal as well as programmatic assessment level issues with fewer than one-third specifically mentioning the Standards, suggesting idiosyncratic approaches to considering fairness in the literature. Adoption of a more sociocultural lens to assessment may improve the Standards to be more broadly applicable and relevant to social contexts of assessment practice.

Authors