Justice-oriented Assessment: Origins, Perspectives, and Arguments

• In Event: Dreaming, Designing, and Doing Justice: Justice-Oriented Assessment Systems in Medical Education

Abstract

Despite increasing discussion and scholarship, equity in assessment is rarely defined and distinguished in a way that allows for shared understanding in medical education. This paper seeks to clarify and expand the conversation about equity in assessment by critically reviewing justice-oriented assessment. Justice-oriented assessment is one approach to advancing equity in assessment. In a world of rapidly changing demographics, social expectations, technology, and political tides, justice-oriented scholars question whether intentions of fairness or granting access to learning via inclusion will ever be sufficient to advance equity. In contrast to approaches which address equity using educational and organizational strategies, justice-oriented assessment is based on the idea that equity in assessment is foundationally a historical, social, cultural, and political problem that is reflected and perpetuated in educational systems. Therefore, advancing and sustaining equity requires more than educational or organizational strategies; it requires explicitly addressing historical, social, cultural, and political root causes by intentional, deliberate, proactive measures. A key assumption of justice-oriented assessment is that no assessment is, or ever can be, truly neutral or impartial. The content, behaviors, skills, and competencies assessed in assessments will always reflect social and cultural characteristics, histories, and sources of knowledge. Therefore, constructs cannot be neutral or objective, and assessments of those constructs cannot be impartial. Instead, we can strive to make assessments just. From a justice-orientation, the central questions are: whose values, histories, cultures, and sources of knowledge are considered construct relevant (and therefore neutral, objective, and assessable)? And whose are considered construct irrelevant (and therefore scrubbed or removed from assessments)? Justice-oriented assessment takes a unique perspective that critiques established assessment constructs and practices from the point of view of marginalized groups. For example, justice-oriented scholars have critiqued existing validity frameworks for their role in upholding white supremacy in assessment, and in response, constructed new frameworks that explicitly take an antiracist stance, recognizing racial and social context and integrating social and cultural knowledge from marginalized racial groups directly into construct development. To promote equity from a justice orientation involves dismantling the current system and having end users, especially marginalized individuals, lead the redesign of assessments. End users might involve marginalized learners, faculty, and leadership, and, in the context of medical education, would also include program staff, interprofessional providers, and patients and community members who are impacted by the quality, safety, and equity of assessments. Importantly, the design process and outcomes should be co-led and co-controlled by impacted users, especially marginalized users. Justice-oriented assessment also recognizes the harms that co-design can cause (e.g., minority tax) and seeks to mitigate those outcomes via accountability and co-ownership. In practice, this can take the form of adhering to agreed-upon design principles, negotiating community decision-making about how assessments are designed and combined, and creating program evaluation processes that prioritize the impacts on the program’s community of marginalized learners and stakeholders over [good] intentions of educators and leaders.

Authors

• Hannah L. Kakara Anderson, Maastricht University

• Marjan Govaerts, Maastricht University

• Layla Abdulla, Children's Hospital of Philadelphia

• Dorene F. Balmer, University of Pennsylvania

• Jamiu O. Busari, Maastricht University

• Daniel C West, University of Pennsylvania