Principles in Practice: Building Justice-Oriented Assessment Systems at the University of Colorado Children’s Hospital and The Children’s Hospital of Philadelphia

• In Event: Dreaming, Designing, and Doing Justice: Justice-Oriented Assessment Systems in Medical Education

Abstract

Enacting justice-oriented assessment requires critiquing, reimagining, and redesigning assessments from the perspective of marginalized individuals and communities, taking an explicitly antiracist and anti-oppressive stance. Within this orientation, assessments are dismantled and re-built through co-design by diverse users - learners, assessors, etc. - in the context of their local programs. Justice-oriented educators have successfully co-designed more equitable assessments in other fields of education, but we currently know little about how to apply justice-oriented assessment to co-design assessment systems in medical education. The focus of this paper is evaluating an innovative approach to co-design justice-oriented assessment systems which was guided by Design Justice. We sought to evaluate: to what extent did the project align with Design Justice principles?
This project took place in the context of two general pediatrics residency program in large, urban, academic, free-standing children’s hospitals in the United States, one in the West and one in the Northeast. First, semi-structured interviews and observations of end users were conducted, including resident learners, faculty supervisors, and program leadership and staff at each site. Documents (i.e., assessment tools/forms) were also collected. Then, site-specific design sessions were facilitated to come to consensus on local needs around equity, brainstorm hopes and values for equitable assessment, and co-create testable prototype assessment systems. Number of design sessions held, and number/demographics of participants involved, was recorded in each step of the design process. Systematically collected data from all design sessions included facilitator notes, de-identified transcripts of session conversation, any outputs/artifacts (pictures, collages, sticky notes, etc.) produced during the sessions, and anonymous feedback cards with answers to session evaluation questions. Three authors independently analyzed all qualitative data by deductive content analysis using Design Justice principles as codes to examine alignment with principles. Analysis of qualitative data showed that some Design Justice principles were salient across all steps. Some Design Justice principles were specific - especially salient during particular steps of the design process.
This project demonstrated that Design Justice can be used to guide the co-design of a justice-oriented assessment system for pediatric residency programs. As programs increasingly try to advance equity in assessment, this project illuminates the importance of not only intending equitable outcomes but also ensuring an equitable process in the act of advancing equity in assessment. Design Justice afforded this project several important things: participants’ feelings of being heard, affirmed, and empowered while designing, as well as the design teams’ iterative reflection on how to make the project accessible, accountable, transparent, and collaborative.

Authors

• Laura Lockwood, University of Colorado Anschutz Medical Campus

• Dorene F. Balmer, University of Pennsylvania

• Pricilla Valeria Senne Cabral, Children's Hospital of Philadelphia