Utilizing Automated Analysis of Narrative Feedback in a Competency-Based Curriculum to Assess a Student's Readiness to Progress

• In Event: Roundtable: Assessment

Abstract

This paper describes early results of a study utilizing text mining and natural language processing (NLP) techniques to summarize large amounts of narrative feedback provided to medical students in an assessment portfolio. We propose that NLP and machine learning methods can be used to reduce the amount of time faculty spend reviewing narrative feedback to students by extracting, summarizing, and visualizing features of narrative feedback comments. This paper describes early efforts to automate the extraction of words and phrases in narratives associated with performance humans found concerning. Certain terms are associated with concerning performance, but the formulaic nature of some feedback made NLP analysis challenging. Our results suggest that NLP can be used to augment human interpretation, not replace it.

Authors

• Christina Maimone

• Celia O'Brien, Northwestern University