AI-Based Assessment of Nursing Students’ Head-to-Toe Physical Assessment Performance: Video Segmentation and Classification

• In Event: Artificial Intelligence in Action: Transforming Assessments

Abstract

Head-to-toe (H2T) physical assessment is crucial in nurse practitioner programs but demands extensive faculty time to evaluate and provide feedback. Despite recent AI advancements in nursing education, few studies focus on AI integration into performance assessment. This study proposed an AI-based performance assessment framework with three functions: segmentation, classification, and scoring. Fifty video recordings of post-licensure nursing students performing H2T assessments were used to train and evaluate vision-based (Moment-DETR, VideoMAE V2) and language-based (Llama 3.1-8B-Instruct) models in segmenting and classifying 15 representative skills. Fine-tuned language models significantly outperformed vision models, while inter-task similarity reduced accuracy across models. Verbal information is crucial for assessing students' performance. Future multimodal approaches integrating visual and linguistic data are recommended for robust clinical skills evaluation.

Authors

• Ikseon Choi, Emory University

• Sarah E Finch, Emory University

• Sejung Kwon, Emory University

• Jongwon Kim, Emory University

• Autherine Abiri, Emory University

• Elizabeth Zarantonello, Emory University

• Christine Nguyen, Emory University

• Rose Hayes, Emory University

• Laika Steiger, Emory University

• Beth Ann Swan, Emory University