Measuring Leadership within a Computational Psychometrics Framework: A Multi-Modal Analysis of Behavioral Traces with LLMs

• In Event: AI for Higher Education Assessment

Abstract

This study develops a novel AI-assisted assessment framework for measuring emergent leadership in collaborative tasks within higher education. Drawing on computational psychometrics, we propose a claim–grant interaction model that captures leadership-related behaviors through multimodal process data, including verbal dialogues, interface actions, and system responses. By leveraging large language models (LLMs), we automate the identification of leadership behaviors and generate high-resolution, time-aligned behavioral indicators. These indicators exhibit strong external validity, correlating with students’ personality profiles. Moreover, asymmetries in claiming and granting behaviors significantly predict task performance, reflecting the formation of leader–follower dynamics. Our findings demonstrate the methodological potential of LLM-enabled behavioral analytics to support scalable, evidence-centered assessment of non-cognitive competencies.

Authors

• Manru Wang, Peking University

• Shiqian Huang, Peking University

• Xiaoting Huang, Peking University