Exploring the Application of Generative AI in Educational Psychological Measurement: Virtual Personas and Their Validity

• In Event: Research In Progress Roundtables Session 4
  In Invited Roundtable: Education Research and Psychometrics (Table 16)

Abstract

The study explores the application of Generative AI in educational psychometrics by generating 2,000 virtual student personas using ChatGPT. These personas completed three academic measurement scales: AMS,MSLQ and AEQ. The reliability and validity of the generated data were analyzed through descriptive statistics, correlation analysis, and principal component analysis (PCA), comparing it with published human subject data. Results show high consistency in score distributions and theoretical factor structures between virtual and human data, highlighting Prompt design’s influence on generated responses. This research demonstrates a low-cost, efficient method for validating psychometric tools and explores ChatGPT’s potential in modeling human psychological behaviors, paving the way for integrating educational psychology and AI technology.

Author

• Huanxiao Wang, University of Pennsylvania