Fine-Tuning a Pretrained Language Model to Classify Student Writing Exercises for Self-Affirming Attributes

• In Event: Modern Psychometric Methodologies and Applications

Abstract

Previous studies have demonstrated that this self-affirmation intervention positively impacts student school performance, and there is active research on this intervention. However, the manual coding of these exercises has proven to be a time-consuming and an expensive undertaking. To assist future self-affirmation intervention studies or educators implementing the writing exercise, we employed our labeled data to fine-tune a pretrained language model that achieves a comparable level of performance to that of human coders (interrater reliability: 0.85 between machine coding and human coders as compared to 0.83 between human coders). We intend to make the final model publicly available so that the community can avoid the burdensome task of manual coding these exercises.

Authors

• Cong Ye, American Institutes for Research

• Trisha Borman, Measured Decisions, Inc.

• Geoffrey D. Borman, Arizona State University