Measuring College Students' Reliance on Large Language Models in Academic Writing: A Mixed-Methods Analysis (Poster 6)

• In Event: Division D Graduate Student Committee In-Progress Research Gala Poster Session

Abstract

This study examines U.S. college students' reliance on Large Language Models (LLMs) for academic writing using a mixed-methods approach. Preliminary findings from 200 students, based on a 5-point Likert scale, reveal moderate proficiency with LLMs (mean = 3.26) but low formal AI training (mean = 1.14). Students reported moderate reliance (mean = 3.05) and familiarity (mean = 4.12). Additionally, 67% found LLMs enhance writing clarity, while 45% expressed concerns about overreliance and ethics. Higher AI literacy correlates with more strategic, reflective use, improving writing quality. The study identifies distinct LLM behaviors and develops a scale for measurement.

Author

• Shahin Hossain, University of Maryland - Baltimore County