Essay Writing Process and Quality in Word vs. ChatGPT Argumentative Writing

• In Event: AERA Poster Session Friday 1:45 pm
  In Poster Session: Instructional Design, Feedback, and Learning Strategies

Abstract

Recent surveys suggest many students have generative Artificial Intelligence (GAI) do their writing for them. Little is known about what happens during the writing process with GAI. We collected data from post-secondary students writing with Word or ChatGPT who read 4 texts and wrote an argumentative essay while thinking aloud. Analyses suggest different approaches to writing, and better essay scores for the ChatGPT-authored essays. Using ChatGPT, participants did more re-reading of our directions, reading ChatGPT’s writing, and noted the length of essay. Using Word, participants more often skimmed the writer’s own writing and verbalized about mechanics. Groups did equal re-reading of the presented texts. Results are likely task-specific so we do not believe findings support using ChatGPT in academic writing.

Authors

• Jennifer G. Cromley, University of Illinois at Urbana-Champaign

• Martin Lehrer, University of Illinois at Urbana-Champaign

• Timothy Thieme, Parkland College

• Matt Gadbury, University of Illinois at Urbana-Champaign

• Pooja Roy, University of Houston