Generative AI meets the heritage bilingual: A classroom-based study on writing feedback

• In Event: Generative AI and Academic Writing: Studies from Diverse Contexts

Abstract

Heritage bilinguals, who grew up speaking a non-English language at home (or heritage language) in the U.S. context, report writing as the most challenging skill for them to develop in their heritage language, often leading to feelings of anxiety (e.g., Callahan, 2010; Carreira & Kagan, 2011; Hedgcock & Leftowitz, 2011). While prior studies have examined heritage bilinguals’ writing development and processes (e.g., Bowles & Bello-Uriarte, 2019; Torres, 2023), little is known about the effects of corrective feedback and the use of generative AI as a source of writing feedback (e.g., Jegerski & Ponti, 2014; Torres & Saiidi Padilla, in review). Accordingly, the current presentation aims to deepen our understanding of how heritage bilinguals respond to and make use of generative AI feedback in their writing. The data for this presentation come from classroom-based studies involving Spanish-English heritage bilinguals enrolled in a tailored writing course. These students used generative AI to revise the first drafts of their argumentative essays. To evaluate the impact of generative AI feedback on students’ writing experience and language use, we drew on the following sources: a questionnaire regarding students’ use of generative AI, transcripts of heritage speakers’ interactions with generative AI, revisions made to their drafts based on generative AI feedback, and students’ written reflections. Notably, we compare students’ use of ChatGPT without elaborate prompt engineering to their use of PapyrusAI with prompt engineering.

Drawing on the data sources, we first conducted a bottom-up qualitative analysis –– from opening to thematic coding –– to identify emerging themes following a grounded theory approach. Second, we examined students’ essay revisions based on generative AI feedback on their written language (e.g., lexical items, transition words) for their first and third argumentative essays. The results revealed five themes, which can be grouped into three categories: essay content and organization, language-related issues, and emotional responses. Furthermore, based on generative AI feedback, students made a higher number of revisions to lexical items, complex T-units, and complex nominal clauses in Essay 1; however, revisions in these categories significantly decreased by their Essay 3, with the exception of lexical items. Lastly, the results also revealed how students responded more favorably to the use of PapyrusAI with prompts tailored for heritage language students. The findings of the current studies will inform future experimental research and pedagogical practices involving the use of generative AI in HL classrooms.

Author

• Julio Torres, University of California - Irvine