Beginning with the end in mind: Using generative AI in undergraduate STEM writing instruction

• In Event: Generative AI and Academic Writing: Studies from Diverse Contexts

Abstract

As generative AI becomes ubiquitous, writers must decide if, when, and how to incorporate generative AI into their writing process. Educators must sort through their role in preparing students to make these decisions in a quickly evolving technological landscape. We created an AI-enabled writing tool, PapyrusAI, that provides scaffolded use of a large language model as part of an NSF-funded research study on integrating generative AI into an upper division STEM writing course.
Drawing on decades of research on integrating digital tools into instruction and writing research, and best practices for generative AI design, we discuss the framework that drove our initial design considerations and instructional resources. We prioritized making the tool reliable (design responsibly), transparent (design for mental models), pedagogically sound (design for appropriate trust and reliance), and simple for both instructors and students to use (design for co-creation and imperfection). The tool was designed to meet the pedagogical needs and concerns of instructors -- allowing them to select which types of interactions with AI are available and providing them access to view the text of the student-AI conversations. Student use is scaffolded because the tool includes a researcher-designed initial prompt, which guides the AI interaction based on sound pedagogy to reinforce and support classroom instruction. AI literacy is supported both through our design of the tool (e.g., the initial prompt is visible to students, allowing them to build accurate mental models of how generative AI works and also providing them with examples of quality prompts), and through instructional resources such as foundational curriculum on how generative AI works, its limitations and biases, prompting techniques, and AI’s environmental impact.
We then share our findings from two years of design-based implementation research. We describe the evolving best practices of human-driven generative AI use, ensuring that students think first, act as “the boss” of the AI through iterative prompting, corroborate and interrogate AI output, and reflect on their writing process. We also provide emerging best practices for instructors, beginning with identifying learning objectives, determining the appropriate AI role, revising the content, reflecting on the revised curriculum, and reintroducing learning as needed. Our instructors had varying styles of teaching and viewed their roles through different lenses; this variety of perspectives was reflected in their use of generative AI in the writing classroom. We will discuss the range of usage that we saw and how instructors were able to successfully integrate the tool into their existing course structure. We will share some examples of quality AI implementation that promote learning, rather than encourage off-loading of essential skill building.

Author

• Tamara Powell Tate, University of California - Irvine