Evaluating AI-Generated Writing and Disciplinary Expertise in the Sociology Classroom

• In Event: 20461 - Teaching and Learning Symposium Roundtable Session
  In Refereed Roundtable (90 minute): Table 2: AI & Teaching

Abstract

As artificial intelligence (AI) tools like ChatGPT become more integrated into academic settings, it is crucial for students to develop critical evaluation skills to assess the accuracy and depth of AI-generated content. We will present an in-class activity designed to help sociology students recognize the role of disciplinary expertise in evaluating information quality.

Through a structured three-part exercise, students first analyze an AI-generated sociology essay, then critique an AI-generated response on a topic within their personal expertise and finally reexamine the sociology essay with a more critical lens. This process helps students identify limitations in AI-generated writing, such as oversimplifications, missing key concepts, and a lack of depth. The activity concludes with a reflection on the importance of subject-matter knowledge in assessing credibility, a skill applicable to their coursework and beyond.

This session will provide attendees with a ready-to-implement activity that fosters critical thinking, media literacy, and engagement with AI as both a tool and a subject of scholarly critique. The discussion will also explore broader implications for teaching sociology in an era of generative AI.

Authors

• JoAnna Boudreaux, University of Memphis

• Kendra Murphy, University of Memphis