Designing Chatbots for Higher-Order Thinking: A Comparative Study of Multiple Persona vs Tutor Chatbots

• In Event: AERA Poster Session Thursday 9:45am
  In Poster Session: Higher Education and Student Learning Outcomes

Abstract

This study examines how different generative AI chatbot designs influence higher-order thinking (HOT) in graduate education. We compare a multiple persona chatbot design that simulates conversations with and amongst multiple education experts/theorists, with a tutor chatbot that uses structured prompts to scaffold the conversation. HOT was examined through four dimensions: Initiation, Conceptual Linkage, Critique, and Transfer. Quantitative and qualitative analyses reveal that the multiple persona chatbot increased initiation and critique, fostering autonomous inquiry. In contrast, tutoring dialogues elicit more conceptual linkage and transfer, but often through prompted reproduction. The multiple persona chatbot elicited independent, real-world transfers. Findings suggest HOT processes and outcomes are dependent on chatbot design. We propose aligning chatbot designs with pedagogical goals to enhance HOT development.

Authors

• Chee Kit Looi, The Education University of Hong Kong

• Ziyan CHE, Central China Normal University

• Dan Sun, Hangzhou Normal University

• Longkai WU, Central China Normal University