Prompt Engineering Techniques for Consistently Relevant Math-Science Conversations With an AI-Powered Student

• In Event: From Animation to AI: Innovative Tools for Modeling and Assessing Science Learning

Abstract

Integrating artificial intelligence (AI) into simulation-based training offers many advantages, but the novelty of AI technology leaves many unaddressed contingencies. This study works to expand our understanding by exploring AI’s role in preservice teacher training simulations. Virtual student agents, powered by fine-tuned large language models (LLMs), help create authentic and cost-effective training simulations where preservice teachers can safely practice pedagogical skills. The objective of this study is to assess the performance of three prompt engineering techniques used in asking STEM-based word problems designed to elicit reasoning from a pretrained and fine-tuned LLM designed to act as a teachable student. Preliminary results indicate inconsistency in accuracy and relevance in math-science conversations by the AI-powered virtual student between testing instances.

Authors

• Alex Barrett, Florida State University

• Fengfeng Ke, University of Maryland

• Saptarshi Bhowmik, Florida State University

• Luke West, Florida State University

• Nuodi Zhang, Florida State University

• Chih-Pu Dai, Texas A&M University

• Xin Yuan, Florida State University