Paper 3: Classroom Studies: How Does an AI partner Influence Small Group Collaboration?

• In Event: Learning from Implementation Research: How Curriculum, Professional Learning, AI Tools and Classroom Studies Can Support AI-Literacy

Abstract

Purpose
This study reports on students’ engagement with an AI partner that intends to support collaboration in small group activity and offers findings to further refine the tool.

Perspective
Collaboration fosters critical thinking, social interaction, and the co-construction of disciplinary knowledge, while preparing students for the 21st-century workforce (Fiore et al., 2018). Teachers play a key role in facilitating small collaboration, but this can be difficult in large classrooms.

A promising use of learning analytics is its ability to deliver systematic, real-time feedback to learners. Project X leveraged this potential to develop an AI partner called [redacted] to support student collaboration during jigsaw tasks (Aronson & Patnoe, 1997). The partner was embedded in Lesson 4 of the Moderation Unit, which focused on moderation approaches in online communities. Students first generated criteria for evaluating moderation (e.g., speed, accuracy, cost), then read and summarized one of five moderation approaches (e.g., volunteer moderators, bag-of-words). As a group, students shared their summaries and ranked the approaches based on their criteria. The AI partner, embedded in an online worksheet, analyzed group discourse and provided minute-by-minute feedback during the activity.

Methods
The AI partner design (see Figure 1) drew on research on productive uncertainty (Chen, 2022; Kirch & Siry, 2012; Manz, 2024). The AI partner feedback was designed to maintain task complexity and invite deeper engagement and collaboration. We trained the partner to provide 4 types of feedback: Problematize, Social Support, Connect, and Stabilize (see definitions in Table 1). A mic and webcam captured small group activity; backend models transcribed and coded the speech to identify 8 possible states (e.g. no speech, uneven speech). Transcripts were used to generate 1-sentence summary and feedback tailored to each group’s collaborative state. The AI partner delivered feedback every minute based on real-time analysis

Data Sources
49 groups of middle school students engaged with the AI partner in Lesson 4. Of these, 41 yielded usable data. The groups were categorized by their frequency of interaction with the AI partner: Really Good (multiple, extensive), Good (>3 interactions), Low (1-2), and None. We examined the five Really Good videos to analyze the range of AI prompts that were generated and how students responded to them. For these data, we coded how students relayed and reacted to the feedback, including emotional valence, number of responders, depth of uptake, to explore the partner’s influence on collaboration and task completion.

Results and Discussion
Students often read AI feedback verbatim. Reactions varied—some expressed amusement, annoyance, surprise, or neutrality. Depth of uptake also varied: some acknowledged feedback briefly, others integrated it into group discussion. Notably, the student at the computer—who received and read the AI feedback—shaped how it was taken up by others. We are currently analyzing how the 4 feedback types relate to students’ response patterns.

Significance
These findings explore how AI can support collaboration in a jigsaw task. Our findings will inform refinements to the AI partner and the participatory structures designed into Lesson 4.

Authors

• Mon-Lin Ko, University of Colorado - Boulder

• Jason Reitman, University of Colorado - Boulder

• Brooklyn Clines, University of Colorado - Boulder

• Sierra Rose, University of Colorado - Boulder

• Emily Watts, University of Colorado - Boulder

• Chelsea Chandler, University of Colorado - Boulder

• Peter W. Foltz, University of Colorado - Boulder