Search
On-Site Program Calendar
Browse By Day
Browse By Time
Browse By Person
Browse By Room
Browse By Unit
Browse By Session Type
Search Tips
Change Preferences / Time Zone
Sign In
Bluesky
Threads
X (Twitter)
YouTube
Objectives and Framework
We present a design-based implementation research (DBIR) case-study, in partnership with a large, K-12 school district in the southwestern United States. The district is engaged in an organizational change effort to help their teachers shift instruction away from students’ standardized performance metrics and support students' holistic experiences. The district sought to equip its teachers with data on students' experiences. However, teachers commonly reject data-centric initiatives due to increased workload without support (which occurred in our partners’ initiative). Uncritically designed data can also reify deficit ideas of students (Bertrand & Marsh 2015). To support the district, our team developed an AI-based tool that takes student reflections and develops analytics about students’ experiences with collaboration, creativity, critical thinking, communication, and character skills. We present on Year 1 of our design study, where we piloted the tool with 1,000 teachers and 16,000 students, gaining insight into how educators made use of our technology to inform their teaching practice.
Methods and Materials
Our data draws from a DBIR cycle that involved (a) design and development of an AI-based platform that we co-created with partner teachers, instructional coaches and leadership, and (b) fourteen focus groups, with over 100 teachers, administrators, and education specialists across sixteen partner schools. The design artifacts – including meeting notes, design notes, software artifacts, user interface prototypes etc. – contribute to our description of the design and alignment of the system that we study. Our first design cycle culminated in a prototype piloted by the partner district in Fall 2023. In focus group interviews, educators shared critical perspectives about the district’s organizational change initiative, the prototype’s design and AI-generated insights of student data.
Results
For some teachers, a performance logic emerged where teachers focused solely on student performance and achievement, which created obstacles for teachers to deeply understand student experiences in their reflections. Conversely, other teachers expressed a growth logic, where they began to see the AI-based analytics of students’ experiences as qualitatively rich information to better understand their students’ backgrounds and the assets they brought to the classroom. These shifts in seeing their students (from performance to growth) were connected to teachers appreciating their students' lives, and developing alternative visions for changing their classroom activities.
Teachers also brought up prescient insights about AI trustworthiness, transparency and ethics that we argue will be evergreen issues in future work with AI and education. For example, teachers employing a performance logic expressed concerns about AI accuracy and validity as a problem for assessing student performance. Conversely, growth-minded teachers became aware of how AI-based analysis introduces equity and ethics issues, such as not capturing the experiences of plurilingual or neurodiverse students, and constraining teachers’ ability to connect with their students.
Scholarly Significance
We share a design narrative that sheds light on pivotal design moves that we enacted and present an efficacy case of utilizing generative AI to aid in instructional change efforts. We also shed light on evergreen issues for teaching and learning that arose as they interacted with our AI system.