The Role of Teachers in Digital Personalized Assessments

• In Event: Designing AI-Powered Personalized Educational Tools by Incorporating Teachers’ Perspectives

Abstract

Objectives

Many teachers would agree that it is important to meet students where they are to provide effective instructional support (Logan, 2011). However, most assessments do not similarly provide a personalized experience (Mislevy, 2018; Sireci, 2020). One contributing factor could be the high cost for teachers to develop, administer, and score assessments that are personalized to the 20+ diverse students in the typical U.S. classroom. Artificial intelligence (AI) has the potential to address this challenge and facilitate the development and deployment of digital personalized assessments (DPA).

Theoretical Framework

The Culturally Enhanced Caring Assessments (CECA) framework (Lehman et al., 2024) provides guidance for the development of DPAs and calls for personalization to be based on a nuanced view of each individual student through considering their personal, cultural, social, and linguistic characteristics and how they experience an assessment. However, the CECA does not prescribe what characteristics or behaviors should be prioritized or how best to personalize based on those that are prioritized. This lack of prescription is because personalization should also be flexible enough to allow for variation across context, which necessitates understanding the perspectives of teachers, students, and other important interest holders.

Methods

To address this need, we conducted three focus-group sessions with nine teachers (6 female) that had an average of 14 years of teaching experience (SD=9, Range=1–28) to discuss several key issues. In the focus group sessions, we first described how we hypothesized that DPAs could help to achieve equity. We then asked teachers their thoughts on our hypothesis and whether DPAs would address any of their current challenges. Next, we asked teachers about the overall potential and concerns about the use of AI to achieve DPAs, the student characteristics that they viewed as most relevant for DPAs, and how they would want to interact with an AI tool to develop and deploy DPAs.

Results

Overall, we found that teachers (n=9) agreed with our hypothesis that DPAs could help to achieve equity. Teachers in one focus group (n=3) highlighted the importance of utilizing already available student information as the collection of new information could be challenging. Teachers in another focus group (n=3) described their current challenges around providing appropriate test-taking experiences for all students. For example, current technology capabilities do not always provide language supports that vary based on a student’s proficiency in either their home language or English.
All of the teachers (n=9) reported potential concerns with the use of AI for DPAs that align with more general concerns about AI in education (e.g., TeachAI, 2024). One focus group (n=3) highlighted that many of their concerns could be addressed by allowing teachers to set the guardrails for when and how AI was used in DPAs. We will present additional findings from these focus groups.

Significance

The findings from the present research provide insights into how to develop DPAs that address current challenges of teachers to meet the individual needs of each student in their classroom in a way that maintains the vital role of teachers.

Authors

• Blair Lehman, Brighter Research

• Reginald Marshall Gooch, Educational Testing Service

• Caitlin Tenison, ETS

• Jesse R. Sparks, Educational Testing Service