Promoting Teacher Identity Reconstruction in Artificial Intelligence–Based Assessment Practices (Poster 11)

• In Event: Artificial Intelligence–Augmented Assessment Tools to Support Instruction in STEM

Abstract

Objective
Artificial intelligence (AI) advances assessment practices because of its capability of automatic scoring, which is usually completed by teachers. AI thus provides the opportunity for teachers to use timely feedback to adjust teaching and design novel activities (Zhai, et al., 2020). This promise can transform the teaching and learning paradigm in classrooms, where the relationships between teachers, students, and technology may need to be redefined. In this study, I specifically focus on the role of AI and how AI alters teachers’ identities in AI-augmented classrooms by systematically reviewing the AI-based assessment literature. The study answered two questions:
(1) What are the pedagogical roles and characteristics of AI in assessment?
(2) What relationships do teachers and AI build in AI-based assessment?
Perspective
This study adopted the Actor-network theory (Latour, 1987) which suggests equal treatment of human and non-human materials and their actions to form dynamic networks. The consequents of networks can be analyzed through human roles, identities, and behaviors.
Methods and Data Sources
A search WoS and Google Scholar returned 745 articles that focus on AI-based assessment. Using a set of inclusion criteria, I found 37 eligible studies to be included in this review. I developed codes according to Actor-network theory and conducted qualitative inductive analysis (Mayring, 2014; see Figure 8).
Results
AI’s pedagogical roles and characteristics. I found that AI plays three different pedagogical roles. First, AI diagnoses students’ real-time performance by monitoring their learning activities, behaviors, and cognition (Arroyo et al., 2014; Howard et al., 2017), which is defined as learning monitor and analyst. Second, 25 out of 37 studies reported AI’s ability to score automatically and provide immediate feedback on complex assessment constructs. This feature refers to AI as the learning assistant. Third, AI deploys various techniques to detect students’ affective and psychological states (Sullivan & Keith, 2019) when assessing students’ performance in multimodality. This suggests AI’s pedagogical role as the affection carer.
The characteristics of AI-human interactivity denotes that AI “understands” varied data naturally in ways of language, gesture, and affect. AI can also process student performance data in an appropriate way for teachers to interact with and make decisions (Buttussi & Chittaro, 2020; Fridin, 2014). Automaticity is AI’s ability to score and return feedback automatically with limited teacher interventions. Autonomy refers to AI’s “understanding” of what is happening in the environment during the assessment process without being overridden by external agency, such as teachers.
Teacher Identity: Teacher-machine partner relationships. The data suggested novel partner relationships between teachers and AI. The paper will present examples of how AI partnered with teachers in many core activities in assessment. AI empowers teachers to perform better during assessments. AI supports teachers to conduct the performance-based assessment in various contexts, and forms partnerships with teachers to better assess student performance.
Significance
This study examines teachers’ assessment identity by suggesting the roles and characteristics of AI and teacher-AI relationships. It implies that teachers would not be substituted by AI; instead, a novel partnership is shaped between teachers and AI.

Author

• Lehong Shi, University of Georgia
  Presenting Author