Addressing Challenges in Formative Assessment Practices by Artificial Intelligence: A Systematic Review (Poster 8)

• In Event: Artificial Intelligence–Augmented Assessment Tools to Support Instruction in STEM

Abstract

Objectives
Formative assessment practices are critical for teaching and learning (e.g., Black & Wiliam, 1998). Artificial Intelligence (AI) has great potential to improve formative assessment practices (Perikos et al., 2017; Zhai, 2021). However, most studies so far focus more on the technology than on the educational pedagogical models behind the AI-augmented tools designed (González-Calatayud et al., 2021; Zhai et al., 2020).
This paper systematically reviews the empirical studies in which K-12 science teachers’ formative assessment-related constructs were measured to identify the challenges teachers are facing in implementing formative assessments and propose potential ways to address the challenges using AI technology. The following research questions (RQs) guided our study:
1. What challenges do teachers encouter in formative use of assessments?
2. What are the potential uses of AI in addressing the challenges?
Theoretical Framework
Researchers have proposed various formative assessment models (e.g., Shirley & Irving, 2015; Sezen-Barrie & Kelly, 2017; Zhai et al., 2018; Duckor & Holmberg, 2019). After reviewing their key components and steps, we proposed a concise formative assessment model (Figure 3) to capture three commonly used components in the formative assessment circle: elicit response, interpret responses and identify learning gaps, and enact instruction (EIE). We used this model to organize the challenges teachers face in formative assessment practices.
------------------------------------------------Figure 3---------------------------------------------------
Method
Figure 4 shows the article search process. We used ProQuest, ERIC, APA PsycInfo, and Google Scholar electronic database for literature search. After the search and filtering, we selected 29 journal articles and nine dissertations that measure formative assessment related constructs and 13 articles that connect AI with formative assessment. We coded the 38 publications using coding rubrics (Miles et al., 2019) that focus on the challenges teachers facing in formative assessment practices.
------------------------------------------------Figure 4---------------------------------------------------
Results
The studies covered teachers in various contexts: (a) Type of schools: elementary schools (23.68%), middle school (28.95%), high school (18.42%), and the combination of different grade levels (28.95%). (b) Type of teachers: Preservice (7.89%) and In-service (92.11%); (c) Countries: United States (68.42%), international (26.32%) and not specified (2.63%). Based on our theoretical framework and codings of the literature in the preliminary analysis, we identified the key challenges in teachers’ formative assessment practice and proposed some potential uses of AI to address those challenges, which are summarized in Table 4 using EIE framework.
--------------------------------------------------Table 4--------------------------------------------------
Significance
This study identifies the challenges science teachers face in implementing formative assessment and suggests the possibility of AI-Augmented tool has not been fully explored. It is necessary and feasible to apply AI to help STEM teachers to meet the challenges in implementing formative assessment practice. This study highlights the importance of educational theories in AI design in addition to the technology development around AI.

Authors

• Ruiping Huang, University of Illinois at Chicago
  Presenting Author

• Yue Yin, University of illinois at Chicago
  Presenting Author

• Sania Z. Zaidi, University of Illinois at Chicago
  Presenting Author

• Ying Chen, Illinois Workforce and Education Research Collaborative (IWERC)
  Presenting Author

• Mikayla Strasser, University of Illinois at Chicago
  Presenting Author