AI-SCORER: An Artificial Intelligence–Augmented Instructional System (Poster 9)

• In Event: Artificial Intelligence–Augmented Assessment Tools to Support Instruction in STEM

Abstract

Objectives
Instructors must know students' thinking and learning progress to make informed instructional decisions. To achieve this goal, performance-based learning tasks are developed and used in classrooms to aid in teachers’ acquisition of this knowledge. However, because considerable time and effort are needed to grade students' responses, teachers can rarely promptly solicit student thinking on these activities in a timely fashion (Zhai, 2021a). Therefore, this study aims to explore how to design a system to assist teachers in making timely instructional decisions using artificial intelligence (AI). We ask this question: What are the principles that can effectively guide the design of AI-augmented instructional system?
Theoretical Framework
Machine learning (ML) denotes the ability of computers to learn from “experience” and then creatively solve problems like human beings (Linn et al., 2014; Nehm & Haertig, 2012). ML has been broadly applied in automatic scoring to target more complex constructs, improve assessment functionality, and ease human scoring burden (Zhai et al., 2020a). To meet the three goals, AI-augmented applications have to be designed according to theories in assessments, learning sciences, and pedagogy.
Methods and Materials
In this study, experts in science education, assessments, and computer scientists developed the AI-SCORER, an AI-augmented instructional platform that can provide timely information about student performance on complex tasks.
Figure 5 shows the architecture of the system: mobile interfaces for teachers and students provide the information required to perform specific tasks with a personalized view for individual students. Cloud service for data backup and inferencing is set up and integrated to establish consistency in the system. Rapid response evaluation functionality is implemented in the student portal so that one can quickly reflect on and correct responses.
----------------------------------------------Figure 5--------------------------------------------
Results
In the paper, we will present six design principles with examples. Representation: Visual information provided to users is sufficient to perform the learning process. The need to avoid extraneous data in digital teaching is highlighted at this point, focusing solely on the presentation of essential information. Personalization: Use of a friendly and familiar mode of expression as well as an effective pedagogical agent to aid the learning process. Consistency: Display reliably information, and users receive the same results for the same action irrespective of the time of trial on mobile kinds and operating systems, such as Android or iOS. Reflection: The advantages of aiding users as they compare predictions and outcomes, as well as at other stages in the learning process, are suggested by the Reflection Principle. It also encourages students to think about how they responded to various types of inquiries. Administrative Support: Provide administrative and procedural support so that teachers are at ease from the cognitive load. Evaluation: Formative and summative evaluations of the system look at the usability, feasibility, and accessibility of the entire learning system, including teachers’ contributions.
Significance
This study generates knowledge about design principles for AI-based applications to support usability, accessibility, and feasibility, providing evidence for the potentials of AI in the classroom.

Authors

• Ehsan Latif, University of Georgia
  Presenting Author

• Xiaoming Zhai, University of Georgia
  Presenting Author