Integrating Instructional Strategies With Automatically Generated Assessment Reports to Support Teacher Timely Instructional Decisions (Poster 7)

• In Event: AI and Formative Assessment: The Train Has Left the Station

Abstract

Objectives
We develop instructional strategies to support teachers’ timely instructional decisions with automatically generated assessment reports (AutoRs, Latif et al., in press) aligned with the Next Generation Science Standards (NGSS Lead States, 2013). A panel of expert teachers provided feedback on the developed instructional strategies for assessment tasks. Three teachers implemented our platform and feedback on using instructional strategies with AutoRs to support their instructional decisions. This study explores: what are the critical features for developing instructional strategies based on expert teachers’ feedback? How do teachers use instructional strategies with AutoRs to support timely instructional decisions?
Theoretical Framework
We structured our instructional strategies into three categories: content-general, content-specific, and inclusive strategies. The general strategies originate from constructive and interactive perspectives of learning activities (Chi, 2009). The content-specific strategies (Magnusson et al., 1999) concentrate on each three-dimensional dimensions (disciplinary core ideas, scientific and engineering practices, and crosscutting concepts) to assist individual students or groups. Inclusive strategies capitalize on students' interests and backgrounds to engage them more profoundly and sustain their learning (NRC, 2012).
Methods
Procedures. We followed a design model (Shin et al., 2023) to develop four types of instructional strategies: firsthand, secondhand, simulation, and multimodal experiences for each task (He et al., 2023). Each instructional strategy is content-specific, inclusive, constructive, and interactive, engaging students with diverse backgrounds. This study developed instructional strategies (see Table 8) for three assessment tasks (i.e., analyzing and interpreting data, constructing explanations, and using models, see an example in Figure 3). We conducted semi-structured interviews to elicit expert teachers’ feedback. Thematic analysis (Braun & Clarke, 2006) was used to find patterns across teachers’ responses.
Data sources. We collected teacher feedback from six teachers on instructional strategies and three teachers in implementing our platform. The school districts represent urban (2), rural (1), and suburban (3) communities. The student demographics span ethnic backgrounds. The six teachers are lead teachers with extensive NGSS teaching and learning. Whereas two are experienced teachers, and one is less experience in the implementing study. Open coding was used to qualitatively analyze the interview transcripts (Strauss, 1987).
Results
Regarding RQ1, we found that the instructional strategies follow four critical features: a) goal orientation – teachers viewed the learning goals as the compass to guide the selection of instructional strategies; b) feasibility – teachers considered hands-on experiences (e.g., manipulation) are vital for students; c) cultural relevance – teachers deemed contextualization necessary with relevant phenomena; d) inclusion and fairness – teachers believed it essential to cater to language, learning modalities, and accessibility. Regarding RQ2, the results of three implemented teachers showed that the designed instructional strategies can assist teachers with limited knowledge and teaching experience in making instructional decisions. Their prior instructions and experience influence their decisions. They tend to select instructional strategies they did not use in their prior instructions.
Scholarly significance
Our study shows the critical features of instructional strategies to support teacher timely decisions. Teachers’ feedback on using instructional strategies with AutoRs will expand the knowledge of AI-assisted evidence-based instructional decision-making.

Authors

• Peng He, Washington State University

• Namsoo Shin, Michigan State University

• Mao-Ren Zeng, National Taiwan Normal University

• Katherine Joy Nilsen, WestEd

• David McKinney, WestEd

• Christopher J. Harris, WestEd

• Joseph S. Krajcik, Michigan State University

• Xinyu He, University of Georgia

• Xiaoming Zhai, University of Georgia