Designing a Teacher Dashboard to Support Multilingual Learners' Understanding of Science (Poster 5)

• In Event: Artificial Intelligence and the Future of STEM Instruction: Designing New Models to Automate Feedback to Teachers

Abstract

With advances in learning analytics and automated scoring, modeling technologies can be powerful meaning-making resources for multilingual learners (MLs) who speak a language other than English at home. These technologies can immediately collect and analyze specific ideas presented in student work to provide real-time, individualized feedback (Gobert et al., 2013; Linn et al., 2014; van Joolingen et al., 2015). Such automated feedback in modeling tools can support MLs’ sense-making process of complex scientific phenomena and engage them in scientific discourse (Negrut & Ryoo, 2021; Ryoo et al., 2019).
However, implementing such tools in classrooms can be challenging as teachers may not have access to student performance data to identify struggling learners and modify their instruction (Aleven et al., 2016; van Leeuwen et al., 2019). Even when data is visualized through learning analytics dashboards, most existing systems focus on overall scores or completeness of activities, rather than providing contextual indicators about students’ learning processes among different language groups (Abel & Evans, 2013; Dickler et al., 2021).
To address this gap in the literature, this study explores how middle school science teachers co-designed teacher dashboard prototypes that leveraged student data collected from their linguistically diverse classrooms. As part of a larger design-based research project, eight eighth-grade science teachers from four Title I middle schools participated in professional development (PD) sessions to co-design modeling tools that provided automated feedback based on specific target concepts and alternative ideas. After implementing the tools in their linguistically diverse classrooms, each teacher participated in semi-structured interviews to reflect on their implementations and discuss initial suggestions about the dashboard features. Six of these teachers engaged in additional design meetings to review prototypes and discuss which analytics should be visualized for the dashboard.
The results from the PD sessions and implementation interviews showed that all teachers reported the positive effects of automated feedback in improving MLs’ understanding of unobservable scientific processes. However, the teachers noticed that some students did not pay attention to or had difficulties interpreting feedback, which often led to trial-and-error behaviors. To guide these students to make more productive revisions, the teachers suggested ways to view student progress data, such as the types of feedback received and scores per revision. Based on the findings, teacher dashboard prototypes were created.
As the teachers reviewed the prototypes, all participants emphasized the importance of being able to see specific misconceptions that students had over time to plan mini-lessons for the next day and identify struggling students who may need individual guidance. In particular, the teachers valued the ability to review and compare those misconceptions among language groups, such as current MLs and former/current English Learners (ELs), to identify misconceptions associated with language barriers and adapt their teaching methods to help MLs use more precise language.
This study extends the current literature by providing insight into the design needs of science teachers for learning analytics dashboards to better support MLs’ understanding of abstract scientific phenomena. The findings of this study can inform future dashboard designs for linguistically diverse students.

Authors

• Kihyun Kelly Ryoo, University of North Carolina - Chapel Hill

• Emily Marie Decker, University of North Carolina - Chapel Hill