A Natural Language Processing–Driven Teacher Dashboard to Support Responsive Instruction

• In Event: A.I. and the Future of STEM Instruction: Designing New Models to Automate Feedback to Teachers

Abstract

The Next Generation Science Standards call for instruction that guides students to integrate disciplinary core ideas (DCI), cross-cutting concepts (CCCs) and science practices (SEPs). Responsive instruction supports students to integrate their ideas by recognizing students’ reasoning, helping students distinguish the evidence that led to their view with evidence from instruction, and forming connections (Linn & Eyon, 2011; Luna, 2018). Teacher dashboards take advantage of advances in learning analytics to support teachers’ responsive instruction. Dashboards have advanced from using multiple-choice assessments to promote recall to assessing and promoting integrated student understanding (Matuk, Tissenbaum, & Schneider, 2019). We describe a Teacher Action Planner (TAP) that uses natural language processing (NLP) to assess students’ science explanations and displays the patterns of understanding in a dashboard (Figure 2). We study whether and how 17 secondary school science teachers used the TAP during remote instruction.
The TAP was designed through a research practice partnership. We developed assessment items that prompted students to write an explanation integrating their ideas related to an NGSS performance expectation (PE). Researchers hand-coded 1,000+ student responses collected from the teachers’ schools to the assessment prompts using rubrics assessing two subscores and knowledge integration (Table 3). Data was used to develop an NLP scoring model (Riordan et al., 2020). The models were embedded into the Web-based Inquiry Science (WISE) units. Teachers taught the units. The TAP was automatically generated in real time, as students submitted their explanations to the embedded assessment (day 5 of 10-day unit) (Figure 2). This study used logged TAP usage data from 17 teachers across 7 public secondary schools; teacher interviews; Zoom classroom observations; and emails, all collected in remote instruction.
Eight of the seventeen teachers (47%) designed an instructional response to address the pattern of student understanding presented by the TAP during WISE instruction (Table 4). All of their responses focused on supporting students to discover key ideas the TAP identified as missing from students’ explanations. Some teachers presented these missing ideas to the whole-class; others prompted their students to revisit specific evidence. Four of these instructional responses also supported students to distinguish among new ideas they gathered and their initial ideas. This was most often done one-on-one or in small groups analyzing selected student explanations. Two of these four teachers also guided students to reformulate the connections among their ideas after adding and distinguishing.
This study suggests that web-based curriculum and dashboards can support responsive instruction during remote teaching. Guiding students to discover ideas that were identified as missing is the most common approach to responsive instruction with a dashboard (Gerard et al., 2020). A few instructors also support students to distinguish among the new ideas explored and their initial ideas. More ways to support students to distinguish ideas are needed. Next steps include using NLP to recognize specific student ideas. Recommendations could then empower teachers to design activities to help students distinguish their generative ideas from other ideas using evidence from the unit. These activities could be personalized using the specific ideas identified for each student.

Authors

• Libby F. Gerard, University of California - Berkeley

• Sarah Bichler, University of California - Berkeley

• Kelly R Billings, University of California - Berkeley

• Allison Bradford, University of California - Berkeley

• Marcia Linn, University of California - Berkeley