Identifying Navigation Behavioral Patterns with Time Warped Longest Common Subsequence on Process Data

• In Event: Advancing Educational Research for All with Large-Scale School Assessment Data: Insights from Innovative Approaches

Abstract

The use of computers as a delivery platform significantly enhances the ability to collect a broader range of records in log files through human-machine interactions (von Davier et al., 2019; Liao et al., 2019). The information extracted from this granular process data has contributed to a better understanding of test-takers’ strategies and behavioral patterns, thereby supporting innovations in new item designs (Ulitzsch et al., 2024; Zhang et al., 2024). This study aims to seek better solutions for clustering action sequences in behavioral data with consideration of time intervals, emphasizing the preservation of temporal information through repetitive actions. We explore three sequence-based methods, the Longest Common Subsequence (LCS; Cormen et al., 2001; He et al., 2021; Sukkarieh et al., 2012;), the Dynamic Time Warping (DTW; He et al., 2023) and the Time Warped Longest Common Subsequence (T-WLCS) that was borrowed from musical retrieval models (Guo & Siegelmann, 2004) on sequential process data and compare their performance in accuracy and interpretability of clustering results. The contribution of this study lies in exploring new methods to handle multi-layered process sequences and demonstrate the effectiveness of T-WLCS in distance measurement and sequence clustering.

We utilized students’ navigation process data in interactive problem-solving and scientific inquiry items (PSI) from TIMSS 2019 to illustrate these two sequential clustering methods and identify those who demonstrate resilience—students who perform well despite coming from low socioeconomic backgrounds. The study seeks to illuminate the distinctive strategies and behaviors underpinning their task completion and success, offering critical insights into the mechanisms of resilience. Based on a sample of 599 students in Norway, it was found that, although the LCS and DTW method have been widely applied in sequence analysis, the T-WLCS method in our study shows better results in detecting behaviors from resilient students. Figure 1 presents the clustering results by first and second principal components. As shown in the three panels, the LCS could not distinctively separate the behavioral patterns as the timing information was not included. The DTW method shows a better membership distinctiveness but the significant differences of sequence lengths divert the similarity measures. The T-WLCS shows the most distinctive clustering results for the behavioral patterns. It was interesting to find that academically resilient students were more likely to exhibit double review patterns to confirm their responses. Additionally, resilient students also displayed more persistent behavioral patterns than their peers, which may cause them to get stuck on challenging items and not easily progress to subsequent items.

Future studies on resilient students could further refine methods for identifying and supporting this unique group. The advanced clustering techniques could also be expanded to explore longitudinal data to better understand how resilience develops over time and across different educational environments. Additionally, studies could investigate the specific interventions or pedagogical approaches that best promote resilience, offering practical guidance for educators and policymakers.

Authors

• Qiwei He, Georgetown University

• Binhui Chen, Georgetown University