Leveraging Student Interactivity Data to Inform Instruction and Monitor Learning Progress

• In Event: From Fixed Test to Endless Data Systems: Analyzing and Communicating Ongoing Formative Assessment

Abstract

Accurate and timely assessment of student progress is important for several reasons: Informative monitoring can capture at risk students and direct appropriate interventions (Mazza and Dimitrova, 2004). It can reveal which study behaviors are most effective. At a course level, aggregating student activity can reveal class-level performance. These factors, in turn, can expose instructional design and be used to modify teacher and institutional effectiveness. Collectively, these reports can be useful to sponsoring agencies that need efficient and accurate measures of learning approaches and progress.

As important as student progression data is, it is extraordinarily difficult to communicate and interpret because of the number of variables associated with learning. The types of activity—such as homework, test, or quiz—as well as the variability of the objectives in the subject domain have to be evaluated across time and instructional contexts. We have no grammar or algebraic means to capture and communicate this multivariate space.

We turned to Exploratory Data Analysis (EDA) to address these concerns in evaluating student progressions in complex educational systems (Tukey, 1977; Behrens et. al. 2012). With student interaction data across 250 courses sharing the same course outline, we created a standard graphical template to compare the courses and student behavior. By applying computational designs as proposed by Schneiderman (1996), we can display a semester’s interaction data that immediately reveals broad course-level interaction and performance data and instructional designs. For example, Figure 2 shows the homework student interactions for a course. We see at a glance that Chapter 4 was skipped and that as student progress through a chapter, they exercise the problems at the end of the chapter for only a few days and then move onto the next chapter without any review.

Contrastingly, Figure 3 demonstrates a self-paced course with pieces of Chapters 4 and 6, and all of Chapter 7 ignored. Additionally, through interactive filtering we can also evaluate specific students and/or assignments within the broader context. In summary, using student interaction data from a higher education mathematical homework and test delivery platform, we demonstrate the various types of information that can be simultaneously revealed and inspected surrounding individual students’ learning progressions, overall class activity, as well as instructional design.

Authors

• Thomas McTavish, Pearson

• Johann Ari Larusson, Pearson