Factors Influencing the Sustainability of a Data Use Intervention

• In Event: Sustainable Professional Development for Data-Informed Instructional Improvement

Abstract

Purpose

Schools often start educational innovations with the ambition to improve the quality of education. Even if this innovation seems effective in the short term, it is not self-evident that the innovation sustains (Gilad-Hai & Somech, 2016). In the Netherlands, a data use intervention has been introduced in over 40 school locations, since 2011 (Author, 2014). However, sustainable implementation of the data use intervention has proven to be a challenge (Author, 2017).
There is increasing recognition that the extent to which innovations are sustained is influenced by many different factors (Author, 2019). More knowledge is needed on what these factors are and how they interact (Wiltsey-Stirrnam & et al., 2012). The purpose of this is study is to gain more insight in the process and influencing factors of sustainability of interventions, such as the data use intervention.

Theoretical framework

The data use intervention embraces a way of working where teams of six to eight teachers and school leaders from the same school participate in a professional development program for one or two years, in which they learn how to systematically use data. The data team intervention has two main goals: 1) professional development in data use and 2) improvement of education. Research has shown that this intervention can help improve education (Author, 2015). Despite the effectiveness, the sustainability of the method varies in the years after the external data team coach left the school (Author, 2019).
Based on a systematic review (Author, 2019), in this study the sustainability of an innovation is defined as ‘integrating and scaling the innovation’s core aspects in organizational routines, which are flexible and adaptive to ongoing work’ and four types of factors influencing sustainability were distinguished:
• School organizational (e.g. collaboration, leadership)
• Innovation (e.g. effectiveness, efficiency)
• Individual stakeholders (e.g. teacher buy-in)
• Context (e.g. acknowledgement formal organization)

Method and Data Sources

The four types of factors formed the basis for a questionnaire for respondents of all levels of the 41 secondary schools (i.e., from district level to teacher level) who worked with the data use intervention. The questionnaire was tested and a confirmative factor analysis was conducted, and reliability of the scales were established. Subsequently, descriptive and multilevel analyses were conducted to study the sustainability of the innovation for data use and it’s influencing factors.

Results

Preliminary results show that school organizational characteristics, such as leadership, seem to be particularly important to sustain educational interventions. In addition context characteristics, such as formal acknowledgement from the school district, influence the sustainability of the intervention. In our presentation we will discuss this further and also reveal the effect sizes of the different variables, and which (combination of) variables seem to matter most.

Educational importance

Continuous improvement instead of embracing one new project after another, saves teachers’ energy and resources. Sustainable data use can lead to better insights in education for students and can help in focusing on improving student achievement. School leadership and policy makers have key roles in ensuring the sustainability of (data use) interventions.

Authors

• Anne Tappel, University of Twente

• Cindy Louise Poortman, University of Twente

• Kim Schildkamp, Universiteit Twente

• Adrie Visscher, Universiteit Twente