Data-Based School Development? Longitudinal Findings on the Relationship Between School Improvement Activities and Student Achievement in Austria

• In Event: Division H VP Session: Past Designs, Present Tensions, Future Paths: Reimagining Educational Assessment Across Contexts

Abstract

In the early years following the publication of international assessments such as PISA, Austria initiated a range of educational reforms aimed at improving quality and reducing disparities in student achievement (Jesacher-Roessler & Kemethofer, 2022). Such reforms aim at providing schools with performance data not only for accountability, but also to inspire quality improvement in education (Altrichter & Maag Merki, 2016), and mediated by that, reduce educational inequalities. However, the question remains: Can data-based quality assurance and development (QAD) activities at schools actually contribute to improving student performance—and thus to more equitable learning outcomes?

This study explores that question by examining the relationship between QAD measures—such as school-based internal and external evaluation practices, and connected with that, professional development measures—and student achievement in standardized tests in mathematics (2012, 2017) and English (2013, 2019). The analysis draws on large-scale data from Austria’s national educational standards assessments (N_students ≈ 300,000; N_principals ≈ 5,500), employing both Cross-Lagged Panel Models and Random Intercept Cross-Lagged Panel Models to test for directional and time-lagged effects.

Findings indicate high stability in average student performance over time but only weak and inconsistent relationships between reported QAD activities and achievement. Interestingly, schools with lower achievement scores tended to report more QAD activity, suggesting reactive rather than proactive engagement with data. From an equity perspective, the study critically questions the assumption that increased QAD intensity at schools automatically translates into improved student outcomes or reduced disparities.

This presentation will place particular emphasis on the potential of data feedback from standardized performance assessment to inform equity-oriented school development. While our initial analysis focused merely on average competency, the presentation will expand on additional analyses of distributional effects as key indicators of educational inequality.

The study adds to international debates on the potential and limitations of data-based school improvement in addressing disparities in student achievement (e.g., Wurster et al., 2017). Ultimately, we argue that the latter requires more than monitoring means: it demands deeper insight into how schools interpret and act on data (in the form of school development activities), and whether these lead to more homogeneous (i.e., less stratified) learning outcomes.

Educational significance: The findings are relevant for policymakers and practitioners seeking to use data as a lever for more equitable education systems. They highlight the need for critical reflection on how data is used: not just for for the sake of accountability, but for inclusive, targeted, and sustainable school development processes that reach all learners.

This contribution examines the intersection of accountability, autonomy, and equity. It showcases how longitudinal, evidence-informed research can help assess whether current governance models truly foster innovative, inclusive, and sustainable learning environments.

Authors

• Jana Groß Ophoff, Thurgau University of Teacher Education

• David Kemethofer, University of Teacher Education Upper Austria

• Christoph Helm, Johannes Kepler University of Linz

• Ramona Zintl, IQS – Federal Institute for Quality Assurance of the Austrian School System

• Martina Ott, University College of Teacher Education Vorarlberg

• Katharina Meusburger, University College of Teacher Education Vorarlberg