Data-Driven Improvement and Accountability

• In Event: Beyond the Rhetoric: Examining the Economic Rationales and the Evidence-Based Support for Current Reform Strategies

Abstract

This presentation offers a critical examination of data-driven improvement and accountability (DDIA): the policies and practices concerning the use of data to inform school improvement strategies and to provide information for accountability. DDIA can lead either to greater quality and equity, or to deterioration of services and distraction from core purposes. Hargreaves and Braun (2013) address the focal question: What factors and forces can lead DDIA to generate more positive and fewer negative outcomes in relation to both improvement and accountability?
Although educational accountability is meant to contribute to improved student learning, there are often tensions and even conflicts between improvement and accountability. Drawing on data from business and sports, as well as education, the presentation identifies five key factors that influence the success or failure of DDIA systems in public education:
(i) The nature and scope of the data employed by the improvement and accountability systems;
(ii) The types of indicators used to track progress or to make comparisons among schools and districts;
(iii) The interactions between the improvement and accountability systems;
(iv) The kinds of consequences attached to high and low performance and how those consequences are distributed;
(v) The culture and context of data use .
In general, we argue that over more than two decades, through accumulating statewide initiatives in DDIA and then in the successive Federal initiatives, DDIA in the U.S. has come to exert increasingly adverse effects on public education, because high-stakes and high-threat accountability, rather than improvement and accountability together, have become the prime drivers of educational change.
Contrary to the practices of countries with high performance on international assessments, and of high performing organizations in business and sports, DDIA in the U.S. has been skewed towards accountability over improvement. It has focused on what is easily measured rather than on what is educationally valued. The high-stakes, high-pressure environment of educational accountability, in which arbitrary numerical targets are hierarchically imposed, has led to extensive gaming and continuing disruptions of the system, with unacceptable consequences for the learning and achievement of the most disadvantaged students. Instead of being informed by the evidence, educators become driven to distraction by narrowly defined data that compel them to analyze grids, dashboards, and spreadsheets in order to bring about short-term improvements in results.
The presentation concludes with twelve recommendations for establishing more effective systems and processes of data-driven or evidence-informed improvement and accountability. The significance of the research presented is that, drawing on data and experiences from different sectors and different countries, it offers a credible and coherent prescription for resolving the tensions between improvement and accountability. These are most likely to be resolved when there is collaborative involvement in data collection and analysis, collective responsibility for improvement, and a consensus that the indicators involved in DDIA are accurate and fair. When these conditions are absent, improvement efforts and outcomes-based accountability can work at cross-purposes, to the detriment of all.
Hargreaves, A. & Braun, H. (2013). Data-driven Improvement and Accountability. Boulder, CO: National Education Policy Center.

Authors

• Andrew Hargreaves, Boston College

• Henry I. Braun, Boston College