From Accountability to Algorithms: Interorganizational Struggle and the Transformation of Chicago’s Dropout Prevention System

• In Event: Politics, History, and Systems Improvement in 21st Century School Reforms

Abstract

Objectives: In 1999, researchers at the University of Chicago Consortium on School Research found that students’ performance in ninth grade predicted eventual high school graduation. In 2005, these early warning indicators (EWIs) were added to the high school accountability system, and have since been transformed from an accountability metric to a just-in-time algorithm to determine students “off track” to a systems-level tool for school improvement. However, this transformation didn’t just happen naturally. Changes came as various state and non-state organizations worked with and struggled alongside each other. This paper provides a historical analysis of the interorganizational dynamics in an effort to highlight key lessons on how history and context interact with school improvement.

Framework: This paper integrates the sociology of quantification and the sociology of organizational learning. Sociological studies of quantification have often focused on the intended and unintended consequences of rankings, ratings, performance measures, and algorithms. However, various technologies and forms of quantification have changed as in the case of lower stakes for federal test-based school accountability. Thus, this paper focuses on the process of organizations like school systems changing quantitative technologies, with lessons on how historical shifts can lead to such changes.

To understand the change in quantitative technologies, I draw on the literature on organizational learning, especially interorganizational learning. In many organizations, change happens as a consequence of organizational learning as they encode inferences from history into specific routines. Interorganizational learning is the process by which organizations can access, acquire and exploit knowledge developed by others. This study highlights how various organizations work and conflict with each other to transform organizational systems.

Data & Methods: This research draws on 95 interviews from individuals who started, spread, and sustained EWIs over 20 years, majority from Chicago. I then supplemented these interviews with research, news articles, annual reports, and unpublished documents totalling 2880 pages. Data were coded and events were chronologically arranged to provide a fuller picture of historical development of EWIs.

Summary of Findings: State and non-state organizations worked and conflicted with each other to continuously transform EWIs, moving it from school-based evaluative accountability to individual-based predictive algorithm to systems-focused improvement tool. Some insights emerge: (1) The introduction of a new accountability technology was met with resistance by teachers and administrators. (2) However, meaningful changes happened when school leaders were engaged, the teachers union was consulted, and when specific schools started creating initiatives. (3) Schools were unimpressed by peer-reviewed data, but were more likely to take action when they had their own school’s just-in-time data. (4) When teachers learn the gravity of the off-track problem, they move from individual solutions to a systems-focused approach. (5) In the process of schools and nonprofits working alongside each other, they reinforce learning.

Scholarly Significance: This research highlights how the investigation of history and power in the transformation of a school improvement effort can show the intimate struggles inherent in cross-sector partnerships. It also uncovers factors that support productive learning across these relationships.

Author

• Jose Eos Trinidad, University of California - Berkeley