The Moral Economy of Quantification Devices: Factoring Class Inequality Into School Valuation in Chile

• In Event: Promising Scholarship in Education: Dissertation Fellows and Their Research

Abstract

Based on content analyses of legislation, policy documents, congressional debates, session minutes from regulatory agencies, and in-depth interviews with education stakeholders, I investigate the morally charged use of statistical devices in the implementation of the Education Quality Assurance Law in Chile. In particular, I follow policy specialist into the policy lab, delving into the process of design and enactment of the so-called School Ordinalization Methodology (SOM), a high-stakes assessment technology set out to hold providers (schools) responsible to consumers (students) and thus perform socially optimal market outcomes (e.g., that all schools provide quality education). I build on the sociology of quantification and valuation to make sense of a controversy that emerged among policymakers when devising the best method to commensurate the quality of all Chilean K-12 schools with a single ordinal metric: Should the SOM establish official categorizations of school quality on the basis of “raw” test-scores as “objective” measure of schools’ compliance with achievement standards? Or should it base quality valuations on the school’s relative contribution to these achievements, net of the entering endowments of the students admitted? Underlying this technical question was a moral dilemma, one that pervades the use of benchmarking rules for high-stake accountability purposes: to what extent is it morally fair to value schools for outcomes that may be only arguably or partially attributable to their action?

My analysis reveals that progressive and conservative technocrats in charge of operationalizing the legal mandate that students’ socioeconomic characteristic be factored into school assessments clashed over two opposed conceptions of “quality” and “fairness”. While the former foregrounded schools’ relative effectiveness and invoked justice in relation to providers (schools) for the sake of a fair distribution of responsibility for educating “socially diverse” students, the latter, less suspicious of proficiency rankings, conceived justice on behalf of consumers (students), understanding quality irrespective of the school’s relative contribution. Unable to address this dilemma openly, however, technocrats reduced it to the selection of the statistical model that yields ranking distributions that better fit their moral preference, taking a vote between ordinary least squares and contextualized hierarchical linear models.
I argue that the way experts aligned in this vote epitomizes the surreptitious ideological dispute over the role of rankings in the legitimation of the actually existing, class-segregated market order. But I also contend that, notwithstanding this controversy, it was not enough to splinter a more fundamental consensus, one which allowed experts to get on with the SOM. Although their moral judgments on the distributive consequences associated with alternative ordinalizations diverged, they ultimately shared an econometric style of reasoning and doing that served as second-order “moral background”. In the end, experts’ translation of incommensurable economies of worth into competing claims for fairness that, nevertheless, were both rooted in shared ways of going about settling their moral disputes, helped bridge the ideological gap between a priori antithetical conceptions of market regulation and accountability.

Author

• Gabriel Chouhy, University of Pittsburgh