Dealing With Performance-Based Accountability: A Conjoint Analysis of School Leaders' Strategies to Improve Test Results

• In Event: Enacting Accountabilities in Education (Part 1): Its Effects on School Practices and Outcomes

Abstract

Objectives
The goal of this investigation is to assess which types of pedagogical and organizational strategies are preferred by school leaders to improve test results, and to explore how different contextual factors are related with different strategical preferences.

Perspectives
In the last decades, most countries in the world have introduced accountability mechanisms in their education systems, often relying on performance-based instruments such as national standardized tests. In some contexts, material incentives and sanctions for schools and school actors are linked to test results. In other ones, bad test results are associated with tighter forms of school supervision and reputational consequences. In PBA systems obtaining good results at national standardized tests thus becomes a central goal. Because of this, it has been argued that PBA increasingly changes educational practices at the school level (Lingard et al., 2013) and eventually favor the emergence of undesired behaviors, potentially threatening educational quality (Berliner, 2011; Booher-Jennings, 2005; Ohemeng & McCall-Thomas, 2013).
Few studies have until now focused on both formal and informal strategies adopted by school leaders to improve results when dealing with national standardized tests (Jennings, 2010). Such strategies range from those related to the use of data from assessments for instructional improvement, to changes in the curriculum, in the staff management and in the way aspiring students are selected.
Two main challenges are related to the study of these strategies. Firstly, as they can coexist, it is difficult to assess the relative importance given to each strategy. Secondly, as some of these strategies are considered as undesired effects of PBA, responses may be affected by social desirability bias when people are asked directly. New approaches are therefore needed to address these challenges.

Methods and Data Sources
A novel survey-based conjoint experiment is used. Introduced in the Seventies in marketing research (Green & Rao, 1971), this design has been recently used in other disciplines because of its numerous advantages in taking into account for multidimentional preferences and in dealing with potential social desirability biases (e.g. Hainmueller et al., 2014; Bansak et al., 2019). The data come from the REFORMED project principal survey conducted among school leaders (n. 235) in a sample of Chilean schools.

Findings
The results show how school leaders particularly prefer strategies that imply changes in the educational focus and in the management of human resources. In particular, the curriculum alignment to the competences tested and the offer of human development opportunities to the teachers whose students have obtained low results in the national tests seems to be the most valued strategies to raise national tests’ scores. Differences in the strategical preferences between school leaders working in different contexts can be also appreciated.

Significance
The paper makes a methodological contribution to educational research, an area where experiments are still underused. Particularly, it shows how survey experiments can allow to overcome the drawbacks of direct questioning and of observational studies and how they can serve to explore the undesired effects of educational accountability mechanisms.

Authors

• Antonina Levatino, Universitat Autònoma de Barcelona

• Gerard Ferrer-Esteban, Universitat Oberta de Catalunya

• Antoni Verger, Universitat Autònoma de Barcelona