Chilean Teachers Under Test-Based Accountability Pressures: Experimental Insights Into Opportunistic Behaviors

• In Event: Survey Experiments in Educational Research: International Perspectives on an Innovative Research Methodology

Abstract

Objectives
We analyze to what extent the test-based accountability (TBA), with particular attention to the consequences it implies, triggers the adoption of opportunistic behaviors by teachers. Afterwards, we explore the extent to which reporting opportunistic behaviors is subject to social desirability bias.

Perspective
Opportunistic behaviors are often associated to TBA, because of the high stakes that can be at play for the accountable actors (Berliner, 2011). However, recent research argues that accountability systems with lower consequences have the potential to generate similar effects to those of high-stakes systems (Barbana, 2016; Thiel et al., 2017). Existing studies on opportunistic behaviors in education have not been able to establish what type of consequences (material/reputational, individual/collective) are more conducive to undesired behaviors yet. This is due to the fact that existing research usually involves studies with a small sample that explore a specific context, and do not have a control group to carry out a counterfactual analysis (Jones and Egley, 2014). Furthermore, addressing this phenomenon is methodologically challenging, since it is likely that school actors, for social desirability reasons, are not willing to report undesired behaviors. With the purpose of overcoming the limits of existing works, this research adopts an experimental approach.

Methods
The experiment used is a vignette, where each participant, randomly assigned to a group, reads a vignette that describes the situation of a teacher to whom a colleague advises cheating in the context of a new national assessment, as a way to obtain better results in the test. The vignette is manipulated with regard to the possible consequences that the test may have if students obtain poor results. After the vignette, the respondent rates both the likelihood of the protagonist of the vignette to cheat and the likelihood of doing so if he/she personally were in the same situation.

Data source
The experiment was embedded in a survey (n=1250) conducted in a representative sample of Chilean schools located in urban zones.

Results
The results show that the respondents who have visualized individual consequences and collective reputational consequences believe that the protagonist is more likely to cheat with respect to respondents who have visualized the vignette where the test did not have any consequences. Collective monetary consequences do not have a statistically significant effect. The comparison of the means of the answers to the question asked in third person and in first person shows how the means are significantly different (p <0.01), confirming how the social desirability bias is an important issue when studying these kinds of behaviors.

Significance of the study
This research provides new evidence on the mechanisms that trigger the adoption of opportunistic behaviors. The results indicate that the reputational consequences play a central role in the TBA systems, challenging the assumption that undesired effects are mainly attached to high stakes. The results also show that reporting this type of behavior may be subject to a social desirability bias, confirming the appropriateness of adopting experimental approaches to investigate topics which are difficult to capture using conventional research designs.

Authors

• Antonina Levatino, Universitat Autònoma de Barcelona

• Lluís Parcerisa, Autonomous University of Barcelona

• Antoni Verger, Universitat Autònoma de Barcelona