Teacher accountability policy and sociocultural context across countries: A mixed-methods study focussing on Finland and Singapore

• In Event: Dissertation workshop session
  In Special Session: Policy Analysis

Proposal

University: University of Cambridge, United Kingdom.
Dissertation faculty advisor: Panayiotis Antoniou.

The objective of this research project is to develop policy-relevant insights about the extent to which effective teacher accountability policy is contingent on sociocultural context. Accordingly, my overarching research question is: how does national sociocultural context affect the efficacy of teacher accountability instruments in promoting desirable student outcomes? Based on my preliminary data analysis, I argue that effective teacher accountability instruments must be compatible with national sociocultural context. This argument pushes back against the 'best practices' discourse in education, thus adding further evidence to the case for considering not only what works, but also for whom and in what circumstances. Besides contentious academic and popular debates about best practices as well as accountability in education, another motivating force for this dissertation was my experience of being a teacher in a high-need Malaysian secondary school; where the collision of exam-based rankings, extensive paperwork, and a cultural emphasis on face-saving resulted in falsified reports, demotivated teachers, and disappointing student outcomes.

I approach this cross-country study with a critical realist stance. This stance integrates a realist ontology (i.e. the assumption that the phenomena under investigation exist independently of any mental representations thereof) with a constructivist epistemology (i.e. the assumption that my investigation of these phenomena is only one of countless possible and inherently limited perspectives) (Maxwell 2012). Critical realism thus accords with my aim of developing practicable insights for policy, while respecting the complexity and contingency of culture, society, and policy implementation. My conceptual framework draws on Pawson and Tilley's (1997) realist approach to policy evaluation, which conceptualises Outcomes as resulting from generative Mechanisms embedded in complex Contexts. Accordingly, this research project examines how teacher accountability instruments interact with national-level sociocultural context to influence teacher-level conduct—via mechanisms such as mobilising stakeholder attention or allocating formal consequences—and, consequently, to affect student-level outcomes.

To answer the overarching research question, I use two different sets of data, each addressing a different sub-question. First, I use multi-country educational data from PISA (2012 and 2015), TIMSS (2015), and TALIS (2013) alongside sociocultural data from the World/European Values Survey and Hofstede's IBM study to address the sub-question: To what extent does national sociocultural context influence the relationships between teacher accountability instruments, teacher motivation, and student outcomes, as measured in cross-country surveys? I analyse these datasets using a series of multilevel models. Each model combines (a) educational data from one assessment cycle and (b) one of six country-level sociocultural constructs that have been identified in the education policy literature as being salient to accountability. Each model also includes an interaction term for teacher accountability instruments and the sociocultural construct in question. The number of countries included in each model ranges from 24 to 64, depending on the overlap in country participation between the datasets. I find evidence that some aspects of sociocultural context (e.g. social trust) do, in fact, moderate the relationship between teacher accountability and student outcomes and, to a lesser extent, the relationship between teacher accountability and teacher job satisfaction.

Having established these bird's-eye-view connections between teacher accountability, sociocultural context, and student outcomes, I turn to semi-structured interviews with teachers to gain a more granular understanding of one pathway through which sociocultural context interacts with teacher accountability instruments. The sub-question for this portion of the research project is: Within the Finnish and Singaporean education systems, how does national sociocultural context influence teachers' responses to teacher accountability instruments? Finland and Singapore were chosen because they have good education systems with contrasting approaches to teacher accountability, within national contexts that differ socioculturally but are comparable in other key ways (e.g. similarly sized populations with some ethnolinguistic diversity but a clear majority group, with major education reforms in 1970s and 80s). I conducted interviews with twelve teachers in Finland and twelve teachers in Singapore. Interview questions were designed to allow subsequent analysis to: (a) test my overall hypothesis (i.e. that teacher accountability instruments should be compatible with sociocultural context); (b) identify the full range of teacher accountability instruments on the ground; (c) map teacher accountability instruments onto mechanisms for influencing teachers' conduct; (d) build a richer understanding of relevant aspects of sociocultural context; and (e) assess the validity of alternative hypotheses.

Currently, I have completed (but not written up) the statistical analysis, and I recently conducted my field interviews in Finland and Singapore and am in the process of transcribing them, after which I will analyse the interview data and proceed to the writing stage.

Author

• Yue-Yi Hwa, University of Cambridge, United Kingdom