Between compliance and resistance: Enacting a quality culture of evidence in colombian higher education

• In Event: University governance through a comparative lens

Proposal

a) • How is the topic relevant to CIES 2024?

The research presented to CIES is situated within the field of evaluation and quality assurance in higher education institutions. Specifically, the findings of this study enable a critical analysis of the forms of domination and resistance to the idea of a university education based on accountability and guided by measurable and comparable outcomes among individuals and populations.

In this sense, this proposal aligns with several of the subthemes, particularly with number 2 (Curriculum and protest), as it discusses how the commitment to educational change can run counter to discourses of compliance and quality. In other words, we question how the discourse of continuous improvement, quality standards, and accountability truly creates conditions for an inclusive and accessible education in a country with poverty and inequality gaps.

b) • Is there a strong theoretical framework that uses theories to guide questions? Which theories?

This study aims to explore the implications of fostering a culture of quality in higher education institutions, in terms of administrative, curricular, and pedagogical practices. By "culture of quality," we refer to institutional governance systems aligned with principles such as self-assessment, results-based planning, and continuous improvement, embedded in contemporary notions of the new public management. The empirical research of the project is based on a case study design in two higher education institutions in Colombia, with the objective of understanding and characterizing their efforts to implement guidelines of an external policy (Decree 1330 of 2019) aimed at strengthening institutional self-assessment logics.

This study is supported by two approaches with methodological implications in the educational field. On one hand, critical realism is a paradigm that allows understanding the generative mechanisms underlying a culture of quality. Inspired by this paradigm, realist evaluation is conceived as theory-driven evaluation, which involves formulating mid-range theories about specific social mechanisms, which are then tested in the study context. Aligned with the methodological tradition of Realist Evaluation, these theories stem from two sources: a realist review of the literature on quality assurance (Bendermacher, et al. (2019)) and consultations with experts situated in the studied institutional context.

Similarly, the policy enactment framework (Ball et al. 2012) serves as an analytical framework for studying the implementation of Decree 1330 in Colombia. This approach acknowledges the need to step away from linear perspectives that conceive of educational policy implementation as strictly top-down and linear. In this context, this study employs policy enactment to address the question of creative translation and mediation through artifacts, which help in understanding the ways in which compliance and resistance to the implementation of Decree 1330 manifest.

c) • How are sources of information used to inform choices about data collection and analysis?

Based on the two proposed analytical frameworks (realist evaluation and policy enactment), several assumptions guide both the selection of actors and data collection. On one hand, the actors are experts on the phenomenon and not just informants. In the process of refining the theories about mechanisms associated with the implementation of the decree that researchers construct, these actors redirect and refine the understandings (hypotheses) put forth. On the other hand, a socio-material perspective like the one posed by policy enactment leads to considering policy artifacts – such as institutional documents on quality assurance – as susceptible to analysis as part of the mechanisms that bring the idea of a culture of evidence into practice.

Within this same notion, policy artifacts are not merely human creations but also actants – following Latour – that exert influence in generating the analyzed mechanisms. Therefore, in this research, this sociology of translations becomes part of the sources that can be analyzed from a comparative standpoint.

d) • How do the research methods and results support the conclusions drawn from the data?

At the time of submitting this proposal, the research team is still in the process of collecting data at the two universities where the study is being conducted. Currently, five causal hypotheses have been identified – based on realist evaluation – which are being refined to explain how the implementation of Decree 1330 shapes a culture of evidence. Following the logic of realist evaluation, two relevant cases have been identified to understand the mechanisms associated with the implementation of this higher education quality policy. For each case, a trajectory has been identified that facilitates this analysis, starting from the managerial level, extending to curriculum committees, and involving university faculty members who are the ones implementing this policy on the ground. It is with this population that it becomes possible to refine, expand, and ultimately test the causal hypotheses initially proposed.

e) • How original is the contribution? What do we learn that we did not know, and why is it important?

In the field of comparative education, especially within the Latin American context, there are very few contributions from a realist perspective and based on the analysis of educational policy enactment (Paz-Gómez et al., 2023). In this study, we aim to offer an innovative approach by combining two relatively unexplored frameworks within both national and international contexts. This approach not only provides insights for comparison with other tertiary education systems, but also addresses questions about the effects of the discourse surrounding quality assurance, accountability based on learning outcomes, and governance at a time when contemporary universities are undergoing transformation and facing crises. The work we are presenting will thus unveil tensions, forms of compliance, and resistance in the face of hegemonic and globalizing discourses that jeopardize the critical capacity of institutions that are expected to carry out this role.

Authors

• Gary Alberto Cifuentes, University of Los Andes

• Juan David Parra, Universidad del Norte, Colombia

• Carmen Lucia Monsalvo Amaris, Universidad del Norte, Colombia