Latin America after PISA: A Comparative Study of Educational Policy in Seven Countries (2000-2015)

• In Event: Highlighted session: Critical reflections on the development, use and policy impact of international large-scale assessments in education

Proposal

The creation of the PISA assessment in the year 2000 started a new era in international education. I coordinated a large cross-national study with a team of researchers in seven Latin American countries (Argentina, Brazil, Chile, Colombia, México, Peru and Uruguay) to study this era until the year 2015, when the last PISA edition took place. It´s a period of time in which not only PISA and the new century started, but also political and economical conditions where part of a new cycle in the region. This period of economic growth and turning point towards left-wing new democracies set the context for many educational reforms.

Our study had three main questions: (1) What where the main educational policies implemented in these seven countries during the period 2000-2015? (2) What where the educational results in terms of access, equity and quality of education, measured by multiple sources? (3) What kind of hypothesis can we formulate to explain those different results in terms of the contexts and the educational policies implemented by each country?

The methodology had three main components: (1) A large matrix to analyze the educational policies implemented during the period and many other contextual and historical characteristics of each educational system. This matrix was completed by a local researcher or team of researchers in each country. (2) A field study with more than 140 interviews in total (more than 15 in each country) with educational policy experts and actors (ministers of education, local authorities, union leaders, etc.), (3) A thorough analysis of all available quantitative evidence from primary and secondary sources to systematize the educational results in terms of access to education, equity and quality. Every standardized assessment at the international level (PISA and UNESCO tests mainly) and at each national level was analyzed and compared.

The main results of the study show that the period was very intense in educational reforms and the ideological direction of the policies was not linear. Many governments implemented policies that changed the status quo of teachers´ careers, expanded schools´ autonomy and set new incentives. We analyzed the period as part of a historical swift from the nineties. The decentralization process of the vast majority of Latin American countries implemented in the nineties saw a new wave of re-centralization during the first decade of the new century. There were two fundamental educational policy devices to conduct this new regulatory process: the distribution of textbooks, as part of a more active curricular regulation, and the growth of high-stakes assessments that had some consequences for schools, such as rankings or economic incentives.

Chile, Brazil and Mexico, with differences among them in many dimensions, share this re-centralization process, based on new curricular regulation by textbooks and assessments. Colombia only went for the results-based policies, adding pressure from assessments devices but with large curricular freedom. Peru was somewhere in the middle, implementing some new forms of curricular regulation. Argentina and Uruguay, however, where more reluctant or incoherent to intervene and more conservative in terms of reforms.

This basic panorama shows a bit of the cross-cultural large-scale study conducted. The mapping of policies allowed us to understand different periods of time (differentiated in three stages), different convergences among countries (with ideological contradictions in some cases) and particular scenarios of each country that can only be understood appealing to historical analysis.

Stepping in the second part of the study, the results were very complex to analyze. We found contradictory evidence from different educational assessment. Although they were design to measure different conceptions of educational quality, in some cases the national assessments showed results that largely contradicted those of international surveys such as PISA. We undertook serious analysis of each assessment to find out in which cases there was some sort of methodological problem. When the last PISA edition was published in 2016 (the PISA edition 2015), we study the intertemporal validity of the test and found some striking problems, that shows us how difficult is to measure the educational quality of an educational system.

However, we use each source of data to create a complete profile of each country. This part of the study shows that some countries have a large improvement both in educational access and students´ graduation rates (such as Chile, Colombia, Brazil and Peru). We also found that some countries had improvements in educational quality measured by different standardized assessments. Chile became the country (among the seven analyzed) with better results in the OECD and the UNESCO tests. Peru had a major improvement in international and national tests. Other countries showed improvements, but they were more fragile and we had to revise some results when we learned about some methodological problems, specially in the PISA 2015 edition, which is no more comparable with the 2000 and 2003 editions.

The study closes with some hypothesis about the nature of educational policy in the Latin American context. No linear or obsessive ideological perspectives help us understand how that educational policy affect the complexities of a whole educational system. We can learn from some improvements. Clearly there are many hypothesis to support the case of Chile´s improvement, even though is disputed in many ways. The institutionalization of many educational policies and the constant wave of reforms are part of the evolution of a system that puts more pressure (for good and for bad) to the schools, teachers and students to learn.

But apart from this broad analysis, the study had another major goal: to describe the complexities and paradoxes of educational policy agendas in Latin America. We can learn more from the many stories of particular educational policy trajectories, frustrations, changes, negotiations and historical elements present in each context, that the large simplistic PISA style comfort of educational results.

Author

• Axel Rivas, Universidad de San Andrés