Measuring the effects of cross-national assessments on media discourse

• In Event: Roundtable Session 11
  In Refereed Roundtable Session: The challenges and impacts of assessment

Proposal

Over the past few decades, both the quantity and spread of cross-national assessments (CNAs) have grown tremendously. One reason for the growth of CNAs is that they can encourage education reform through identifying weaknesses and highlighting high and low performing countries (OECD, 2019). Germany, for example, is a commonly cited case of “PISA shock” in which poor performance on PISA in 2000 spurned great media discourse within Germany about its failing education system, leading to many radical reforms (Neuman et.al., 2010; Tillman et. al. 2008). Yet, aside from Germany’s compelling example and a few others such as Spain and Japan, many other consistently poor performing countries do not see the same uptick in reforms, bringing into question whether CNAs actually have an effect on the reform process despite the OECD and IEA’s assertions (OECD, 2019; IEA, 2024). To better understand this ambiguous relationship, I explore the mechanisms through which CNAs can lead to reform. Specifically, I am interested in the effect CNAs have on domestic media discourse, and media’s subsequent effect on the reform process. I explore the effect of CNAs on media discourse using a novel dataset which collects news articles across 140 countries in 40 languages. This dataset improves upon previous media data collection methods which often utilize either only English or have more limited time frames. I theorize that in the case of Germany and other “PISA shock” countries, they experienced a high increase in media discourse about education following their low results, leading to increased pressure on the government to enact education reforms. However, in many other poor performing countries, they may not experience a similar increase in media discourse, potentially because of already low expectations of performance. Thus, lacking the pressure of the media, the government is not as incentivized to increase the rate of reform despite poor performance on CNAs. Through this dataset, I further elucidate the mechanisms through which CNAs can affect the reform process and contribute to the ongoing conversation on the role of CNAs in education.

Author

• Marcia Yang, Stanford University