Exploring the Paradoxical Relation Between Achievement and Self-Beliefs: A Country-Level Analysis of TIMSS Trend Data

• In Event: Combining International Student Assessments for Country-Level Longitudinal Analyses

Abstract

Does positive academic self-concept and other self-beliefs play an important role in academic success? While achievement and self-concept are positively correlated within countries, the association is negative across countries (Marsh et al., 2018; Seaton et al., 2009) The finding that students in high performing countries report on average low academic self- concept has been replicated in several studies and is frequently referred to as the attitude achievement paradox. In the present study, we argue that the attitude achievement paradox is a methodological artifact related to a cultural response style. Cultural response style is defined as the tendency that students in different countries or cultural context use response scales differently regardless of the item content (e.g., Min et al., 2016; He & Van de Vijver, 2016).
Our study aims to investigate if the negative country-level correlation between
academic self-beliefs and achievement vanishes when using longitudinal data. We argue that the cultural response style is stable over time, and for this reason, it is possible to circumvent response style bias by studying the relationship between change in achievement and change in self-beliefs longitudinally. For this purpose, we combine data from different years to construct a dataset with a longitudinal component on a country level.
To explore the relationship between self-beliefs and achievement, we use 8th grade student data from all cycles (1995, 1999, 2003, 2007, 2011, 2015, 2019) of the Trends In International Mathematics And Science Study (TIMSS). We observe the relationship between achievement scores and three Likert-type items that reflect different aspects of self-beliefs and that were administrated in all previous TIMSS cycles: enjoyment in learning the subject, self- perception of doing well in the subject, and the need of doing well in the subject to enter the university of choice. The analysis were replicated for two subject-domains, namely mathematics and science. Using ordinary least squares (OLS) regression with fixed effects for countries and years, we study the relation between academic self-beliefs and achievement on a country level. We present four models in each analysis. We begin with OLS and build upon this model using a fixed effect estimation framework to account for unobserved characteristics across different dimensions. Our final data base for the country-level analysis consists of 64 countries with up to 6 cycles (in total 224 units of country-level data).
As a baseline for our main analyses, we estimate the cross-sectional association between self-beliefs and achievement and observe the paradoxical negative relationships for all included variables in mathematics and some in science. After adding fixed effects for years and countries, the negative relationships vanished. This holds for mathematics and science in 8th grade students.
The study contributes to the academic field in two ways. First, we make use of the rich amount of data collected over 20 years around the world, which allows us to look at within- and between-country variation. Second, we suggest an alternative approach to address the “attitude achievement paradox”. Our study illustrates how researchers and policymakers can use trend data to address, for example, biases that adhere to country-level characteristics.

Authors

• Mojca Rozman, IEA Hamburg

• Rolf Strietholt, IEA Hamburg