Different Stakes, Different Motivation? Swedish Studies of Test-Taking Motivation in Different Assessment Contexts

• In Event: The Impact of Test-Taking Motivation and Test Consequences on the Validity of Test Score Inferences

Abstract

Are test-takers motivated to do their best, regardless of the characteristics of the test they are taking, or does effort and motivation vary with the stakes of the test? How can the test-taking motivation construct be theoretically understood and modeled? And how is test-taking motivation related to test performance and, thereby, to test score validity? These overarching questions have served as the basis for an ongoing research project whose findings will be summarized in this presentation.

It can be argued that test-taking motivation and other psychological aspects of tests are important to acknowledge and assess, as they may have a construct-irrelevant impact on test performance and, therefore, affect the reliability and validity of test scores. Theoretically, one would assume test-taking motivation would be higher – and thereby less of a threat to score validity - for tests where the results have consequences for the individual test-taker (high-stakes tests) than for tests with no such consequences (low-stakes tests). Previous studies have supported these arguments to some extent, but the testing community would benefit from further empirical substantiation.

With the purpose of better understanding test-taking motivation, theoretically and empirically, a number of studies of test-taking motivation were performed in different Swedish assessment contexts. Likert-type, self-report scales measuring reported test-taking effort, and in some samples also perceived test importance and test anxiety, were administered to multiple Swedish samples (n ≥ 1,000 in each sample) taking large-scale tests such as the Trends in International Mathematics and Science Study (TIMSS/TIMSS Advanced, Grade 8/Grade 12), the Programme for International Student Assessment (PISA, Grade 9), the Swedish National Tests (Grade 9), and the Swedish admission test for higher education (the SweSAT, typically 19-20 year-olds). TIMSS and PISA are low-stakes tests in terms of consequences for the individual test-taker, whereas the national tests and the SweSAT have higher personal stakes attached to them. In all studies, the scales used have shown acceptable psychometric properties; models have shown acceptable fit to data, and the scales used have shown expected relationships with other variables and with performance.

In general, results suggest that tests with higher stakes are associated with higher ratings of test-taking effort and test importance, and also higher ratings of test anxiety, although findings are not entirely consistent and there are indications that ratings of test-taking motivation on low-stakes tests may be related to age and experience with testing. When regressed on test score, test-taking motivation (in terms of reported test-taking effort) predicts performance in all studied assessment contexts, even when other relevant variables are accounted for. Further, in assessment contexts where the stakes are higher, test anxiety is a somewhat stronger predictor of performance than is test-taking motivation.

In conclusion, we believe this research, although non-experimental and to a large extent exploratory, indicates that test-taking motivation is a construct that is important to consider for a number of reasons, and that it is important to continue monitoring test-taking motivation in different assessment contexts, for validity purposes and in order to further the understanding of test-taking psychology.

Authors

• Hanna Eklof, Umea University

• Eva Knekta, Umeå University