Does Motivational Instruction Affect College Students’ Performance on Low-Stakes Assessment? An Experimental Study

• In Event: The Impact of Test-Taking Motivation and Test Consequences on the Validity of Test Score Inferences

Abstract

Students’ motivation during low-stakes assessments has been of concern in higher education. The purpose of this study was to test the effect of a practical motivational strategy that institutions can use to enhance test-taking motivation and improve the validity of test scores. Three research questions were addressed:

How does a motivational instruction affect students’ test-taking motivation?
How does a motivational instruction affect test scores?
Are there differences in item response time between students in the motivation and control groups?

Research has examined the relationship between test-taking motivation and test performance. Few studies have included an experimental design to randomly assign students to manipulated motivation conditions (e.g., Braun, Kirsch, & Yamamoto, 2011; Kim & McLean, 1995; Liu, Bridgeman, & Adler, 2012; O’Neil, Sugrue, & Baker, 1995/1996). Consistent conclusions have been generated from prior research: (a) there is a significant association between test-taking motivation and test scores, (b) students with higher motivation perform better on low-stakes tests, and (c) strategies designed to improve motivation vary in term of effectiveness. One area that warrants further research is to identify strategies institutions can employ when administering low-stakes assessments.

For the current study, the ETS Proficiency Profile (EPP), a college-level test assessing critical thinking, reading, writing, and mathematics, was used to investigate the effect of a motivational instruction. EPP was administered to 136 seniors (101 females and 35 males) at five college campuses. Students were randomly assigned to a control or motivation condition. In the control condition, students received these instructions:

“You are about to take the ETS Proficiency Profile. The test takes about 2 hours. Your score on this test will have no effect on your grades or academic standing, but we do encourage you to try your best.”

Students in the motivation condition received an instruction designed to enhance test-taking motivation:

“You are about to take the ETS Proficiency Profile. The test takes about 2 hours. Your score on this test may be used in aggregate to evaluate the quality of instruction at xxx (the name of the institution). It may also affect how xxx (the name of the institution) compares to other institutions nationally. The ranking of xxx (the name of the institution) in the comparison may affect the value of your diploma. We strongly encourage you to try your best on this test, regardless of how well you think you can perform, for the sake of xxx’s (the name of the institution) national standing.”

After completing the EPP, students completed the 10-question Student Opinion Survey (SOS, Thelk et al., 2009), a widely used measure of test-taking motivation.

The motivation instruction had a significant and practical impact on motivation (p<0.01; d=0.87). The motivation group also performed significantly better than the control group on EPP, in terms of total score (p<0.01; d=0.63) and subscale scores (d=0.53 for critical thinking, 0.72 for reading, 0.60 for writing, 0.58 for mathematics). The motivation group spent more time on items compared to control students (49 seconds on an item, vs. 34 seconds; p<.001, d=0.76).

Authors

• Ou Lydia Liu, ETS

• Joseph A. Rios, Educational Testing Service

• Victor Borden