The Effect of the Time When the Test Is Taken on Student Performance in Statewide Assessments

• In Event: 41.083 - Roundtable Session 15
  In Roundtable Session: 41.083-17 - Roundtable: Examining Construct-Irrelevant Factors Impacting Test Score Interpretations in Statewide Assessments

Abstract

Administration of Arizona’s new AzMERIT online tests are untimed and schools may flexibly schedule students to take the tests in computer labs during the testing window. Thus, students in the same grade or taking the same end of course (EOC) test within same school are not required to test on the same day. This is a marked departure from Arizona’s administration of AIMS in which students statewide were administered tests on the same date and time, ensuring that any effects of test administration time or day were held constant.
Because the days and times on which test administrations can be administered is variable, the possibility arises that performance factors associated with time of day or day of week may influence student test scores. Because the online test delivery system records time and date information for all test administrations, we can examine time of day and day of week effects on student performance.
A series of regression models were developed to predict student performance using the day of the week and time of the day variables, as well as the duration of the test administration from test start to test end. The dependent variable for these analyses was the spring 2016 AzMERIT scale score. To control for student achievement, we first covaried previous achievement using AzMERIT scale scores from spring 2015 administration. Because of the need to covary previous achievement, the analyses reported here are limited to the grade 4 to 8 and high school EOC assessments in mathematics and ELA tests. The day of the week was coded as 1 to 5 with 1 for Monday, 2 for Tuesday, and so on. The time of day and the duration were continuous variables using the actual time. The grade 4 ELA regression model (R2=0.69) shows that the day of the week when the test was completed, the time of the day when the test was completed and the duration of the test were all highly significant predictors of student performance. Assessments completed later in the day and later in the week were associated with higher test scores. In addition, long test administrations were associated with lower performance. .
To further examine these effects, a series of ANOVA analyses were conducted to evaluate specific contrasts between test administration start and end times and days. The start day contrasts for grade 4 ELA show that starting tests on Friday yield lower test scores than starting on any other day of the week, while tests that end on any other day than Monday are associated with greater performance. The contrasts further reveal that assessments ending at any other time than early morning are associated with higher performance, and that tests ending late in the afternoon are associated with poorer performance than those ending early afternoon.
Similar pattern was observed in other grades ELA grades and Math grades. The results will be presented in the full paper.

Authors

• Yuan Hong, American Institutes for Research

• Irene Hunting, Arizona Department of Education

• Stephan A. Ahadi, American Institutes for Research