Pathway From NAEP to HiTCH? Measuring Competencies of Historical Thinking With Standardized Instruments

• In Event: International Perspectives on the Assessment of Historical Thinking

Abstract

Objectives
This paper provides evidence for the possibility of measuring historical thinking competencies using standardized instruments that include U.S. history example items.

Theoretical framework
The American “Historical Thinking Standards“ (National Center for History UCLA, 1996), as well as international frameworks of historical thinking (e.g., Seixas & Morton, 2013; Van Drie & Van Boxtel, 2008), and frameworks in Germany (e.g., Körber, 2014) are based on narrative theory (Danto, 1965; Ricoeur, 1983; Rüsen, 2005). In all these frameworks, knowing and doing history are the main goals of history lessons. Knowing history means having substantive knowledge of historical events and dates, being thoughtfully able to use terms like revolution or democracy, as well as conceptual ideas like evidence or significance. Doing history addresses posing and answering rich historical questions and using strategies like searching out evidence or making sense of the perspectives of the sources’ authors (e.g., VanSledright, 2014). Obviously, assessing historical thinking in standardized test formats is a challenge for researchers (Ercikan & Seixas, 2015).

Methods
For the development of a competency test of historical thinking (HiTCH) in Germany, we used, among others, some publicly accessible items from the U.S.’s National Assessment of Educational Progress (NAEP). One open-ended task based on the comparison of two pictures was transformed into a standardized test format. On the way to optimizing the measurement instruments, we conducted three data analyses. In the first step, items with high discrimination were selected based on classical test theory models (due to restrictions of the sample size in the first pilot study), as well as a 2-parameter-logistic model grounded in Item-Response Theory (de Ayala, 2009). In the second and third step, 1-parameter logistic models were used for the scaling model.

Data sources
In the first sample (pilot study, 2013), we tested 1701 students (51.2% female, 59.8% highest school track, age: 14.89 years). 221 students (50.5% female, 64.3% highest school track, age: 14.82 years) filled out the U.S. history test booklet. The second sample (pilot study, 2014) contained 1296 students (52.0.5% female, 64.3% highest school track, age: 14.80 years). 2853 students (50.6% female, 49.1% highest school track, age: 14.42 years) took part in the third sample (validation study, 2014).

Results
The final HiTCH-test had a satisfactory variance of the latent variable (Var(Θ) = 0.88), and a high score-reliability (Weighted Likelihood Estimator (WLE) Person Separation Reliability = 0.91) with an acceptable item fit. The distribution of item difficulty was consistent with personal ability. Out of 15 total tasks, two themes dealt with U.S. history. In one item, students had to contrast two master narratives of North American history (self-development). In the second item, inspired from an open-ended format in NAEP, students critically compared two pictures depicting the settlement of the American West.

Scientific significance
The transformation of open-ended tasks into standardized test formats seems to be an appropriate way for assessing historical thinking—not only factual knowledge—with psychometrically sound instruments.

Authors

• Christiane Barbara Bertram, University of Tübingen

• Wolfgang Wagner, Universitat Tubingen

• Ulrich Trautwein, University of Tubingen