Assessment of Future History Teachers' Narrative Competence With Open Text Formats: Methods and Empirical Results

• In Event: International Perspectives on the Assessment of Historical Thinking

Abstract

The objective of this presentation is to present a methodological approach for the assessment of narrative competence with open text formats and to report empirical results from a study with future history teachers.

Theoretical framework
Storytelling about the past follows the rules of narration for creating historical sense (Ruesen, 2005). Thus, there exists a certain consensus that teachers and students have to go beyond the facts presented in historical narratives (e.g., VanSledright, 2014). They have to analyze the assumptions on which the narratives were created for “deconstructing history” and they should be able to create historical narratives following the standards and rules of historians. These two main processes of historical thinking are called “re-constructing” and “de-constructing competencies” in the framework of Historical Thinking (Koerber, 2014). The present study investigates narrative competence including both processes.

Method
Data collection took place in four universities of teacher education in Switzerland. The final sample included 153 student teachers in their second academic year (53.6 % female, 26.4 years in average). Two thematic test booklets were developed broaching the issues of Swiss emigration to Brazil in the 19th century and the Cold War. Although the first topic is rarely taught in Swiss high schools, the latter topic usually is included in the curriculum. Both test booklets followed the same structure including a short introduction into the topic and two primary sources that illuminated different aspects of the historical issue and a writing prompt. The two booklets were randomly assigned.

The quality of texts was rated by three experts using a high-inference rating approach assessing the argumentative quality, as well as accuracy (i.e. contextualization, the use of references and citations to support source-based evidence, dealing with the perspective of the two sources (including corroboration), consistency of substantive arguments, clarity of the historical concepts, justification of normative orientation) on a four-point Likert Scale (1 = not applied, 4 = fully applied). A sufficiently high interrater reliability was achieved (> .70 generalizability score). T-tests were then used to analyze group differences.

Results
15% of the texts contained a brief assessment of the chosen theme coupled with a single normative historical testimony, but there was no historical narrative. 85% of texts represented a historical narrative. They were rated rather low regarding the use of references, the clarity of historical concepts as well as the justification of normative orientation (1.5 to 2.8 mean average). Comparing the two topics, contextualization, consistency of arguments and the elaboration of perspectives were rated higher with the Cold War. Overall, the quality of the present texts was certainly heterogeneous.

Conclusion
The present texts do not allow insight into intermediate steps of de- and re-construction, they only show the final result. However, some indicators of narrative competence could be traced. The rating method used, which focused on argumentation and accuracy, complements previous Anglo-American rating categories, which tend to focus mainly on argumentation (e.g., Monte Sano & De La Paz, 2012). The empirical findings point to an excessive demand to foster narrative competences of future history teachers during high school and their first year of teacher education.

Authors

• Monika Waldis, University of Teacher Education Northwestern Switzerland

• Martin Nitsche, University of Applied Sciences of Northwestern Switzerland

• Philipp Marti