What Data Do Practitioners Use and Why? Evidence From Germany Comparing Schools in Different Contexts

• In Event: Data Use in Education: Alluring Attributes and Productive Processes

Abstract

Poor performance in international student assessment has led to calls to enhance evidence-based practice in the German educational system. Subsequently, a new form of interplay between accountability and school autonomy has been advocated, and several instruments of a standards-based school reform have been implemented, such as statewide exams or school inspections.


Grounding actions and decisions in evidence is considered to be a prerequisite for effective practitioner performance, and an increase in student achievement (Honig & Coburn, 2008). However, accountability is still comparably low in Germany and indications about the extent to which German practitioners actually use data is limited. Little is also known about factors influencing data-driven school improvement (e.g., school organizational characteristics). Considering the expenses arising from extensive monitoring systems, proof of the usefulness of data is key. Research has focused mainly on the effects of single instruments such as school inspections, whereas comparative examinations of different information sources have failed to appear.

As research on data use seems to be rather “disorganized” (Coburn & Turner, 2011, p. 175), an inconsistency of concepts and terms can be observed. Therefore, we tried to conceptualize ‘data’ or ‘evidence’ as a first step, understanding these terms in a rather broad way.

Using data from three different mixed-methods studies that were carried out in the German states of North Rhine-Westphalia and Rhineland-Palatinate respectively, we examined the perceived usefulness and the application of 13 different sources of information that can inform practitioners’ practice by means of standardized questionnaires.


In deepening interview studies, we also examined reasons for and against the use of evidence in schools. Altogether, data from 1,040 questionnaires (911 teachers, 129 school leaders) and 59 interviews (44 teachers, 15 school leaders) were included in the analyses. The interviews were transcribed and analyzed by means of qualitative content analysis (Mayring, 2007). The coding system was developed from the interview guidelines and was supplemented by categories that emerged when analyzing the empirical data. To reveal a potential influence of the school context on evidence-based school improvement, we compared data use practices in schools located in deprived areas to data use in schools in more favorable conditions.


The results showed that practitioners attributed little usefulness to a standards-based reform and consequently hardly used these data. Instead, they claimed to prefer process-oriented information sources, such as student feedback. A comparison of the different samples indicated that data use might be lower in schools in challenging circumstances. In face-to-face interviews, a considerable proportion of the interviewees mentioned little data use due to a lack of time. Furthermore, problems in recontextualizing evidence and adapting it to personal needs were mentioned. Some practitioners also stated shortcomings in the quality and validity of the data, which was particularly true for findings from school inspections and standardized testing of students.

The analyses provide findings regarding the degree to which evidence-based decision-making is implemented in schools. They also hint at factors fostering or hindering practitioners’ data use and ways to strengthen the use of evidence in schools (e.g., support, capacity building, administration/university-school-partnerships).

Authors

• Denise Demski, Ruhr-Universität Bochum

• Kathrin Racherbäumer, University Duisburg - Essen