When Data Use Efforts Don't Succeed: Factors Hindering the Work of Data Teams

• In Event: 47.070 - Collaborative Data Use: Why, How, and When Does It (Not) Work?

Abstract

Objective
A preponderance of research has focused on identifying what can make data use work more effectively. There is less attention to circumstances in which data use doesn’t become a meaningful part of school improvement. Meanwhile, learning from failures is just as important as learning from successes (Stoll & Myers, 1998; Stringfield, 1998). In this paper, we will use data gathered in middle schools in two countries to provide an in depth look at instances in which data teams do not contribute positively to school improvement.

Perspective
Studies show that characteristics of successful data teams include:
• having learning conversations with a high depth of inquiry (Schildkamp, Poortman, & Handelzalts, 2016)
• applying internal attribution (i.e., educators use data to look at their own functioning) (Schildkamp et al., 2016)
• using multiple forms of data to create a portrait of student achievement (Datnow & Park, 2018)
• using data change teachers’ and school leaders’ thinking and/or actions in the classroom and in the school (Farley-Ripple, May, Karpyn, Tilley, & McDonough, 2018)
• focusing on data use for school improvement (Datnow & Park, 2018).

Several factors can contribute to data teams being more or less effective in their functioning. These include school organizational characteristics (e.g., leadership), teacher characteristics (e.g., data literacy), and data characteristics (e.g., access to data) (Schildkamp & Kuiper, 2010). However, focusing on the presence or absence of certain characteristics may be overly simplistic. This paper highlights the nuanced way in which data use efforts unfold.

Method and data sources
Case study methods were used to examine two struggling data teams in middle schools in the Netherlands and the US. Both were part of broader research studies. In the Netherlands, the data team consisted of one school leader, four mathematics teachers, and one internal data expert. In the US, the data team consisted of seven math teachers, led by a school administrator. The data team meetings in the Netherlands and US were observed (n=17), and the data team members were interviewed (n=29). Interview and observational data from the cases were compared and contrasted in the process of analysis.

Results
Similar factors in both the Dutch and US case hindered the work of the data teams. These include: A lack of time, a lack of trust between school leaders and teachers, and data use focused on accountability instead of improvement. Some hindering factors seem to be more context specific. For example, in the Dutch case a lack of ownership over the problem and a focus on achievement instead of skills, hindered the work of the team. In the US, for example, an overly prescribed process for the examination of data and inconsistencies in district priorities around assessment and data use, hindered the work of the team. It is important to address these constraints to turn unsuccessful data teams in successful ones.

Significance
Learning why, how, and when data use efforts do not go smoothly provides an important contribution to the research and practice literature in the field of data use and school improvement more broadly.

Authors

• Kim Schildkamp, Universiteit Twente

• Amanda L. Datnow, University of California - San Diego