"Drowning in Data": A District's Efforts to Develop Data Literacy at Scale

• In Event: Implementation and Effects of Standards-Based Reform: A Five-Year, Five-State Study of the Multiple Levels and Dimensions of Ambitious Instructional Reforms

Abstract

This paper reports on a case study of one suburban district in Ohio and the role that data played in shaping how state-level standards were implemented. We trace how the district, following the state’s data-focused Ohio Improvement Plan, organized professional development to help educators access and use student-level data to drive decision-making. We focus both on what worked for stakeholders in such a data-driven policy environment and the challenges they reported facing when navigating what they called the ‘deluge’ of available data. We examine the district’s assumption (based on the state’s framework) that big data would lead to better decision-making across classroom, school, and district scales – and, its influence on student learning and achievement.

To understand the role of data in district policy, we draw on two models of data literacy in education. Mandinach and Gummer’s (2016) data literacy framework for teachers outlines specific data skills and practices needed in the current standards-focused policy environment while offering insights into challenges (e.g., conflation of assessment/data literacy; inadequacy of federal, state, and teacher PD guidance). Phillip, Schuler-Brown, and Way’s (2013) critical approach to big data in education broadens such a focus to consider power, race, and the history of data’s uses within education.

This case study of one Ohio suburban school district’s implementation of CCR standards included interviews with 7 district leaders, 4 principals, 31 teachers, and 4 coaches. We also observed 8 lessons in ELA and math classrooms at two elementary and two high schools. We drafted daily memos after concluding interviews and observations. We qualitatively coded all fieldnotes, documents, transcribed interviews, and memos, with rounds of emergent coding of stakeholders’ understandings and theoretical coding using the policy attributes theory and data literacy frameworks.

Findings show (1) the district offered multiple supports for educators to generate and access data (e.g. common assessments; centralized district-level data warehouse) alongside mechanisms for ensuring compliance (e.g. weekly teacher-based-team meetings; mandated PD; teacher-principal reports). However, (2) stakeholders at all scales (district, building, classroom) found the scope and volume of the available data to be overwhelming (“we are drowning in data”) and, thus, difficult to operationalize in instructional decision-making. In this context, (3) educators tended to interpret the district’s data-focus as assessment-oriented, and defaulted to this position in their planning and teaching – contradicting some stakeholders’ stated purposes for data-oriented education. Given the growing emphasis on data-driven instruction in local and federal policy (Coburn & Turner, 2012), such findings elucidate opportunities and frictions that surface as stakeholders across scales negotiate the varied meanings and uses of educational data – and the implications for teaching and learning.

Coburn, C., & Turner, E. (2012). The practice of data use: An introduction. American Journal of Education, 118(2), 99-111.

Mandinach, E. B., & Gummer, E. S. (2016). What does it mean for teachers to be data literate: Laying out the skills, knowledge, and dispositions. Teaching and Teacher Education, 60, 366–376.

Philip, T. M., Schuler-Brown, S., & Way, W. (2013). A framework for learning about big data with mobile technologies for democratic participation: Possibilities, limitations, and unanticipated obstacles. Technology, Knowledge and Learning, 18(3), 103–120.

Authors

• Amy Stornaiuolo, University of Pennsylvania

• T. Philip Nichols, Baylor University