“We want whatever would bring progress to our children”: School leaders’ experiences in generating and using evidence to improve learning

• In Event: Learning at all levels: Building schools’ capacity to generate and use relevant evidence to strengthen instruction practices

Proposal

Evidence is key in crafting appropriate and effective education policies, processes, and practices. At the school level, administrators and teachers need access to high-quality, relevant evidence to develop and apply teaching and learning practices that improve learning outcomes.
To advance this area of work, our study explored the evidence needs, practices, and experiences of schools across 7 countries (Ghana, Cote d’Ivoire, Uganda, Nigeria, Kyrgyzstan, Switzerland, Colombia). We interviewed 19 school leaders to learn about schools’ data collection and evidence generation and use practices, the data and evidence schools want, schools’ use of technology, and schools’ engagement with larger education organizations around evidence. We complemented these school-level conversations with survey data from 20 education network organizations.
As anticipated, the study found that schools’ practices varied widely in a) generating data, b) generating evidence (by analyzing raw data), and c) using evidence.
All schools recorded basic data, such as attendance records and term-end assessments, often through non-digital (pen and paper) means. Fewer schools went further with their data collection (for example, by collecting more frequent assessment data or social-emotional data), and those that did so tended to use digital data collection methods. Schools took a variety of steps to generate evidence from those raw data, including by conducting analyses to 1) identify outcomes that were below expectations at the student, class, or cohort level, and 2) identify subject matter results below expectations. School administrators also described how they or their staff then used that evidence at the individual, class, school, and community level. For example, principals reported teachers contacting families about absences, and schools and communities working together to enact community bylaws to promote attendance. Using more complex learning data, schools can adjust whole-class level of instruction, form ability groups and offer tailored instruction to each group, or develop individualized instruction or support plans.
Across low-, middle- and high-resource settings, school leaders expressed a desire to generate more evidence and use evidence more effectively, regardless of where they were on the spectrum of evidence generation and use. However, schools varied widely in the areas of evidence practice where they needed additional support. For example, in lower resource settings, school leaders expressed a desire to automate basic assessments. Some—but not all—school leaders in low-resource settings signaled a hope to visualize their data using technology, while most leaders in middle- and high-income settings actually hoped to reduce the presence of technology in the classroom. Both groups of school leaders cited a need for more hands-on supports and teacher training on data collection and analysis. In higher income settings, school leaders also wanted ways to connect their data to other sources, including social-emotional and longitudinal data.
These insights suggest that the organizations supporting schools—including local districts, network organizations, and central government ministries—should work with schools to identify their specific needs and goals related to evidence, and craft differentiated policies and programs to support varied schools in deploying evidence to maximize impacts on learning outcomes.

Author

• Josh R Meuth Alldredge, Mathematica