Impact of School Planning Using Education Data Platform on Students' Learning Outcomes in Indonesia

• In Event: Education improvements: interventions and impacts

Proposal

To what extent does providing data about school conditions to schools improve students' learning outcomes? Educational Report Card is a school-level technological platform providing data on schools' conditions and achievements based on national assessments. Schools may use the platform's data to pinpoint school problems, identify the solutions, and improve educational quality. In other words, data from the Education Report Card may be used by schools to create data-driven improvement plans. The research is grounded in the Data-Driven Decision-Making theory, in which knowing more about the school conditions can help the principal allocate resources more optimally, ultimately increasing student achievement. This research is intended to empirically prove this theory with the case in Indonesia. Data-driven decision-making in literature has been found to be impactful in improving the performance of institutions, with solid evidence from corporations (Brynjolfsson et al., 2011). In the education sector, school accountability policy, which consists of giving information about school achievements as well as putting pressure on schools to meet specific performance, has been found to make schools use the accountability data to reduce the achievement gap and improve school members' professional capacity in engaging with data. However, at the same time, the high-stakes accountability created a narrower school curriculum. It incentivized the school to focus on the best-performing students, possibly worsening the inequality of learning (Brill et al., 2018). Thus, the combination of data availability and high-stakes accountability may create more complex results. Nevertheless, what if the schools only get data availability without pressure from the government, which is the case in Indonesia? Since the platform's first publication in 2022, did this program really improve student learning outcomes? This research aims to explore this question and evaluate the impact of evidence-based planning using Educational Report Card. A difference-in-difference approach is used to examine this, exploiting the difference between regions with high and low numbers of schools downloading the Educational Report Card. In this approach, the progressions of student learning outcomes from 2022 (publication of the first educational report card) to 2023 in regions with a high rate of schools downloading educational report cards are compared with the low-intensity regions. This approach eliminates the other confounding variables influencing learning outcomes because of the difference in regional and time characteristics. We used data on elementary school student achievements, inequality level, learning quality, and teachers' learning habits from the educational report card in 50 regions randomly chosen over 514 regions. The student achievements are measured by literacy and numeracy testing of randomly chosen students in all schools of Indonesia, which is also used to measure the level of learning inequality. Meanwhile, learning quality and teachers' learning habits are measured from student, teacher, and principal surveys. The report card's download rate is also used to determine the treatment and control regions. The study finds that regions with higher download rates of educational report cards have higher literacy and numeracy development one year after the policy implementation than regions with lower download rates. It also finds no change in literacy or numeracy inequality index due to educational report cards. These inequality indices consist of gender, socioeconomic status, and rural-urban. The fact that the report card impacts learning outcomes but does not worsen learning inequality implies that students with low socioeconomic status, girls, and students in rural areas reap the optimal benefit from this technological platform. Moreover, we find that there is an improvement in the learning quality due to the educational report card, which consists of class conductivity, cognitive activation level, and affective support level. Nevertheless, there has been no significant change in the teacher's learning habits or reflection index. The fact that school planning with education report cards impacts learning quality but not teachers' learning habits means that teachers' learning habits are relatively low and need to be improved to achieve more optimal learning outcomes. Overall, this result proves that educational report cards positively impact student learning outcomes without any significant adverse effect on learning equality and that this impact is mediated by improving learning quality in the class, not the level of teacher reflection and learning habits. This research confirms that scaled-up technological intervention in the form of giving information on school achievements in a digital platform is beneficial to improve school quality in developing country contexts like Indonesia. Relative to the widespread accountability reform, this intervention has lower costs due to the absence of government pressure and constant monitoring, but it is able to impact learning outcomes without increasing inequality, which is the unintended consequence of accountability policies. The results also point out the area of improvement for the policy, that is, how to improve teachers' learning habits with Educational Report Card, as that aspect is also related to student learning outcomes. These results are important for informing educational policy regarding the Educational Report Card so that the government can optimize resource allocation and create more targeted interventions. This intervention can also be a reference for other developing countries regarding the provision of data using digital technology for school decision-making. This research contributes to scarce literature on data-driven decision-making in school contexts in developing countries. It provides a novel insight into the impact of data availability without high-stakes accountability, which is relatively underexplored in the literature. Also, it contributes to school improvement literature, technology implementation in an educational context, and school management.

Authors

• Faris Hafizh Makarim, TEACHERS COLLEGE COLUMBIA UNIVERSITY

• Santoso Hadisucipto, Article 33 Indonesia