Analysis of Facilitators and Barriers in the Use of External Formative Assessment of Pupils' Learning by Elementary School Teachers in Chile

• In Event: The use of assessments for student learning and educational improvement

Proposal

Context and problem

In the context of the Educational Reactivation Plan implemented by the Ministry of Education in Chile to address the effects of the COVID-19 pandemic, the Education Quality Agency (ACE) has made available to schools an external formative evaluation instrument, Comprehensive Learning Diagnosis (CLD). CLD monitors students' learning in the socio-emotional and academic areas at three moments during the school year: initial diagnostic, intermediate at the end of the first semester and closing at the end of the year. Each assessment covers content and skills aligned with the national, mandatory curriculum. CLD is designed to help teachers track progress in the context of the learning loss that resulted from the prolonged school closure in Chile. As in other jurisdictions, during school closure, inequities in access to remote learning increased the socioeconomic gap in students’ achievement (Kuzmanic et al., 2024).

CLD uses technology to make this assessment available online, providing teachers with immediate results that inform attainment levels by each learning goal, class, and individual student. This digital application exemplifies transformations in how teachers can access high-quality external assessments that complement their assessment of pupils. In contrast to another assessment implemented by ACE, the high-stake summative assessment of students’ learning SIMCE, which is widely publicised to inform parents’ choice and target external support programs, only the school leadership team and individual teachers can access CLD results. The report includes predefined templates suggesting how teachers can use the results to identify the learning goals they need to address, define required curriculum planning adjustments, and identify students who require additional academic support Agencia de Calidad, 2022.)

The education system has positively received the CDL. In its first implementation (2020, while schools were using remote instruction), 7,500 schools administered the test, and this number remained high after students returned for place-based instruction (Agencia de Calidad de la Educación, 2022). This positive reception generates high expectations regarding its utility to address learning gaps and monitor progress. The current study aims to examine how this innovation impacts teachers' and school leaders' instructional work.

Research questions

Schildkamp et al. (2017) proposed a conceptual framework identifying three factors influencing whether and how data informs instructional decisions: organisation, teachers, and data characteristics. Using a sequential quantitative, qualitative explanatory design (Creswell, 2013), the current study sought to answer the following questions:
· What are the organisational conditions for teachers’ use of CLD?
· What is the utility that teachers grant to CLD?
· To what extent and how do teachers collaborate to use CLD?

Method

A survey was first administered in 20 schools, and 80 grades 1-4 teachers, the principal, and the head of curriculum participated. This convenient, volunteered sample includes schools that are in different performance categories, as classified by the Education Quality Agency1: High (n=3), Medium (n=8), Medium-Low (n=4), and Insufficient (n=5). Regarding their administrative dependency, 12 are public, and the others are private subsidised schools receiving all funding through an attendance-based state voucher. In 11 grades served include K-8, in all others, K-12th. The enrolment varied between 175 and 995 students, with a median of 456. Regarding the social vulnerability of students' intake, there is a variation between 19% and 88%, with a median of 55% of students below the poverty line. In the second stage, in-depth interviews and observations were conducted in three schools selected based on their responses in the quantitative phase: 2 public and one private subsidised.

Results

Results show a high level of application of the CLD tests in the academic area. The immediate feedback disaggregating by learning goals and students is perceived as highly useful. Almost three out of four teachers stated that the CLD Diagnostic test helped them: a) identify the learning goals to which they should pay more attention in their classes; b) implement specific pedagogical actions with students that the CLD indicates require more support; and c) give feedback to students.

A facilitator for using CLD results is the collaborative work among teachers, their joint work with special education teachers, and guidelines provided by the school's principal and head of curriculum. Collaboration was implemented differently in the three school case studies, and these differences can be explained by how principals configure teachers’ work.

As barriers, teachers identified a lack of resources to attend to students requiring additional support (52%) and that other priorities did not allow them to meet the needs of individual students (81%). In addition, 68% reported that they were teaching the learning objectives of the previous school level, given the extent to which students are lagging. Thus, there was a discrepancy between what was taught and what was assessed by CLD. Additionally, these teachers reported a discrepancy between their differentiated evaluation practices and the lack of differentiation in the COLD assessments:

“So, of course, for us, it is quite complex, and I am not going to say that it is not stressful because it is very stressful. We know that children have to have this [learning goals defined for the grade level], but our reality is this. So, we have to try to work with that and get as close as possible to the standard of what they are asking for [the national curriculum], but we also have to consider all these other things.” (Teacher, Green School)

Discussion and Conclusions

While acknowledging the limitations of this exploratory study, the findings provide valuable insights and clues about possible modifications to the CLD for increased alignment between the tests, the curriculum taught, and assessment practices to differentiate instruction (Mandinach & Schildkamp, 2021; Visscher, 2021). In addition, there is a clear need to provide professional development for teachers and principals for a more systematic use of data to inform pedagogical decision-making. This will impact the tests’ utility and lessen teachers’ stress.

Author

• Carmen Montecinos, Pontificia Universidad Catolica de Valparaíso