Adapting and piloting a classroom observation tool in a resource-poor context: a participatory approach

• In Event: Exploring Teacher Effectiveness, Technology Integration, and Equity Across Global Contexts

Proposal

The benefits of digital approaches in education are well documented and have many applications in schools. However, such approaches are often not feasible in resource-poor contexts, such as post-conflict areas, many of which may not have stable access to electricity. In these contexts, educators rely on offline tools, particularly classroom observation tools (COT). Our study examined how a COT can capture aspects of playful learning to improve teaching for enhanced student learning outcomes in a post-conflict context in Burundi. The tool was used to research a programme that uses play as the key teaching approach, and our adaptation of the COT aimed to capture the class dynamics and interactions between teachers and students.

Our research contributes to the literature on the use of classroom observation as a cost-effective way of monitoring and improving teaching quality (Akyeampong et al. 2023), and of supporting teacher professional development for improving teaching practice (White et al. 2021). In addition, our study provides new insights into how a universal tool can be adapted to suit specific programme needs, as well as how the involvement of observers and teachers being observed can improve these adaptations.

In recent years, a small number of open-source COT have been developed which can be adapted for use in low- and middle-income countries, for example, the World Bank Teach tool. However, little is known about how countries and interventions adapt COT for use in their education systems (GPE, 2022). Furthermore, while some organisations refer to COT adaptation for initiatives in their education programmes (e.g. World Vision, 2019), the process of alignment to programme needs and the COT itself is not always documented and available. Very little is known about how lesson observers and teachers being observed are involved in the process of adaptation and contextualisation of COTs, and this is key, given the obvious role of teachers in learning.

To enhance the adaptation and functionality of the tool, we opted for a qualitative approach to gather thorough and in-depth insights. Engaging local education stakeholders, such as teachers and programme staff, in a consultative process was fundamental to our study to ensure that the tool we had adapted was effectively aligned to the non-profit’s programme and was best placed to support teacher professional development. We started with a desk review of COT to assess which elements of other tools could inform the adaptation and whether those elements were appropriate for a resource-poor context. Girls’ participation in school and students’ socio-emotional wellbeing are strong areas of focus in the programme. We therefore focused on gender equality and socio-emotional learning, as well as how teacher and student behaviours are documented by lesson observers, as these aspects of the existing COT were identified as being underdeveloped. The desk review also included familiarisation with the programme documents to support the contextualisation of the adaptation. We then adapted the COT for review by programme staff on the format, further contextualisation and wording.

Next, we organised a workshop with local education officials and short interviews with headteachers and teachers in the schools where we piloted the COT to gather everyone’s feedback and provide opportunities for an open discussion. We asked these stakeholders to focus particularly on the wording of the tool, to inform both contextualisation and clarification. We were inspired by the cognitive interviewing approach, which focuses on whether participants interpret the wording of research instruments as intended (Willis and Ardino, 2013). This was important to help ensure the COT was adapted to capture contextually relevant and useful insights into teachers’ classroom practice.

We found that all education stakeholders shared useful insights on the appropriateness of the questions; the viability of observer expectations, for instance in large class sizes of over 50 pupils; information on classroom; and school context. There were some practical suggestions for the improvement of the COT, including a reduction in length and some changes in the wording of the scoring instructions for clarity. Teachers and headteachers expressed that the tool needed to be able to capture the demanding reality of their classroom experiences, especially given the large class sizes in their schools. They were hopeful that the feedback from classroom observation could help them improve their practice and provide practical guidance. All stakeholders highlighted that the measurement tools needed to consider the challenges of working in low-resource settings, for example, regarding physical infrastructure and safety. These insights guided further tool adaptations as well as the development of an accompanying user guide to support observers in the use of the COT.
A key learning from our study was that investment in resources for reviewing tools with programme staff and education stakeholders is needed in a participatory approach. This includes access to local translators with an understanding of the local education landscape. Furthermore, the availability of stakeholders needs consideration to ensure they have the opportunity to effectively share their input and feedback. Another learning which emerged from study participants’ feedback was that the COT needed to be more concise. This feedback informed further adaptations which focused more closely on capturing teacher-learner interactions and the quality of the learning environment, since these are more important for students’ wellbeing in a post-conflict context than, for example, monitoring student time spent on learning activities.
The study insights will be of interest to audiences working to improve learning environments and outcomes through teacher professional development initiatives in low-resource contexts, where digital options are limited or not feasible. This includes organisations that implement programmes where classroom observation could serve as an important source of information, when appropriately aligned with the local context. Crucially, the participatory approach we used in our study and the subsequent findings underscore the importance of working with education practitioners to develop tools that are practical, relevant and reflective of real classroom dynamics and contexts.

Author

• Geeta Gambhir, National Foundation for Educational Research