Using technology to open the black box of the classroom to Government – sustainable implementation of Tangerine:Tutor at scale in Sierra Leone

• In Event: Educational technology and faculty development

Proposal

Introduction
In Sierra Leone, human resources to support schools within the Government system are severely limited. This means that schools and teachers are often isolated and very rarely receive supportive visits which limits their ability to professionally develop. A further consequence of this is that decision makers at district and national level know very little about day-to-day activities in the classroom, with data from schools limited to one-off data collection exercises such as the Annual School Census or ad hoc reporting from limited numbers of schools.
What was needed was an approach to support teachers in the immediate term and also a mechanism to open up the black box of the classroom by providing better information on classrooms and teaching for decision makers to use to prioritise performance improvements over the medium term. Cambridge Education, with support from RTI, are implementing an approach that aims to achieve both of these things, using Tangerine:Tutor technology across all non-private Junior and Senior secondary schools in the country.

Mobilising immediate support for teachers
With human resources to support schools so limited, there was a need to develop a cadre of government-aligned but project-employed staff to provide immediate support to teachers. One hundred and twenty-five school support officers (SSOs) were rapidly recruited and deployed in September 2017. These SSOs were tasked with visiting schools regularly, observing lessons and providing coaching feedback to improve teaching and learning in English and Maths. They were also responsible for feeding back on key indicators to staff at district level to provide some basic regular information on activities in classrooms.
What became clear was that these coaches also needed regular support to help them improve and focus on the most important elements of feedback for teachers. In addition, the data collection process was paper-based and time-consuming leading to only minimal amounts of data being aggregated and transferred to decision makers. A better approach was needed.
Use of Tangerine:Tutor to enhance SSO function and improve classroom level data availability across the whole country
Tangerine:Tutor was that better approach. Key to its applicability in this context was that it provides offline-first data collection functionality, is intuitive to use and easy to programme. The coaching automated feedback function also provides immediate tangible benefit to coaches like SSOs. For these reasons Cambridge Education began working with RTI to digitise existing paper-based forms and also add some additional features, like pupil attendance tracking. The tool was piloted in 2 districts (one urban and one rural) with 26 SSOs. The success of the pilot was very encouraging. In just 6 weeks of full implementation over 900 lesson observations were entered into the system, all 26 SSOs used it successfully and the vast majority of comments recorded in pilot log books and end-of-pilot interviews were positive. As soon as data started to come in Cambridge Education and RTI worked to rapidly develop a prototype ‘live dashboard’ to enable district and national government decision makers to view the information coming from classrooms in real-time via simple online indicator dashboards.

Institutionalised and sustainable use
The success of the pilot intervention led to two decisions. Firstly, the decision was taken to immediately move to the newly developed 3rd Generation version of Tangerine to benefit from additional features such as remote updates and integrated lesson plans to further enhance SSOs support for teachers. The second decision was to map out a pathway to institutionalisation and sustainability of the system in the medium term.
This pathway starts with achieving national coverage across a network of now 175 SSOs reaching all junior and senior secondary schools. This rollout started in September 2017. Concurrent with this is the engagement of district and central government on the development of the first nationwide coverage live dashboards. These will be developed by the end of 2018.
Plans for medium term institutionalisation and sustainability include a phase 2 national rollout to district supervisors in all districts. This is now a more sustainable option given the Ministry has recently announced it is hiring an additional 160 supervisors and revising their job description to incorporate many functions of the SSO. Supervisors support primary schools as well as secondary, so this phase of rollout will extend benefits of Tangerine:Tutor to primary schools. Phase 3 of the approach will see Tangerine:Tutor deployed to all secondary school Principals (over 1,200) to help provide daily feedback to teachers within schools and enable daily data transmission from every school to agreed national dashboards.
The risk now is that there is more data than decision-makers know what to do with and so efforts are underway to build the capacity of decision makers at all levels to understand the data and how to use it for evidence-based planning and performance review. The need for evidence-based planning is particularly critical for the government given its recent introduction of a free quality school education policy. Conversations have also already begun about how to integrate data from Tangerine into the national EMIS system and how to transfer ownership of the backend of Tangerine:Tutor in Sierra Leone.
There is still much to be done, but achieving national coverage and having pathway to sustainable and institutionalised use mapped out within 8 months is an encouraging start.

Authors

• Kayode Sanni, Cambridge Education

• Daniel Waistell, Cambridge Education

• Matt Finholt-Daniel, RTI International / University of Minnesota