Impact Evaluation: PEAS Uganda’s school improvement programme in government schools (Inspect & Improve)

BACKGROUND

In Uganda, there has been a significant increase in access to secondary education since 2007, however even before the Covid-19 pandemic learning outcomes had been declining. While Uganda has made large gains in increasing enrolment at secondary level since Universal Secondary Education was introduced in 2007, education quality has lagged behind the government’s ambitions. Between 2007 and 2014 there was a decrease in the percentage of Senior 2 students rated as proficient in English Language and Maths (NAPE, 2014) The school closures throughout 2020 have led to significant learning losses, and make it more important than ever to understand how schools and school leaders can improve learning gains cost-effectively.

The Ministry of Education and Sports has highlighted inspections and effective follow up support as a key area for improvement in the Uganda education sector that has the potential to lead to significant learning gains.** In the Education and Sports Sector 2017/18-2019/20 Strategic Plan, a top priority is to ‘strengthen the current inspection system and approaches by increasing the frequency of inspection of schools and institutions’, with a focus ‘on the quality of leadership and management, teaching and learning process and learner achievement’ (MOES, 2017) The Directorate of Education Standards (DES) has made strides in this direction by reforming its inspection framework in 2018 with support from international experts. The new approach to inspections was rolled out nationwide in 2019 and moves from a system of checking compliance with input standards, towards one that ensures quality education. MoES/DES now want to ensure comprehensive adoption and that inspections are coupled with meaningful support to drive school improvement.

DES has been working with PEAS (Promoting Equality in African Schools) since 2019 to implement the ‘Inspect & Improve’ pilot in government schools to strengthen the support and supervision provided to schools and empower school leaders to drive improvements. PEAS is a non-profit organisation working to expand access to sustainably delivered, quality secondary education across Africa. Since 2008, PEAS built a network of 28 secondary schools in Uganda and 4 in Zambia which aim to provide a quality education to all young people regardless of their background or circumstance. PEAS and DES co-designed the ‘Inspect & Improve’ pilot to generate evidence about how to improve quality in Ugandan schools by adapting and implementing components of the PEAS support and supervision model in 10 selected government schools. The project is contributing to wider reform efforts that the Directorate of Education Standards (DES) is leading designed to develop a national school improvement model. The Inspect & Improve project involves PEAS working with DES and local government representatives to carry out inspections using the newly reformed DES inspection process. This is followed by PEAS adapted school improvement support to help school leaders respond to their inspection findings.

The project builds on international evidence and PEAS proven track record in supporting and supervising schools in Uganda to deliver high quality education cost-effectively. International evidence identifies a positive relationship between strengthening school management practices and improved learning outcomes (Bloom et al., 2015; Crawfurd, 2017). PEAS’ inspections and school improvement practices have been recognised as an area of strength. An external evaluation found that inspections in PEAS Uganda schools are highly correlated with learning outcomes and that inspectors provide leaders and teachers with meaningful feedback to improve performance. In addition, an independent study by the University of Sussex found that PEAS schools are substantially better managed than other school types in Uganda and perform correspondingly better in terms of student value-add.

Initial findings suggest the project has had a positive impact on school leaders’ ability to drive improvement. Despite disruptions to the external research and evaluation process due to Government-mandated school closures during 2020 due to the COVID19 pandemic, positive indications of the pilot’s impact were evidenced through a PEAS-DES assessment. In particular, the assessment found that school leaders highly valued the support they received through the programme, were observed to be successfully managing lesson observations systems, monitoring teacher attendance, and taking steps to increase girls’ engagement. In the majority of schools, improvements could be seen in both learner and teacher attendance, as well as enrolment. An external evaluation of the pilot is being conducted by the National Foundation for Education Research (NFER). The endline evaluation to verify the impact of the pilot is pending and the baseline report can be found here.

In 2021, DES and PEAS will be extending the Inspect & Improve programme to generate further evidence about the programme’s impact to embed into the Government’s approach to school improvement. Next year, the Inspect and Improve programme will be extended to an additional 40 schools across all regions in Uganda to better understand the programme’s impact at scale. DES and PEAS will also be piloting sustainable digital tools to explore their potential to increase the efficiency of inspections and follow-up support. This phase 2 of the project will be rolled out in two waves; the first 20 new schools will be added to the programme in Term 1 of 2021 and the final 20 new schools will be added in Term 2 2021. The digital tools will be piloted in the original 10 partner schools who will have had the longest exposure period to the programme. The programme will run until March 2022 for all schools, with the potential for further extension pending additional funding and partner commitment.

Learnings from this programme also have the potential to contribute to the international debate about school improvement and help create a greater understanding of how to improve school leadership and management using cost-effective approaches based on implementation experience. International evidence has shown that across low and middle-income countries, poor quality school leadership and management is a significant barrier to quality education (Biamba 2016; Moorosi and Bush 2019). The findings from the evaluation of the Inspect & Improve programme could contribute to the evidence base regarding how to strengthen school leadership and management practices, in turn providing students with a high-quality education that helps them fulfil their potential.

HOW WILL LEARNINGS BE USED?

PEAS and DES are seeking to develop a ‘learning partnership’ with a research partner(s) to provide a formative, robust evaluation of the programme’s impact and potential for scale. Emphasis is placed on ‘partnership’, rather than a formal research and evaluation as this evidence is primarily sought by PEAS and DES to enable informed improvements to the ongoing programme, rather than to provide a summative evaluation. The findings will be used in the following ways:

• By PEAS to adapt and improve the delivery of the project during its lifetime.
• By DES to learn lessons from the project in order to embed successful elements of the programme into business as usual and at national scale.
• By PEAS and DES to advocate for additional resources to be committed to support and supervision of schools by existing and new partners and stakeholders in order to scale-up the activities /benefits delivered by the project.
• By PEAS to learn lessons from the project for the purpose of replicating what works elsewhere.
• By other education networks, Governments, and stakeholders in the education sector to inform wider understanding of the mechanisms that contribute to school improvement.

OBJECTIVE OF LEARNING PARTNERSHIP AND RESEARCH QUESTIONS

The objective of the learning partnership is to assess the delivery, effectiveness, value for money and impact of the programme and report the findings and lessons learnt throughout the process. The research partner will be required to develop an evaluation approach that answers the following overarching questions:

1. What impact, if any, did the intervention have on the quality of leadership and management in intervention schools? How has it impacted on the ability and effectiveness of school leaders to respond to girls’ needs?
2. How did the intervention demonstrate the observed impact? What worked (and did not work) to improve the quality of leadership and management in intervention schools?
3. What other areas of the school’s operations, such as – for example – staff management or planning practices – if any, did the intervention affect, and how?
4. How much did the intervention cost to deliver? Did it represent value for money?
5. What has DES identified through the project that could be embedded into Government guidelines and be implemented at a wider scale?
6. What digital tools have been utilised in partner schools and what has determined whether schools actively engage with use of tools?**

EXPECTED DELIVERABLES

The main deliverables for this project are as follows:

• Inception report: setting out the design of the MEL strategy and plan and associated planning, logistics, quality assurance, child protection measures and risk management information including gender analysis.

• Baseline study report: design, conduct and submit a baseline study that describes the initial conditions (during year one of the project) against which progress can be measured or comparisons made to show the effects and impacts of the project in the final project research and evaluation report. Prior to the baseline being conducted data collection tools will need to be developed and piloted.

• Final project research and evaluation report: design, conduct and submit a final project research and evaluation report that assesses the effectiveness, impact and VfM of the project.

Report requirements:

• All reports should be submitted in electronic form and should be submitted in English.

• The research partner will provide face-to-face presentations in-country of findings

• The Research Team will be expected to provide a fully ‘cleaned-up’ dataset, including recordings, transcriptions, and translations of any qualitative data. .

Detailed work plan:

• Applicants are required to provide a detailed work plan incorporating all relevant tasks from start to finish of the research and evaluation study.