Evaluation Purpose The purpose of this final external evaluation is two folds: accountability and institutional learning. The findings will inform the partners of GIDP about the programme’s achievements and support institutional learning of the stakeholders engaged, both from the Government and development partners. The evaluation is to guide UNDP -UNCDF and government partners to take stock of the progress made in public administration reforms and identify areas for making governance inclusive for all. The draft 9th NSEDP aims at improving the efficiency of the public sector in providing basic services to citizens through DDF, ODSCs, digitalization of services and in creating opportunities for citizens to provide feedback on improving services. It is, therefore, critical for the UNDP-UNCDF teams to understand the main strengths and weaknesses of the current project and to use it a reference guide for developing projects for improving governance. As such, the aim is not to only evaluate the achievements of GIDP but informing the direction and design of next public administration reforms project and UNDP’s continued involvement with MoHA within the governance sector in Lao PDR.
With the 9th NSEDP- 2021-2025) currently being finalised by the GoL, this evaluation should identify ways for the next phase of public administration reforms in line with the NSEDP and in particular, the MoHA Five Year Plan (2021-2025) on improving governance through public administration reforms. Evaluation Scope and Objectives The independent evaluation will be forward-looking and explore opportunities for UNDP to position itself as a strategic partner of GoL in the governance sector. The scope of the evaluation will specifically encompass the role MoHA, MPI, MoJ and in taking ownership of promoting wider governance improvements after the end of the programme and on the basis of the results of the GIDP. The evaluation will look at systems strengthening, and application of various tools developed by GIDP to improve accountability and citizen’s participation. The extent to which the programme learnings have fed into the national policy dialogue will be one of the areas of evaluation. Evaluation criteria and key guiding questions 1. The evaluation will address 3 fundamental questions: What did the project intend to achieve during the period under review? 2. To what extent has the project achieved its intended objectives? 3. What factors have contributed to or hindered the project’s performance and eventually the sustainability of the results? Relevance • To what extent has the GIDP programme addressed key governance issues as outlined in the 8th NSEDP-Outcome 2, MoHA Sector Plan 2021-25 and recommendations of the RTIM pertaining to governance? • To what extent is the programme aligned with the national development needs and priorities including Sam Sang Directive and has been able to address relevant targets under SDG 16? • As the programme is about inclusive development, how well does the design of the programme address the needs of the vulnerable groups such as women, ethnic groups, and persons with disabilities in the country? • What opportunities has the programme created or identified in improving local governance? • Has the programme pro-actively addressed emerging demands and opportunities unforeseen during the design of the intervention, adapting its theory of change to respond to changes in the country context and governance landscape, including national priorities, legislative and policy updates, changes in power relation among key stakeholders? • Has the programme resulted in empowerment and capacity development of the local authorities and PPA members as envisaged in the programme design and if so, are the efforts sustainable? • Is there any indication of the government continuing the efforts of promoting accountability and citizen’s as well as NPA /private sector engagement? Effectiveness • In which component did the programme have the least number of tangible achievements? What have been constraining factors and why? How can they or could they be overcome? • How and to which extent have the three main components complemented each other, strategically contributed to the project’s objectives and made use of the initially proposed interlinkages of the three main components in a virtuous loop? • To what extent were the overall objectives achieved / are likely to be achieved? How/to which extent have the activities/outputs strategically contributed to those? • What were the major factors influencing the achievement or non-achievement of the objectives? • To what extent are the intended beneficiaries satisfied with the results? How well are gender and ethnicity considerations been considered? • To what extent has the programme work been able to form and maintain partnerships with other development actors, including other UN agencies, Development Partners, Civil Society Organisations, or government agencies? • Has there been regular reviews of the progress to ensure that the programme is on track to achieve the desired results and to inform course corrections if needed? How has the programme reacted to challenges identified, including addressing recommendations from the Mid-Term Review and the GSWG Secretariat Capacity Assessment conducted in 2019? • Has the governance mechanism (GIDP Programme Board) provided their guidance and functioned well? If not, comments and recommendations to be provided. • Are the outcome indicators measured against baseline and target values (if available) and reflects quantitative and qualitative dimensions of the achievement? • To what extent has the programme achieved or led to unexpected development results or outcomes, which were not originally envisaged in the Project Document and Theory of Change? Efficiency • How cost-effective and time-efficient is the implementation of activities to achieve the outputs by programme implementing partners during the evaluation period? What measures are being taken to ensure competitiveness? • What are the transaction costs for each component? • How efficiently did the various modalities of the joint partners, UNDP and UNCDF provide the required support to the government in implementation of the programme? • To what extent are the planned funding and timeframe enough to achieve the intended outcomes? • How well did the implementing partners mobilise resources to fill the funding gaps as envisaged in the programme document? What lessons can be learned from this element? And how can the programme do better? Sustainability • Are stakeholders and national partners fully engaged in the decision-making, implementation, and monitoring of the programme? • To what extent has the programme contributed to the wider governance improvements and what are the indications that the government will benefit from systems strengthening, capacity development and tools developed by the programme to realize inclusive development after the project’s completion? • What were the major factors which influenced non-achievement of sustainability of the programme? 6 • To what extent is the Government of Lao PDR increasing its capacity and ownership for improving public administration during the period in question? What impact has this had on external support? • Is the Exit Strategy/Transition Plan developed by GIDP being implemented d is the plan regularly reviewed and adjusted according to the project progress, including its financial commitments and capacity? • To what extent did the benefits of a programme or project will be able to continue after donor funding ceased? • To what extent has the lessons learned fed into national policy dialogue on socio-economic development through the Governance Sector Working Group? Inclusion • As the programme is about inclusive development, how well did the implementation of the programme in accordance with the plan address the most marginalized and vulnerable groups such as women, people with disabilities and ethnic minorities in the country. • To what extent has the project contribution to wider governance improvement result in promotion and protection of marginalized and vulnerable groups such as women, persons with disabilities and ethnic minorities? Gender Equality • How well did the project ensure that women, girls, boys, and men have equal access to basic service delivery? What lessons can be learned from this element? And how can the programme do better? • As the programme is about increasing accountability and citizen engagement, how well did the programme ensure that women’s meaningful participation in the decision-making process at the local level? Methodology The evaluation methodology will adhere to the United Nations Evaluation Group (UNEG) Norms & Standards. The evaluation will be carried out by an independent evaluation team. The evaluation team should adopt an integrated approach involving a combination of data collection and analysis tools to generate concrete evidence to substantiate all findings. Evidence obtained and used to assess the results of UNDP support should be triangulated from a variety of sources, including verifiable data on indicator achievement, existing reports, evaluations and technical papers, stakeholder interviews, focus groups, surveys and site visits where/when possible. Methodological approaches may include some or all of the following: ? Document review of all relevant documentation. This would include a review of inter alia ? Project document (contribution agreement). ? Theory of change and results framework, ? Programme and project quality assurance reports. ? Annual workplans. ? Quarterly and annual reports. ? Results-oriented monitoring report. ? project board minutes. ? financial reports. ? Mid-term external project evaluation (2019) and UNDP CPD evaluation ? Exit Transition Plan (2020)
? Semi-structured interviews with key stakeholders including key government counterparts, donor community members, representatives of key civil society organisations, UNCT members and implementing partners:
? Development of evaluation questions around relevance, effectiveness, efficiency and sustainability and designed for different stakeholders to be interviewed. ? Key informant and focus group discussions with men and women, beneficiaries and stakeholders. ? All interviews should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals. ? Surveys and questionnaires including participants in development programmes, UNCT members and/or surveys and questionnaires involving other stakeholders at strategic and programmatic levels. ? Field visits and on-site validation of key tangible outputs and interventions. ? The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners and direct beneficiaries. ? Other methods such as outcome mapping, observational visits, group discussions, etc. ? Data review and analysis of monitoring and other data sources and methods. o Ensure maximum validity, reliability of data (quality) and promote use; the evaluation team will ensure triangulation of the various data sources. The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP, stakeholders and the evaluators. In line with the UNDP’s gender mainstreaming strategy, gender disaggregation of data is a key element of all UNDP’s interventions and data collected for the evaluation will be disaggregated by gender, to the extent possible, and assessed against the programme outputs/outcomes. Evaluation products (deliverables) These products could include: ? Evaluation Inception report (10-15 pages). The inception report should be carried out following and based on preliminary discussions with UNDP-UNCDF after the desk review and should be produced and endorsed by UNDP before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the country visit in the case of international evaluators (see template in the annex). ? Kick-off meeting. Evaluators can seek further clarification and expectation from UNDP, SDC, UNCDF and Government partners in the kick-off meeting ? Evaluation debriefings. Immediately following an evaluation, UNDP-UNCDF may ask for a preliminary debriefing and findings. ? Draft evaluation report (50 pages max including executive summary and lessons learned).1 The programme unit and key stakeholders in the evaluation should review the draft evaluation report and provide an amalgamated set of comments to the evaluator within an agreed period, addressing the content required (as agreed in the TOR and inception report) and quality criteria as outlined in these guidelines. 1 A length of 40 to 60 pages including executive summary is suggested.
? Evaluation report audit trail. Comments and changes by the evaluators in response to the draft report should be retained by the evaluators to show how they have addressed comments. ? Final evaluation report (see template in the annex). ? Presentations to stakeholders ? Evaluation brief and other knowledge products or participation in knowledge-sharing events, if relevant.
|