National Consultant to review the CPD Outcome 3 – ‘Data and Knowledge Foundations for Evidence-based Policy Development’ – For Sri Lankans only

Mid-Term Evaluation cum Strategic Direction Setting of United Nations Development Programme (UNDP) Sri Lanka Country Programme 2018-2022

The UNDP Country Programme Document 2018 – 2022 (CPD) outlines UNDP Sri Lanka’s contribution towards the national development priorities in Sri Lanka and is aligned with the United Nations Sustainable Development Framework (UNSDF) 2018-2022 and the UNDP Strategic Plan (2018-2021).

As the CPD reaches the halfway mark, UNDP plans to commission a Mid-Term Evaluation (MTE) cum Strategic Direction Setting exercise, to assess the attainment of the country program’s results across all outcome areas, and provide course corrections through the strategic direction setting. This exercise comes in the wake of several key contextual changes in the country; including ushering of a new government, a global pandemic and a growing budget deficit impacting the development trajectory of the country and the positioning of UNDP therein. From a political standpoint, contextual factors such as the constitutional crisis in 2018, the Easter Sunday terror attacks of April 2019, the Presidential elections of November 2019, the mostly recently concluded Parliamentary elections in August 2020 are significant.  The 20th Amendment to the Constitution of Sri Lanka which was passed on 22nd of October 2020, has direct impacts on the country program; i.e. reconfiguration of the state architecture, centralization of power, roll back of Transitional Justice institutions, etc.

The UN Development System Reform which began in 2019, resulted in the delinking of the Resident Coordinator’s (RC) Office. This led to a repositioning of UNDP’s role as an SDG integrator for the UNDS, offering tools, resources and platforms to convene multi-stakeholder groups to address complex and interconnected development challenges.

The appointment of a new majority Government by President Gotabhaya Rajapaksa in August 2020 rolled out a new policy framework titled Vistas of Prosperity and Splendor that serves as the key national development policy document aimed at achieving the fourfold outcome of a productive citizenry, a contented family, a disciplined and just society and a prosperous nation.

In July 2020, following the impact of COVID-19, the World Bank downgraded Sri Lanka from an Upper Middle-Income Country (UMIC) to a Lower Middle-Income Country. At the same time the knock-on impacts globally, regionally and nationally threaten a follow-on national social and economic crisis. The public debt situation and weakening fiscal space remains a key concern. It also comes in the wake of Moody’s downgrade of the Government of Sri Lanka’s long-term foreign currency issuer and senior unsecured ratings to Caa1 from B2 in October. The economic shock is likely to be transmitted through a number of channels – particularly trade, export industries (especially apparel), tourism, agriculture, employment, investment, remittances, and SMEs (vast majority of Sri Lanka’s enterprises are MSMEs, accounting for nearly 80% of GDP and 45% of employment in 2018).

A protracted crisis will have an adverse impact on specific populations groups (women, older persons, persons with disabilities etc) who are already vulnerable due to existing socio-economic inequalities. With considerable clustering of Sri Lankan population around the poverty line, a protracted crisis will have an adverse impact on poverty and inequality adversely affecting the baseline trajectory of Sri Lanka reaching the SDG targets and making the Agenda 2030 even more challenging. The UN Country Team’s “UN Advisory Paper on Immediate COVID19 response” technically led by UNDP refers to many of these challenges and offers guidance on priorities for Sri Lanka’s (GoSL) national recovery and SDG plans.

The MTE takes place at an opportune time; while being cognizant of the challenges with COVID-19. With a new government, revised institutional architecture, new policy priorities the MTE offers UNDP to take stock while realigning its support in line with UNDP’s 2.0 offer to effectively position itself as a partner of choice. Findings from the evaluation may also offer inputs to the CCA exercise now underway for completion by April of 2021. The strategic Direction Setting component of the exercise will offer inputs to the new UNSDCF process. Lessons from Sri Lanka may also be leveraged to inform UNDP’s new Strategic Plan 2022-2025 under development.

Purpose and Objectives

The overall objective of the Mid Term Evaluation (MTE) and Strategic Direction Setting is to take stock of the progress in achieving the results of the Country Programme, its relevance, efficiency and effectiveness of strategies and interventions in light of the development priorities, and emerging development issues including in the context of COVID-19 impact at the national and sub-national level. Specifically, the review will:

• Assess contribution of the Country Programme 2018-22 including through policy engagement towards the achievement of the national development priorities and objectives within the framework of three Outcome areas and the UNSDF;
• Verify key results achieved, analysis of key factors that have contributed to the achievement or non-achievement of intended outputs and results;
• Review strategies and programme performance including cross sectoral approach / linkages and analyze resource mobilized under the respective outcome areas;
• Identify mid-course adjustments, if any, required for the remaining period to ensure strategic positioning and relevance of the Country Programme;
• Provide strategic direction, identification of lessons learned, best practices and actionable strategic recommendations including in the context of UNDP’s COVID response and recovery framework towards 2030 to feed into the formulation of the next Country Programme 2023 – 2027.

The primary audience for the use of MTE findings and recommendations will be UNDP Sri Lanka.

The MTE outcome will inform UNDP’s partnerships with the government at national and sub national level and key development partners.

For full Terms of Reference of the Entire Team, please refer Full TOR of the MTE Team

Team : Consultant (4) positions – 1 (One) Team Lead/International Consultant & 1 (one) Local Consultant for each of the three Country Programme Outcome areas.

The MTE will cover the work undertaken during the period January 2018 to November 2020 and provide an initial assessment of the progress and contribution of UNDP to the development results set out in the CPD results framework 2018 – 22.

The MTE cum strategic direction setting report will consist of two parts; the mid-term evaluation and the strategic direction setting component.

The scope of the evaluation will include a review of the original CPD assumptions against the change in circumstances that have occurred since their drafting,  review of the CPD Results Matrix and M&E framework, analysis of existing contributions and gaps in light of the SDGs and key themes which may not have been reflected in the CPD and revisit the theory of change with potential updates to it. The context analysis component that will be carried out as part of the UN country team commissioned Common Country Analysis (CCA) can provide valuable input towards this exercise as well.

The findings of the evaluation will inform the strategic direction setting component of the report; which will include recommendations for improvements in performance and results in the remaining period of the Country Programme cycle; identifying implementation challenges and ways to overcome them; provide lessons learned which can support the delivery of results in the remaining period of the existing CPD and opportunities for scale up and support the preparation of the next CPD.

The findings/recommendations need to be evidence-based, supported by an open and participatory consultative process for adequate stakeholder engagement.

Review Criteria and key guiding questions

The MTE exercise will comply with the UNDP evaluation guidelines. The evaluation questions should be grouped according to the four OECD-DAC evaluation criteria: (a) relevance; (b) effectiveness; (c) efficiency; and (d) sustainability (and/or other criteria used).  

Relevance: This will examine the relevance of the Country Programme in line with the national development priorities, evolving context and opportunities, partnership environment, achievement of the SDGs in the country, alignment with the UNDP Strategic Plan 2018 -21 and specific measures, if any, that will require to be undertaken to ensure full alignment and impact. In doing so, review the theory of change, UNDP’s comparative advantage and strategic positioning including policy engagement etc will be considered. Considering the emerging COVID 19 crisis, it would be useful to assess the extent that this Country Programme is appropriately responsive to the needs of the national and sub-national constituents and changing development partner priorities. 

Effectiveness: This will look into what extent has the Country Programme achieved output results and evidence of their contribution to the outcomes. In doing so, the review will inter-alia look at whether the principles of leaving no one behind including gender equality were applied, the requisite capacity enhancement of the national and sub-national counterparts took place, leveraging resources and capacities, whether cross cutting approaches such as partnerships, innovations and acceleration, south-south cooperation etc were adequately considered. The review will also look into the programme areas most relevant and strategic for UNDP to scale up or consider going forward.

Efficiency: This criteria looks into what extent the programme results obtained justifies the efficient use of resources (financial and human) and if there is alignment between what has been expended (resource wise) and what has been achieved. Other considerations such as quality and timeliness of the results, value for money, partnership strategies and resource mobilization, use of programming and partnership modalities conducive to the delivery of programme outputs, adequate oversight and monitoring mechanisms, extent to which UNDP policies, processes and decision making have contributed to the achievement of the Country Programme results. The Country Office business processes to support the timely implementation of programming particularly in the context of the COVID crisis may also be considered.

Sustainability: This will look into what extent did UNDP establish mechanisms to ensure the sustainability of the results attained including capacities of national and sub national partners, institutional mechanisms including sustainability and transitioning strategies in place to sustain the programme results. The MTE can look into the longer-term projects and review progressive growth and evolution of the projects to assess improvement in capacities of national partners. It also assesses, among others, to what extent do strategic partnerships exist with other national and sub national institutions, COS/NGOs, UN agencies, private sector and development partners to sustain the attained results and to what extent have partners committed to providing continuing support.

Guiding questions will need to be further refined by the MTE external team and agreed with UNDP. Please refer No. 2 under “List of detailed Information”. Methodology

The MTE will be guided by the purpose, scope and objectives outlined above. The MTE should employ a combination of both qualitative and quantitative evaluation methods and instruments. The MTE should employ a participatory approach whereby interviews with key stakeholders will provide and verify the substance of the findings. Information will be triangulated (i.e. verified from different sources) to the extent possible, and when verification is not possible, the single source will be mentioned. Analysis leading to the evaluative judgements should always be clearly spelled out. The limitations of the methodological framework should also be spelled out in the review reports.

The following methodologies for data collection are expected to be applied during the MTE:

a)           Desk review of key programme documents and relevant sources of information;

b)           Semi-structured interviews, questionnaires with key stakeholders as agreed in the inception report;

c)            Regular consultations with the UNDP Sri Lanka senior management and MTE Reference Group;

d)           In addition, any necessary methodologies for ensuring that the MTE addresses the Country Programme multi-faceted nature of work, changes in programming context and needs of vulnerable groups in the CPD.

The final methodological approach including interview schedule, questionnaire, and data to be used in the MTE will be clearly outlined in the inception report and be fully discussed and agreed between UNDP and the MTE external team.

The MTE will be divided into three stages:

Stage I: Planning Phase (will be conducted remotely)

With the COVID 19 crisis, ensuring the safety of the MTE teams, Phase 1 of the MTE will be conducted remotely / virtually by the MTE external team which includes remote arrangements to conduct five key tasks (a) desk review of key documents and relevant sources of information; (b) context analysis in which the Country Programme has been operating since its inception in 2018 and current evolving context (e.g. COVID crisis); (c) finalize inception report; (d) design of MTE methodology, tools and questionnaires; and (e) orientation of UNDP Country Office team on the MTE process, methodology, timelines etc. The Inception Report will be finalized after it has been reviewed and cleared by the UNDP Sri Lanka MTE Reference Group.

Stage II: Validation (virtual or on site/ face to face subject to the COVID 19 pandemic situation and the uncertainties around travel etc.)

Option 1: Virtual validation

Consultations with and inputs from various stakeholders will be critical to feed into and validate the programme contribution and results. In case of travel and movement restrictions, the MTE team will collect data and seek inputs and feedback virtually through relevant web-based surveys, virtual meetings, electronic exchanges etc with key government partners, CSO, private sector, donors, implementing partners, project beneficiaries, UN and other development partners as well as internally with programme and project teams. Photographs, videos, satellite imagery etc could be used to gather data & evidence. Based on the methodology, analysis plan and desk review, the external MTE team will identify stakeholders for data collection in consultation with UNDP and finalize the check list with identified areas of query.  To the extent possible efforts will be undertaken to gather feedback from the project beneficiaries to ensure the voices of all groups including the vulnerable and marginalized will be taken into account.

Option 2: On-site or face to face validation

If the COVID 19 situation permits, the MTE external team (local consultants) will visit select field sites identified by UNDP to undertake in-depth interviews with government stakeholders both at national and state level, development partners, stakeholders involved in programme and project implementation and beneficiaries, discussion with UNDP Team Leaders, programme and project teams.  Focus Group Discussions to be held wherever appropriate (specially recommended for beneficiaries). All interviews should be undertaken in full confidence and anonymity.

Stage III: Analysis, Debriefing and Report Writing (Remotely or in UNDP premises in Colombo subject to the COVID situation and uncertainties around travel etc.)

Upon completion of the desk review and data collection, the MTE external review team will analyze all information and develop a draft MTE report for the UNDP Sri Lanka CO. The external review team is required to ensure validity, credibility and reliability of data including through triangulation of the various data sources. Prior to the drafting of the MTE and Strategic Direction Setting report, the MTE external team will debrief UNDP programme and management teams with preliminary findings ensuring that they are based in evidence. Debriefings with key stakeholders will be organized virtually or face to face where possible.

The draft report including the recommendations will be shared with staff and the management for their feedback, inputs and address any factual errors. Following the revision of the draft report, a workshop (or virtual equivalent) will be organized with internal and external stakeholders facilitated by the external review team to share and validate the draft report. The outcome of this workshop will be incorporated in the MTE final report.

Process Management and Timelines

Under the guidance and oversight of the UNDP Resident Representative and Deputy Resident Representative (Programme), the Results and Resources Management Team Lead with support from M&E Associate will be responsible for overseeing the MTE process and ensure quality and timeliness of the deliverables. An MTE Reference Group comprising UNDP senior management, programme and operational team leads and key programme and project staff will support the MTE process and give comments and directions at key stages of the MTE process. The Group will review the MTE deliverables for quality and completeness and request for adjustments from the external MTE review team as needed. The external MTE review team is expected to present drafts and final findings and recommendations first to UNDP Senior Management and the MTE Reference Group. The MTE Reference Group will review and clear the draft reports and final findings and recommendations.

The mid-term evaluation component, which will be undertaken by the international consultant/team lead.  The strategic direction setting component, while benefitting from the recommendations of the first component, will be led by a team from UNDP’s Bangkok Regional Hub which will build upon the evaluation recommendations and support the country office to craft any adjustments to the CPD within the contours of the UNDP 2.0 offer.

Below is the proposed schedule and number of person days:

Deliverables/Outputs – Estimated Duration to Complete

• Desk review and analysis – 5 days
• MTE Inception report with final design, methods and tools and presentation / de-brief to senior management followed by presentation to staff – 3 days
• Stakeholder interviews, data collection and analysis – 6 days
• MTE Draft Report – 4 days
• Debrief on draft findings and recommendations to the management and MTE Reference Group – 4 days
• MTE second report incorporating feedback and presentation to staff – 3 days
• MTE final draft and presentation to management and MTE Reference Group – 8 days
• MTE Final Report – 7 days

The multiple reviews and revisions may be necessary for the draft report, and the final payment will be made to the Consultants only when UNDP senior management approves the final MTE report. Comments will be incorporated as deemed appropriate by the MTE team. A ‘Response to comments matrix’ will be prepared by the MTE team to show how comments received have been dealt with in the Final MTE Report. The process is considered complete once the final report is approved by UNDP.

Key Deliverables:

• Inception report (6-8 pages max) including key questions and tools.
• Draft MTE Report (30 – 35 pages max) excluding annexures. The report should be strategic, future-oriented, results-driven and analytical. The report should also include a tabulated summary of strategic progress overview against each CPD Output, and adjusted / aligned CPD Results Framework matrix.
• Final MTE Report including a final tabulated summary of strategic progress overview against each CPD Output, and adjusted / aligned CPD Results Framework matrix.

Duration of the Work:

The total duration of the MTE will be 40 days over a period of three months from December 2020 to February 2021.

Key technical and core competencies of the MTE team:

• Knowledge of the political, social, cultural, and economic context of the Country; Ability to lead informed and strategic discussions with government counterparts and stakeholders;
• Ability to undertake gender sensitive review and analysis and demonstrated understanding of issues related to gender-mainstreaming and other cross cutting development issues;
• Knowledge of qualitative and quantitative evaluation methods; Proven experience in data analysis as well as report writing; ensuring the triangulation of the findings, obtaining strong evidence for the analysis of information from multiple sources;
• Strong analytical skills (quantitative and qualitative) and strong ability to communicate and summarize this analysis in writing; maintaining comments and feedback matrix; Proven ability to produce high quality reports;
• Experience of conducting stakeholder interviews and collecting data remotely;
• Ensuring comments and feedback of the stakeholders is sufficiently addressed in the draft and final reports. Thematic / Outcome Consultants to provide inputs to the consolidated draft and final MTE report and join the Lead Consultant in presentations and sharing of key findings to the government and stakeholders, briefing and debriefings with UNDP Sri Laannnka team;
• Deliver the agreed MTE products to the right standard and quality;
• Excellent inter-personal, teamwork and communication skills;
• Flexibility in remote working arrangements and experience of conducting stakeholder interviews and collecting data remotely;
• Openness to change and ability to receive and integrate feedback;
• Excellent report writing, presentation and editing skills in English;
• Keep to the standards and ethical principles in line with UNEG norms and standards and Ethical Guidelines. 

The Consultants must be independent from any involvement in designing, executing or advising any aspect of the intervention that is the subject of the Mid-term evaluation.  To this end, interested Consultants will not be considered if they were directly and substantively involved, as an employee or Consultant, in the formulation of UNDP strategies and programming relating to the CPD under review.

• Master’s Degree in Social Science, Economics, Data Sciences or any other related field; knowledge of current development issues, evaluation discipline is a must.

Experience:

• Minimum 7 years relevant experience in multi-sectoral program/project management, capacity building and policy advisory.
• Proven recent experience in conducting program and project evaluations.
• Proven experience in conducting Evaluations in line with UNDG and UNEG Evaluation Norms and Standards and Ethical standards, as well as OECD/DAC evaluation principles.
• Strong analytical capacities (quantitative and qualitative) and strong ability to communicate and summarize this analysis in writing.

Payment Terms 

This is a fixed output-based contract price, regardless of extension of the herein specific duration. Annex

Payment milestone / Payment Amount

1.           Finalized Inception Report / 20%

2.           Draft Report with all comments incorporated / 30%

3.           Final Evaluation cum Strategic Direction setting report duly cleared and approved by UNDP Management / 50%

Evaluation Method & Criteria:

The consultant would be selected based on the following criteria:

• Technical: 70 points
• Financial: 30 points

Criteria for Technical Evaluation would be as under:

*  Educational Qualification, relevant development and work experience – 50 marks

*  Technical Proposal – 20 marks

Technically qualified consultants who score more than 70% i.e. 49 marks with respect to the above-mentioned evaluation criteria, will be considered for final evaluation.

HOW TO APPLY

To apply please access UNDP Job Site

The Consultant is required to submit the following documents, in a single combined PDF file, as the system has provision for uploading only one attachment:

• Indicate upfront the position for which you are applying (i.e. Team Lead; CPD Outcome 1 with title; CPD Outcome 2 with title; or CPD Outcome 3 with title)
• Personal CV with relevant experience to the TOR; and at least 3 professional references 
• Short technical proposal (max 2-pages) including methodology, approach & assessment criteria, process followed, data collection and analytical tools.
• Financial Proposal (Format can be accessed via link Letter of Confirmation of Interest and Availability )
• Individual consultants will be evaluated based on a combination of factors including cover letter, the credentials on offer and brief interview (optional) and the offer which gives the best value for money for the UNDP.

Not submitting of all above mentioned documents together with the CV, will be treated as a disqualification.

FINANCIAL PROPOSAL  

All Inclusive Lump Sum Fee: LKR (_______________________________________)

Or

All Inclusive Daily Fee: LKR (_____________________________________)

Amount in words:  (Rs._____________________________________________________)

Note: Payments will be based on invoices on achievement of agreed milestones i.e. upon delivery of the services specified in the TOR and certification of acceptance by the UNDP. The applicant must factor in all possible costs in his/her “All Inclusive Lump Sum Fee” including his/her consultancy and professional fee, travel (economy class flights is required to and from the Duty Station) honorarium, board and lodging, and any other foreseeable costs in this exercise. No costs other than what has been indicated in the financial proposal will be paid or reimbursed to the consultant. UNDP will only pay for any unplanned travel outside of this TOR and Duty Station on actual basis and on submission of original bills/invoices and on prior agreement with UNDP officials. Daily perdiums and costs for accommodation/meals/incidental expenses for such travel shall not exceed established local UNDP DSA rates.

For an Individual Contractor who is 65 years of age or older, and on an assignment requiring travel, be it for the purpose of arriving at the duty station or as an integral duty required travel under the TOR, a full medical examination and statement of fitness to work must be provided.  Such medical examination costs must be factored in to the financial proposal above. Medical examination is not a requirement for individuals on RLA contracts. 

List of detailed Information (Annexes) :

1. Roles of the International Consultants and Local Consultant 
2. Sample Evaluation Questions
3. Inception report structure / outline
4. MTE final report content structure / outline
5. Response to MTE comments matrix
6. List of key documents to be reviewed

1)  Roles of the Team Leader (International Consultant) and Local Consultants 

Role of the Team Leader

• Develop a robust methodology for the evaluation.
• Design the evaluation questions in consultation with the national team of experts.
• Participate in key consultations and focus group discussions with National counterparts, UNDP staff and other partners.
• Structure the inception report and the final evaluation report.
• Coordinate input from the national experts for the final evaluation report.
• Draft the strategic direction setting component, in consultation with the team at the UNDP Bangkok Regional Hub.
• Finalize the evaluation report, taking into consideration feedback from the UNDP Country office and Bangkok Regional Hub.

Role of the National Team of Experts

• Provide inputs to development of the methodology and evaluation questions.
• Carry out desk reviews, secondary data gathering and literature review, in line with the methodology and scope of each of the CP outcomes. 
• Organize stakeholder consultations and interviews on some of the key topics identified: This may include interviews with key stakeholders and partners and focus group discussions, which may include UNDP staff, UN heads of agencies, UNSDF Working Groups/Pillar groups, relevant government officials at both national and sub-national levels, development partners, and civil society/youth/women representatives (if/where deemed relevant).
• Provide inputs to the draft and final evaluation reports.

2)  Evaluation Sample Questions

Relevance

• To what extent is the initiative in line with the UNDP mandate, national priorities and the requirements of targeted women and men?
• To what extent is UNDP support relevant to the achievement of the SDGs in the country?
• To what extent did UNDP adopt gender-sensitive, human rights-based and conflict-sensitive approaches? 
• To what extent is UNDP engagement a reflection of strategic considerations, including the role of UNDP in a particular development context and its comparative advantage?
• To what extent was the method of delivery selected by UNDP appropriate to the development context?
• To what extent was the theory of change presented in the outcome model a relevant and appropriate vision on which to base the initiatives?

Effectiveness

• To what extent has progress been made towards outcome achievement? What has been the UNDP contribution to the observed change?
• What have been the key results and changes attained? How has delivery of country programme outputs led to outcome-level progress? 
• Have there been any unexpected outcome-level results achieved beyond the planned outcome?
• To what extent has UNDP improved the capacities of national implementing partners to advocate on environmental issues, including climate change issues and disaster risk reduction?
• To what extent has UNDP partnered with civil society and local communities to promote environmental and disaster risk awareness in the country?
• To what extent have the results at the outcome and output levels generated results for gender equality and the empowerment of women?
• To what extent have marginalized groups benefited? 
• To what extent have triangular and South-South cooperation and knowledge management contributed to the results attained?
• Which programme areas are the most relevant and strategic for UNDP to scale up or consider going forward?

Efficiency

• To what extent have the programme or project outputs resulted from economic use of resources?
• To what extent were quality country programme outputs delivered on time?
• To what extent were partnership modalities conducive to the delivery of country programme outputs?
• To what extent did monitoring systems provide management with a stream of data that allowed it to learn and adjust implementation accordingly?
• To what extent did UNDP promote gender equality, the empowerment of women, human rights and human development in the delivery of country programme outputs?
• To what extent have UNDP practices, policies, processes and decision-making capabilities affected the achievement of the country programme’s outcomes?
• To what extent did UNDP engage or coordinate with beneficiaries, implementing partners, other United Nations agencies and national counterparts to achieve outcome-level results?

Sustainability

• To what extent did UNDP establish mechanisms to ensure the sustainability of the country programme outcomes? • To what extent do national partners have the institutional capacities, including sustainability strategies, in place to sustain the outcome-level results?
• To what extent are policy and regulatory frameworks in place that will support the continuation of benefits?
• To what extent have partners committed to providing continuing support (financial, staff, aspirational, etc.)?
• To what extent do mechanisms, procedures and policies exist to carry forward the results attained on gender equality, empowerment of women, human rights and human development by primary stakeholders?
• To what extent do partnerships exist with other national institutions, NGOs, United Nations agencies, the private sector and development partners to sustain the attained results?

3)  Inception report structure / outline

Following the contracting, the MTE external team will prepare a brief inception report that contains:

• Background and context illustrating the understanding of the programme and outcome areas to be reviewed;
• MTE objectives, purpose and scope. A clear statement of the objectives of the MTE and main aspects or elements of the initiative to be examined;
• MTE review criteria and questions. The criteria the MTE will use to assess programme performance and rationale. The stakeholders to be met and interview questions should be included and agreed as well as a proposed schedule for field site visits and / or virtual interviews;
• Clear approach, methodology and data collection tools on the basis of which the programme will be reviewed with a description of data-collection methods, sources and analytical approaches to be employed, including the rationale for their selection (how they will inform the MTE) and their limitations; 
• Clear review framework including a detailed list of review questions for each of the thematic areas/outcomes, sources/methods of data collection, and list of key stakeholders and other individuals, who should be consulted, developed with the assistance of the UNDP India team, including main line of interview questions for each of the stakeholder groups;
• Details of how cross-cutting issues required to be evaluated, considered and analyzed throughout the MTE process. The description should specify how methods for data collection and analysis will integrate gender considerations and ensure inclusion of diverse stakeholders, including the most vulnerable where appropriate;
• Schedule of key milestones, list of activities, deliverables and responsibilities / division of tasks including the MTE stages (desk review, data collection, data analysis and reporting);
• Outline of the MTE draft / final Report with table of content, tables and Annexes;
• List of documents that will be reviewed and consulted by the review team;
• List of stakeholders to be consulted during the MTE process.

4)  MTE final report structure / outline 

The key product expected from this MTE is a comprehensive analytical report in English, meeting the required criteria outlined for objective, scope, structure and content. The report should be strategic, future-oriented, results-driven and analytical and should, at least, include the following content:

• Table of contents
• List of acronyms and abbreviations
• Executive summary
• Introduction and overview (what is being evaluated and why)
• Description of interventions being evaluated.
• Scope and objectives; description of the review methodology; data analysis
• An analysis of the situation in line with the scope of the MTE
• Outcome wise key results and findings including a tabulated summary of strategic progress overview against each CPD Output under each Outcome area; and an adjusted / aligned CPD Results Framework table
• Overall Key findings and Conclusions
• Recommendations 
• Lessons learned
• Annexes include but not limited to: TOR, field visits, list of partners and stakeholders consulted and interviewed, questionnaires and other tools followed, list of documents reviewed, list of UNDP managers and project staff consulted and interviewed, and Others as deemed necessary by the MTE team to support the MTE findings and recommendations in the final MTE Report

5)  Response to comments matrix

Chapter and section number: ……………………………………………………………………………………………………….

Paragraph number / line number: ……………………………………………………………………………………………………….

Comments: ……………………………………………………………………………………………………….

MTE team response and /or action taken: ……………………………………………………………………………………………………….

6)  List of key documents to be reviewed. This would include a review of inter alia

• UNDP Strategic Plan (2018-2021)
• UN Sri Lanka Sustainable Development Framework 2018-22
• UNDP Sri Lanka Country Programme and results framework 2018-22
• Country Programme Review Report 2013-17
• UN Framework for the Immediate Socio-Economic Response to COVID 19
• UNDP Corporate COVID Response and Recovery Framework
• Project documents and annual work plans 
• Results Oriented Annual (Programme) Reports (ROAR) 2018 and 2019
• Project mid-term reviews and evaluation reports for the period under review
• Project Steering Committee minutes
• Project quarterly and annual progress reports
• Donor contribution agreements and donor progress reports
• Other documents as relevant