Final Project Evaluation “Promoting protection, well-being and social cohesion for vulnerable groups in Ukraine”

1. Background

With the aim of increasing greater social cohesion and strengthening psychosocial well-being in conflict affected populations and vulnerable host communities, the purpose of this 48-months project was to increase and strengthen Ukrainian Red Cross Society’s (URCS) capacities in four areas. By focusing on supporting URCS to improve its volunteer recruitment, care and management practices, URCS branches have increased organizational capacities and improved staff and volunteer management. By improving the capacity of URCS staff and volunteers to deliver psychosocial support (PSS) and social cohesion activities, the project strengthened the pscyhosocial and social cohesion framework of URCS. By strengthening psychosocial capacities and technical PSS skills of URCS staff and volunteers, the Project provided relevant, timely and quality PSS and social cohesion activities to vulnerable populations. And finally, by enhancing humanitarian diplomacy and advocacy skills of URCS, the focus was to strengthen the capacity of URCS to identify and engage decision makers on protection concerns. The direct beneficiaries targeted are 62 900 people (29 414 Male; 33 486 Female) living in conflict affected and host communities in Ukraine, including URCS’ staff and volunteers. The Danish Red Cross (DRC) with support from the MFA of Denmark implemented the project.

The project faced a range of implementation challenges, including the impacts of the COVID-19 pandemic. Wherever feasible, the project adapted to the changed context through a proactive and need-based planning and implementation modality. Two mid-term evaluations were undertaken to review two Project components and facilitate the learning process.

In order to be able to assess achievements of the entire project Danish Red Cross is commissioning this final external evaluation. The final evaluation is planned in October-December 2021 – during the last months of the completion of all activities of the project and two months before completion of the project itself (in December 2021).

2. Purpose, Objectives and Scope

2.1 Purpose

The main purpose of this final evaluation is to assess and document project objectives and outputs as well as identify lessons learned in terms of project’s approach and activities. This will allow The Ukrainian Red Cross Society and Danish Red Cross to consider the final evaluation’s findings and apply its key recommendations, in terms of approach and activities, into the formulation of new technical capacity development projects in the future.

2.2 Objectives

The project’s final evaluation has a joint focus on accountability and learning. More specifically, final evaluation will:

A. Assess the project’s achievements vis-à-vis its intended objectives and outputs as set out in the project logical framework with a special focus on Output 4 and the recommendations of the mid-term reviews for the Outputs 1-3;

B. Assess the efficiency of the project’s coordination model for achieving the goal and the immediate objective;

C. Identify lessons learned and key recommendations from the project that could feed into program formulation in future.

2.3 Scope

The evaluation will cover the project period in its entirety both in terms of timeline (1^st January 2018 – 31^st December 2021) and geographic coverage (Luhansk GCA, Donetsk GCA, Kharkiv, Dnipro, Zaporizhzhia, Kyiv oblast, Odesa, Poltava, Ternopil, Kyiv City, Lviv, Kherson, Mykolaiv, and Vinnytsia). The evaluation will focus on the complete range of engaged stakeholders primarily, direct beneficiaries, local communities, protection and advocacy partners, URCS staff/volunteers.

2.4 Research questions

The final evaluation shall comprise but not necessarily be limited to the following areas/ questions:

Relevance

· What could be improved in the project’s targeting mechanisms, data dissagregation (by gender, age, population group, location) and the vulnerability assessment tool?

· To what extent have the targeted branches reached to conflict-affected populations (CAP), including internally displaced persons (IDPs), and to host community members?

· Have the needs assessments been helpful to achieve the immediate objective?

· To what extent are the specific interventions relevant to the needs and priorities of the CAP and host communities?

· What was the impact of Community Engagement and Accountability activities within the project, and as part of mid-term project evaluations?

Effectiveness

· To what extent have the project objectives and outputs been achieved?

· To what extent have the project activities been of the necessary technical quality and timeliness?

· To what extent has the advocacy capacity of selected Ukrainian Red Cross Society branches improved by the project?

· What conditions facilitated the achievement of the immediate objective? What conditions were obstacles to achieving the immediate objective?

Efficiency

· To what extent did the project implement activities in the most efficient and impactful way?

· What could be improved in project’s policy analysis and prioritization of local protection concerns, stakeholders mapping and targeting, advocacy interventions planning, collaboration and cooperation with authorities, coalition building with partners, campaigning, and gathering evidence?

· To what extent did the project successfully digest the lessons from the mid-term evaluation exercises?

Sustainability

· To what extent are the branches confident in delivering PSS services to the conflict-affected population and sustaining social cohesion at the local level?

· To what extent are linkages established for communities and authorities to engage in discussion and cooperation with regard to protection issues likely to sustain beyond the end of the project?

· How could DRC better support URCS’ ownership for delivering high-quality sectorial programs?

3 Methodology

This final external evaluation shall be carried in close collaboration between URCS and DRC. The review will involve a combination of methods as follows:

· Secondary data review (relevant documents and reports, advocacy and communication materials, etc.).

· Key Informant Interviews (remote, semi-structured) on the chain of results and project coordination and learning should be conducted with staff, volunteers, and external experts: local authorities, local social services providers, and civil society representatives.

· Knowledge, Attitude, Practices (KAP) survey (end-line; semi-structured interviews focused on relevance, effectiveness, and efficiency of specific project activities; sample size of 300 beneficiaries).

· Focus Group Discussions (FGD) with beneficiaries in eastern and non-eastern project locations on the chain of results, relevance, effectiveness, and efficiency of specific project activities

· Structured survey via web-forms with URCS staff and volunteers

· The Consultant should focus on the key questions (section 2.4) while developing questionnaires and conducting interviews and discussions.

· Inception report with proposed methodology and timeframe shall be provided and agreed with the DRC team.

· A presentation containing key findings and a draft report will be sent to DRC for comments and further discussion. After receiving the comments, the Consultant will finalize the report and submit it to DRC.

4 Roles & Responsibilities

The final evaluation team will be led by the Сonsultant in close collaboration with the Local Consultant, DRC and URCS.

DRC shall be responsible to:

· Facilitate access to documentation and contacts lists

· Support the coordination and the debriefings

· Facilitate logistics, linking with URCS project staff and volunteers.

· Make payments in UAH as per the signed contract with the Consultant within two weeks of the submission of the invoice and the final evaluation report.

Consultant shall be responsible to:

· Lead the evaluation team and evaluation process

· Prepare and submit an inception report

· Conduct a debriefing and/or PowerPoint presentation summarising the findings and recommendations of the evaluation.

· Present the initial findings to DRC CO team for the suggestions and feedback.

· Prepare and submit the draft report and final report to DRC as per the DRC final evaluation template.

URCS shall be responsible to:

· Recruit the Local Consultant/team

· Facilitate access to documentation and contacts lists of beneficiaries, staff, and volunteers

· Identify URCS’ staff/volunteers engaged in data collection

· Facilitate the logistics

Local Consultant shall be responsible to:

· Participate in development of methodology and schedule

· Prepare data-collection tools and techniques

· Instruct the URCS’ staff/volunteers engaged in data collection

· Conduct the data collection and the data analysis

· Contribute to the consolidation of the draft final report.

5 Schedule

The assignment should last up to 25 working days and get started in October and completed by 20^th December 2021. Due to COVID-19 potential travel disruptions, potential travelling in country will be discussed between DRC and the consultant. The exact timeframe will be agreed in joint consultation between DRC and the national Consultant.

6 Skills & Qualifications

The selection of the external evaluation consultant/s will be based on the qualifications outlined below:

· Sound understanding of Social Cohesion, Psychosocial Support, and Organizational Development fields;

· Sensitive to the complexities and constraints associated with Red Cross and Red Crescent mandate;

· Excellent written and spoken English skills required, as well as excellent analytical and presentation skills;

· Knowledge and experience in Ukraine would be an advantage.

7 Budget

The Consultant is asked to submit a budget proposal outlining the costs for working days (fee), along with a tentative work plan including the working days for the main activities and milestones.