RFP-005EBL20-MADAD End of Project Evaluation Consultancy

Terms of Reference

End of Project Evaluation Consultancy

Strengthening resilience and income generating opportunities and supporting early recovery and stabilisation in areas of return in Iraq

Project title

Strengthening resilience and income generating opportunities and supporting early recovery and stabilization in areas of return in Iraq

Summary:

Since January 2019, the Cash Consortium in Iraq started implementing a recovery portfolio thanks to Madad funding to enhance the economic security and resilience of conflict-affected households through integrated activities focusing on livelihoods, rehabilitation of shelter and public infrastructure, local governance, social cohesion and protection.

Objectives of the action

Overall objective: Conflict-affected households have increased confidence in their economic future and economic and social stability in their locations

SO1. Population participates in community recovery efforts and has improved access to public services and sustainable housing solutions

SO2. Population is more economically resilient and has improved ability to secure livelihoods in the medium/long term

SO3. Population has increased access to information, legal counselling, services and referral pathways

Country Programme

Iraq, MENA region

Location:

Anbar, Diyala, Ninewa and Salah al-Din governorates

Coordinator/Manager:

Helene Bailly, CCI Economic Recovery Coordinator

Alicja Zasadowska, CCI MEAL Specialist

Head of Programs:

Geoffrey Baeumlin, HoP

Leading organisation

DRC

Consortium partners **

MC, NRC, Oxfam

Project title

Strengthening resilience and income generating opportunities and supporting early recovery and stabilisation in areas of return in Iraq

Project duration

24 months

1. Background

Starting in 2014, the so-called Islamic State (IS) advanced through the governorates of Anbar, Diyala, Ninewa, and Salah-al Din, reaching the Duhok governorate. The conflict and subsequent military operations led by the Iraqi Security Forces (ISF), Kurdish forces and militias to retake the captured areas have displaced millions of Iraqis, while others chose or were forced to stay in their communities under the control of IS. As of August 2020, more than 1.3 million people remained displaced from their homes while over 4.7 million people have returned to their areas of origin. In areas affected by the 2014-2017 conflict, the population still encounters damaged or destroyed infrastructure and property; limited access to electricity, water, municipal services, and local markets; and tensions and mutual suspicions among families with contrasting displacement experiences. Requirements for equitable and sustainable return and reintegration of mixed communities are varied but interlinked. Community and intercommunity economies must be regenerated by re-establishing businesses, supply chains and markets, and rehabilitating economic and associated services and infrastructure. Formal and traditional local governance must be inclusive and responsive, and representative of different community groups.

1. Description of the project

The aim of the project is to contribute to increase confidence of conflict-affected households in their economic future and the economic and social stability in their areas of return in Iraq. The CCI has identified a three-pronged approach to contribute to this objective, which fits into the CCI’s broader strategy for early recovery:

a) Empowering populations to participate in community recovery and improve their access to public services and adequate shelter: facilitating a community-based approach to recovery, emphasising transparency, inclusivity, and participation, through establishing and building the capacity of community structures. Community committees will be empowered to organise, prioritise and advocate for the needs of the local population by developing and implementing community action plans. Shared community assets and infrastructure will be rehabilitated, improving public service delivery and creating immediate access to income for vulnerable individuals through low-skill labour-intensive projects. In parallel, the action will support the rehabilitation of damaged shelters for highly vulnerable households. By engaging with local communities to design selection criteria for the rehabilitation programme, the action will mainstream the community-driven approach and foster communal ownership of the rehabilitation process. By providing the technical, financial and material means to rehabilitate shelters, the action will reduce the financial burden on families limiting households’ use of negative coping strategies, such as incurring debt or spending savings, while also stimulating local economies and creating income-generating opportunities for members of the community. The community committees will facilitate identifying citizen concerns and communicating with local/regional/national government officials and strengthen inter-communal cohesion by providing a forum for community dialogue and collaboration around shared challenges.

b) Fostering economic resilience and the ability to ensure livelihoods in the medium/long term: using market-based solutions to strengthen the employability of conflict-affected individuals, identify potential opportunities and placement arrangements to apply these skills, support the incubation and scale-up of MSMEs, and facilitate their access to financial and technical resources.

c) Improved understanding of and access to legal and civil documentation (LCD) and housing, land, and property (HLP) rights: enabling community members to understand and exercise their LCD and HLP rights (including establishing legal ownership and pursuing compensation and restitution) and resolve disputes.

1. Objectives of evaluation

The primary purpose of the impact evaluation is to understand the intended and unintended impact of the project activities on the beneficiaries and the wider community. The evaluation of the specific objectives is to provide data on the performance, impact, and sustainability of project interventions. The findings and recommendations will contribute to a learning process which enables CCI and the partners to draw lessons from their experience to improve the quality of support to conflict affected communities in Iraq.

If it should not be the core of the evaluation, the process should consider the impact of Covid-19 on activities and results of the programme and the impact on the needs of the population

The impact assessment will seek to answer the following lines of inquiry:

a. Impact: What were the positive and negative, intended or unintended, direct, and indirect changes on the different targeted individuals and communities:

● Has the project produced any unintended, positive or negative impact to the beneficiaries or their community?

● Did the program improve the economic viability and profitability of SMEs and did it create additional opportunities for income generation for community members?

● Did the programs provided sustainable incomes for participants who received business grants

● Did participants who received relief support (MPCA/cash-for-work or LGA) in addition to a business grants avoid negative coping mechanisms

● Were the supported household able to refer and rely on governmental safety net programs if sustainable livelihood could not be achieved?

● Did the establishment of community committees provide opportunities for trust-building, interaction and collaboration across lines of division?

● Did the community committees facilitate inter-communal cohesion by providing a forum for representative community dialogue and conflict mitigation across lines of division?

b. Relevance: Was the project appropriate for the needs of the targeted population.

● Was the timely provision of SME/VT amongst the highest priority for the target population?

● Were business grants the most appropriate way to create long-terms income generating opportunities? Is there a difference between start-up and scall up grant?

● Was the training the most appropriate to the need? Is is relevant in terms of educational level of participants

● Was the development of SMEs relevant for both man and woman? Is it relevant for both youth and elderly ?

● Has the project implementation use the set criteria for identification and selection of beneficiaries in accordance with its objectives and CCI policy? If inconsistencies exist, flag recommendations for the selection criteria process in country

● How did the programs supported households who cannot generate income via wage employment or business support? Was the referral to others international aid programs or to government social programs relevant,

● Did the established community committees act as a forum for community dialogue and foster the trust between community members. Are they well-known and perceived by communities as fair and transparent?

● Was the capacity of the committee members strengthened enough to be in position of identifying key advocacy issues and bring them to relevant authorities in order to create changes?

c. Sustainability: Are the benefits of the project likely to continue?

● The action supports a gradual transition from dependence on international humanitarian aid to reliance on sustainable livelihoods or on functioning government social nets and services, depending on the status and capacities of individuals and households. To which extend, the action of CCI on supported SMEs was sustainable? What positively impact the sustainability of the action and what negatively impacted it **

● To what extend are the beneficiaries able to rely on governmental social nets and services after the end of the project?**

● How the availability of and the safe access to local services (NGOs and Governmental) affect the sustainability of SME Grant/VT interventions and complement them (e.g. Female HoHH, persons with disabilities, etc.)

● Were the capacity building efforts conducted among the participants adequate to ensure sustainability of the project (employability, business training etc)?

● Are the community committees members perceived as long-terms interlocutors for community members and authorities.

● How are the dialogue engaged between communities and authorities facilitate the sustainability of community action plan-driven projects?

d. Effectiveness: Did the project achieve its objectives and its results?

● Was the SME/VT amount adequate to be effective to establish a business/ get employment

● What were the persistent/common issues influencing the achievement of the SME/VT objectives (improving employment/ income and/or savings levels of targeted beneficiaries)?

● How has the SME/VT capacity-building workshops provide opportunities for improving intergroup perceptions and relationships and disproving negative stereotypes and reduce prejudice among the trainees?^[1]

1. Methodology and Scheduling

The evaluation requires collaborative and participatory mixed methods approach that was drawn on both existing and new quantitative and qualitative data to answer the evaluation questions.

Quantitative: Quantitative aspects of project aim to measure the project outcomes by using monitoring reports and collecting data from the targeted beneficiaries, ensuring equal gender representation wherever possible.

Qualitative: This is to acquire in-depth information based on the five evaluation areas and around the overall and specific objectives of the program. A suggestion is to use sex and age disaggregated focus group discussions (FGD) and be gender balanced when conducting FGDs (with Community committees members, Cash for work beneficiaries, grantees, trainee graduates of business management and employability training) and Key informant interviews (with Program staff, vocational training centres, companies hosted apprentices)

Evaluation will cover operational areas where Madad project activities are implemented, and population mainly includes the project direct beneficiaries with some non-beneficiaries and market actors. The reason for including non-beneficiaries and market actors is to measure the indirect impact, positive or negative, on overall population in targeted location. This is specific to some activities like infrastructure rehabilitation work, social-cohesion, and other market related interventions, i.e. business grants. Evaluation is also expected to cover mandatory segregation of gender and age. In addition, each intervention outcome indicators will be required to measure throughout the data collection and analysis process.

Scheduling

The evaluation is expected to start on 1^st of February for an estimated duration of 60 working days. the below table outlines and estimated timetable of this evaluation:

Duration

Activity

Evaluation deliverables

Week 1

● Inception meeting between CCI and Consultant to review ToRs and clarify timeframe and deliverables

● Minutes of meetings

Week 1

● Undertake desk review of the relevant program documents.

● Develop an Inception Report detailing the process and methodologies to be employed to answer the evaluation questions.

● CCI to provide feedbacks on the inception report and tools

● Evaluator to incorporate feedbacks and finalize inception report and data collection tools.

● Inception report which covers the detailed methodology, schedule (workplan, timeframe for deliverable) and conceptual framework (understanding of the evaluation and how the evaluation questions will be addressed). All these are expected to be aligned with the project proposal and other relevant project documents.

Week 2

CCI/Evaluator meeting to discuss expectations and logistics

Week 2-3

● Hire Enumerators/Surveyors.

● Train Enumerators/Surveyors; Pre-test data collection instruments.

● Finalize data collection instruments

● Evaluation tools in English and Arabic (HH survey, FGDs, KII) designed and tested

Week 3-5

● Conduct data collection

● Oversee data collection

● Data collection completed in all targeted locations – Weekly reports on data collection to be shared (original field notes for all in- depth interviews conducted)

Week 6

● Encode and analyse data

● Database (raw data)

Week 7

● Prepare draft evaluation report (including a success story and a learning story)

● Draft evaluation report for CCI review

Week 7

● Conduct debrief meeting to present draft report, collect initial feedback from CCI

● Minutes of meeting with CCI to present key findings of the evaluation

Week 7

● CCI to provide detailed feedback to he draft report within 5 working days

Week 8

● Finalize report, produce presentation of findings, and share back with CCI

● Final evaluation reports with comments addressed (including executive summary, methodology, results and supporting analysis, lessons learnt and recommendation) including all raw data, original field notes for all in- depth interviews conducted^[2]

● Presentations of results to CCI partners, TWG and steering committe

* Schedule depends on the movement restrictions and lockdown.

1. Evaluation methodology

The impact assessment methodology will be developed by the Consultant and presented in the inception report, including a detailed impact assessment matrix. The impact assessment should be based on combined quantitative and qualitative research methods. The data collection strategy should include the use of appropriate tools to gain a deeper understanding of the outcomes of the project, including:

a) Desk review of background documents, such as the project proposal, log frame, budgets, annual plans (CCI’s SOPs, understand the selectin criteria, IM process, forms for data collection W1-4) and review existing data, including the market assessment, baseline surveys, minutes of meetings, various M&E data, including monitoring reports, progress reports and donor reports;

b) Quantitative analysis of the data already in CCI’s servers (base-endline) – no new data collection at this point.(W1-4); Based on that analysis doubts will appear, for example why male are less likely to drop in their income, or why CFW beneficiaries in Anbar (or DRC) have lower NCS reduction.

c) Design survey for beneficiaries, partners NGO Staff, other NGOs in area, local officials to answer the questions in the TORs and the ones that came up in the quantitative analysis. (W 4-5)

d) Present desk review findings (including “opportunistic” questions) and the first draft of questionnaires (week 5-6) and agree on additional qualitative questions on surveys.

e) Develop an appropriate sampling design and data collection strategy that endeavors to capture adequate data to answer all evaluation questions raised in the evaluation criteria.

Suggested data collection: Face-to-face interviews with a randomly selected list of beneficiaries selected by the consultant from the database. This must be a randomised purpose sampling to have enough cases from all NGOs and geographical areas. Suggested sample is 120 in-depth interviews(30 per partner and 30 per Gov). The consultant firm will familiarize and understand each of the individual cases selected for in-depth interviews. Given COVID movement restrictions the consultant will have research team in each governorate to secure data collection and enough preparation and time dedication by case.

f) Interviews with other humanitarian actors (including cluster leads) implementing similar programmes in Iraq and if possible, also representatives from the Iraq government/authorities.

1. Evaluation deliverables

The consultant will deliver, based on an agreed upon work plan:

a) Inception report which covers the detailed methodology, schedule and conceptual frame-work, key impact evaluation questions. All these are expected to be aligned with the project proposal and other relevant project documents.

b) Evaluation tools in English and Arabic (designed and tested).

c) Data collection completed in all targeted locations

d) Draft evaluation report for CCI review

e) Final evaluation reports with comments addressed (including executive summary, key recommendations and supporting analysis)

f) Presentation of results to CCI partners, TWG and steering committee etc (3 presentations to be planned)

1. Intended Use of Review

The primary users of the impact assessment findings will be CCI Management Unit, CCIs partners and their country programmes, specifically the livelihoods and social cohesion teams. The impact assessment will be published online as part of efforts to share learning globally among CCIs internal and external partners and the livelihoods sector.

1. Consultant qualifications/ Selection Criteria

The successful applicant will be a reputable local or international organisation/individual with demonstrable presence in Iraq, and experience in conducting large-scale field data collection. Some of the key requirements are:

a) 5 to 8 years of experience in emergency and early recovery context (Clearly indicate in the proposal, the number of months/years for each person working on the project)

b) Proven experience in conducting programs evaluations or research (at least 6 previous projects) preferably in Iraq or Middle east and on Recovery projects (Clearly indicate in the proposal, the number of project and months/years for each person working on the project)

c) 2 to 3 years of experience in cash, economic recovery, livelihood programming (Clearly indicate in the proposal, the number of months/years for each person working on the project)

d) Demonstrated experience in both quantitative and qualitative data collection and data analysis techniques, especially in emergency operations.

e) Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner;

f) Demonstrated experience of working both in urban and rural settings with strong work exposure on IDPs, refugees, returnees, and host communities;

g) Access to / presence of a field team for data collection in Iraq is preferable;

1. Documents to be included in the proposal:

a) Rationale (Developed understanding of the evaluation purposes and objectives)

b) Developed methodology to be used including sampling methodology, data collection, data storage, data protection, data analysis

c) Outlines of the inception and final evaluation reports

d) Sample of previous inception and final reports

e) Sample of training material for enumerators

f) CV of team leader and team members (please fill the application form (ANNEX F) for each team member)

g) Financial proposal including all costs (detailed per items) in euros.

1. Proposal Schedule of Payments.

The consultant shall be paid the consultancy fee upon completion of the following milestones.

● 20 % after adoption of the inception report.

● 30 % after presentation of the draft report.

● 50 % after the approval of the final report.

Technical criteria #

Technical criteria

Weighting in technical evaluation

**

1

Technical quality of the research proposal

50%

1.2

Demonstrated understanding of the methodology and requested deliverables, all important components of the ToR are sufficiently addressed and considered

20%

1.2

Strong methodological approach to data (data collection, data storage, data analysis). The methodology is explained in detailed

15%

1.3

Style, language, sophistication, and presentation

15%

2

Expertise and access

30%

2.1

Demonstrated previous experience of working in Middle-East (prior experience in Iraq is preferable)

10%

2.2

Demonstrated experience in conducting final evaluation and research pieces focusing on relief and recovery programming

15%

2.3

Must have access to or presence of a field team for data collection in Iraq and a team of researchers available in country of assignment with English language and Arabic language skills;

5%

3

Personal Qualifications

20%

3.1

A BA degree on social sciences or other relevant subjects

10%

3.2

Must have a minimum of 5 years of experience with assessments or monitoring and evaluation and experience in undertaking assessments in a fragile context

10%

Total Maximum Score

100%

[1] This question is based on the contact theory to improving intergroup perceptions and relationships, which stipulate that sustained interactions within a livelihood/social cohesion project and capacity-building workshops will steadily improve perceptions, disprove negative stereotype and reduce prejudices, replacing them with positive experiences and mutual understanding and build positive relationships within project beneficiaries/trainees. The Contact Hypothesis approach, which stated that intergroup attitudes will improve, and prejudice will decrease if people get to know one another under defined conditions, dismantling the negative stereotypes and misperceptions that drive prejudice, replacing them with positive experiences and mutual understanding.

^[2] CCI has sole ownership of all the final data and any findings shall not be reproduced or shared without the written permission of CCI.