FINAL PROJECT EVALUATION

TERMS OF REFERENCE

PROJECT EVALUATION

Safety and dignity for Syrian refugees and vulnerable Jordanians with disabilities

Project implementation dates: April 2018- March 2021

1. General information

1.1 About Humanity & Inclusion

Outraged by the injustice faced by people with disabilities and vulnerable populations, we aspire for a world of solidarity and inclusion, enriched by our differences, where everyone can live in dignity.

Handicap International changed its name and became « Humanity & Inclusion ». Humanity & Inclusion (HI) is an independent and impartial aid and development organization with no religious or political affiliations operating in situations of poverty and exclusion, conflict and disaster. We work alongside people with disabilities and vulnerable people to help meet their essential needs, improve their living conditions and promote respect for their dignity and fundamental rights.

HI has offices in 63 countries; for further information about the association: www.hi.org

1.2 About Humanity & Inclusion in the country/region

HI has been operating in Jordan since 2006, implementing projects ranging from the provision of comprehensive rehabilitation services, directly, through service providers or through a community-based approach; inclusive local development, disability movement strengthening through capacity building of Disabled People’s Organizations and multi-stakeholder dialogue, among others.

In line with the Jordan Response Plan and the 3RP Regional Refugee and Resilience Plan, HI has always sought to integrate resilience and sustainable elements throughout its interventions, by adopting a twin-track approach which seeks to simultaneously address the direct needs of beneficiaries all the while engaging in dialogue with them, with local authorities and local actors to empower them to claim and address their rights to access these services. And to mainstream disability in policies, programs and services.

Since late 2015 and in line with the evolution of the Syrian crisis into a protracted one, HI adjusted its strategy of intervention to progressively shift from direct to indirect service provision through partners. As of July 2017, the Jordan mission comprises a single entity seeking to ensure the continuity between emergency response, resilience and development.

1.3 The current context in Jordan and the impact of the COVID-19 crisis on the project intervention

Since the beginning of the COVID-19 outbreak, the Jordanian government has followed the recommendations and updates provided by the WHO. A series of preventive and control strategies at the national levels have been implemented in order to limit the spread of COVID-19 inside Jordan.

The Jordanian government has implemented strict rules and measures relating to travel, education, religious and social events, as well as working within various industries including INGOs. On the 17^th of March 2020 the government put into effect the national defence law in order to mitigate the spread of COVID-19 in Jordan. On the 20th of March 2020,a nationwide curfew was declared with strict rules on individuals’ mobility and fines for violations.

The COVID-19 crisis and the protective measures put in place by the Jordanian government significantly impacted HI and its’ partners’ ability to implement the intervention and reach beneficiaries during the months of March, April and May 2020. However, HI was able to reach the most vulnerable beneficiaries through tele rehabilitation and caregiver messages while waiting for the lockdown to be lifted. The lockdown was gradually lifted in May 2020 and most activities were able to resume. In September 2020 the number of positive COVID-19 cases started to increase again which forced HI to implement a number of mitigation and protective measures such as the suspension of activities in isolated areas and areas that had seen significant increases in cases. However, most activities are still being implemented either remotely or in person albeit with strict adherence to the COVID-19 protective measures (mask wearing, respiratory etiquette, social distancing, limited gatherings etc.) .

2. Context of the evaluation

2.1 Presentation of the project to be evaluated

The project due for review is funded by The Foreign, Commonwealth and Development Office (FCDO) as outlined below and has been implemented in Amman governorate as well as in Zaatari and Azraq camps. The project aims to contribute to reducing the vulnerability and to promote the dignity of crisis-affected Syrians and the most vulnerable Jordanians with functional limitations. The project includes a rehabilitation, Early Detection/Early Intervention, Inclusion and Community Based Rehabilitation component and is implemented with various stakeholders at the authority, service provider and community levels. The project has one impact, three outcomes and three outputs as outlined in the table below:

Project title

Safety and dignity for Syrian refugees and vulnerable Jordanians with disabilities

Implementation dates

April 2018 to March 2021

Location/Areas of intervention

Amman, Zarqa (Azraq camp) and Mafraq (Zaatari camp)

Operating Partners

Project title

Safety and dignity for Syrian refugees and vulnerable Jordanians with disabilities

Implementation dates

April 2018 to March 2021

Location/Areas of intervention

Amman, Zarqa (Azraq camp) and Mafraq (Zaatari camp)

Operating Partners

Noor Al-Hussein Foundation (NHF)

Service providers

Ministry of Health (1 Hospital, 21 Primary Health Centres), Ministry of Social Development (Al manar centre, 1 foundation) and UNRWA (2 Community Development Centers).

Project Budget

5,000,000. GBP

**

Noor Al-Hussein Foundation (NHF)

Service providers

Ministry of Health (1 Hospital, 21 Primary Health Centres), Ministry of Social Development (Al manar centre, 1 foundation) and UNRWA (2 Community Development Centers).

Project Budget

5,000,000. GBP

Project Impact

Project Indicator

Reduce the vulnerability and promote the dignity of crisis-affected Syrians and most vulnerable Jordanians with functional limitations

% of beneficiaries reporting improved social participation

Outcome

Syrian crisis affected most vulnerable population, with an enhance focus on children 0-8, have access to quality comprehensive rehabilitation services and other essential services through strengthened Jordanian service providers and a more inclusive humanitarian response

Outcome 1

% of beneficiaries showing improvement in functional independence

Outcome 2

% of children who are successfully referred after detection

Outcome 3

# of humanitarian working groups within the JRP demonstrating progress in disability inclusiveness

Output 1:

Indicators:

Vulnerable populations with functional limitations are identified, have their needs assessed and receive comprehensive rehabilitation services and/or complementary support through referrals to external existing services

1.1. # of most vulnerable people who are identified and whose needs are assessed

1.2. # of vulnerable people who received comprehensive rehabilitation services

1.3. # of vulnerable persons with rehabilitation needs using services from local rehabilitation providers covered by the rehabilitation safety net

1.4. % of trained staff able to implement users centered approach in their rehabilitation practices

1.5. # of most vulnerable people referred to external services

Output 1 Activities:

1.1: Definition of capacity building action plan including coaching mechanism for local rehabilitation service providers

1.2: Setting of community based rehabilitation network in the camps and host communities

1.3: Identification of the most vulnerable Syrian-crisis affected people with functional limitations in camps and host communities, assessment of their basic and specific needs and definition of action plan

1.4: Provision of comprehensive quality rehabilitation services in the camps and host community based on beneficiary tailored rehabilitation plan according to needs including donations for devices , discharge kits

1.5: Provision of necessary donation/equipment (devices, discharge kits, etc.) to the beneficiaries based on their individualized action plan.

1.6 Provision of education sessions to beneficiaries’ caregivers in order to enhance the success of the rehabilitation process

1.8: Continuous coaching and support to community based volunteers on rehabilitation and social participation of people with disabilities

Output 2:

Indicators:

Children aged 0 to 8 with impairments are identified and receive timely intervention to reach their full potential.

2.1. # of children (Birth -8 years old) assessed in targeted area by PHC trained staff

2.2. # of children screened positive for impairments enrolled in EI programs

2.3. # of PHC whose staff are trained on early detection protocol

2.4. % of children with EI plans who complete at least 70% of the set objectives.

2.5. # of families participating in EI for their children

Output 2 Activities:

2.1: Deliver training on early detection and referral to staffs of PHC and other peripheral facilities

2.2: Develop ITC Early Detection and Early Intervention (EDEI) tools for ED screening and EI to support monitoring

2.3: Support the formation of peer-led groups for family members/caregivers and provide ongoing guidance

2.4: Create and support ED/EI committees to support community based network and ensure sustainability of community interventions

2.5 : Case management and monitoring of external referral

2.6: Develop basic skills of CBR workers on assessment and management of children enrolled in early intervention and train rehabilitation staff

2.7: Training of family members/caregivers on how to promote development of children at home

2.8: Meet the cost of services for children and provide stimulation equipment and materials (positioning, toys etc. ) to promote development

Output 3:

Indicators:

The visibility and inclusion of refugees with disabilities in the response to the Syrian crisis is enhanced

3.1. # of community members who are sensitized on factors leading to exclusion and attitudinal barriers.

3.2. # of projects which reflects improved inclusive practices after HI tailored coaching to the service provider

3.3. # of awareness and training sessions targeting management level organizations members of various humanitarian working groups

3.4. # of leaders that are empowered to support persons with disabilities to expressing their needs both in community settings and camps to influence service provision

Output 3 Activities:

3.1: Support Disability Task Force (DTF) to define some common positions for representing disability in the working groups, attendance to the working groups and regular DTF meetins

3.2: With key humanitarian actors from other sectors, organize training sessions at management level to support the institutional development towards inclusion

3.3: Make propositions at the revision of the Jordan Response Plan to account for disability aspects within the strategic planning

2.2 Justification of the Evaluation

The evaluation aims to assess the achievement of the project outputs and outcomes, assess the impact of the intervention on the beneficiaries, stakeholders and partners of the project as well as to inform programme design of the next phase of the project. HI is currently designing the next phase of the intervention and the recommendations and findings of this evaluation will help guide and shape the intervention modalities and framework of the next phase of the project.

Who will take part in and benefit from the findings of the evaluation?

This project is implemented at three levels; the authority level, the service provider level and the community level. At the authority level, HI works closely with the MoH and MoSD to strengthen the rehabilitation system in Jordan as well as to strengthen early detection and early intervention in Jordan. As part of this project and in collaboration with the MoH and WHO, HI has helped design and develop Jordan’s national rehabilitation strategy which will be endorsed by the MoH in October 2020. In addition, HI works closely with the Women and Child Health Care Directorate to reinforce and strengthen early detection of impairments and disabilities in young children as well as with the rehabilitation department of the MoH to strengthen the rehabilitation system in Jordan. In addition, HI works closely with the MoSD to establish and strengthen the early intervention system and service provision in Jordan. At the service provider level, HI provides technical support and capacity building to health staff and specialists and contributes to improving the accessibility of rehabilitation and EDEI services at the service provider level through supporting accessibility work/infrastructure and equipping the centres to enhance the quality of service provision. At the community level, a network of 150 community based rehabilitation volunteers ensures community outreach and identification of beneficiaries through home visits, assessments, awareness sessions, referrals and home based rehabilitation sessions.

All stakeholders of the project including MoH, MoSD, health staff, specialists, beneficiaries, HI staff and volunteers will be involved in the evaluation of the impact of the project intervention. The participation and feedback of the project stakeholders will enable the evaluator team and HI to use the findings and results of the evaluation to inform the design of the next phase of the intervention, respond better to beneficiary needs as well as the needs of the MOH and MoSD, improve HI’s intervention and accountability towards the project stakeholders.

3. Objectives of the evaluation

3.1 Overall objectives and expectations of the evaluation

The final evaluation aims to assess the effectiveness, level of change and sustainability of the project approach / interventions as well as the level of the stakeholders’ participation including the authorities (MoH, MoSD), Service providers and beneficiaries. Furthermore it aims to provide recommendations to inform the design of the next phase of the intervention in order to improve access to and quality of services provided to persons with disabilities in Jordan. In addition, the evaluation shall measure the impact of the intervention on beneficiaries, stakeholders and partners of the project.

Eventually, the final evaluation process will mobilize all project stakeholders including NHF, MoSD, MoH and CDCs, staff and beneficiaries.

3.2 Specific objectives

• ;

• Assess the effect of the COVID-19 crisis on beneficiaries and the effectiveness of HI’s remote response during the national lockdown in 2020 as well as HI’s adjustment of activities since the onset of the COVID-19 crisis.

• Assess the complementarity/comprehensiveness of services between rehabilitation, CBR and EDEI;

• Effectiveness of the community component to achieve the intended objectives;

• Develop specific recommendation/good practices on the project design, approach and implementation based on the results of the evaluation.

3.3 Evaluation criteria and evaluative questions

3.3.1 Efficiency

• How cost effective the project has been with regard to resource utilization such as assistive technology, construction work and equity fund?

• How does the project adapt to evolving needs and risks (constraints and opportunities)?

• To what extent did the structural changes of the team contribute towards its efficiency?

3.3.2 Sustainability

• Have mechanisms been put in place to ensure the sustainability of project results? Are they adequate? Why?

• How has the post-project phase been anticipated and planned?

• How sustainable is the equity fund? What could be the alternatives in the long run with specific focus on community solutions?

3.3.3 Changes

• Is there any unintended impact of the project on the target group and stakeholders?

• How has the project contributed to the development of the capacity of the beneficiaries to carry out their tasks in an environment of change in terms of individual learning?

• How did the project adapt to the COVID-19 crisis? How effective was this in responding to beneficiary needs?

3.3.4 Co-operation

• Has HI been successful in building the capacities of MoH and MoSD staff to improve access to and quality of services provided to persons with disabilities in Jordan?

• Has the partnership with the ministries been effective and what elements or factors have challenged this partnership and will need further attention in the future?

• Has HI been able to successfully transition responsibilities and project elements to the ministries?

• Does a preliminary partnership analysis of implementation and downstream partners were enough?

• To what extent has the partnership strengthened organizational capacity and promoted knowledge exchange among all partners?

3.3.5 Capabilities

• How has the project helped build the capacities of local stakeholders to ensure their autonomy at the end of the project?

• To what extent has the project team and partners developed the skills necessary for implementing the project?

• In which ways has the project contributed towards internal and external learning dynamics?

3.3.6 Accountability

• How has the project ensured accountability towards the affected populations?

• To what extend have the accountability mechanisms put in place been effective in ensuring accountability towards beneficiaries?

• Has HI actively seeked feedback from beneficiaries and has HI provided timely information to beneficiaries either directly or through partners?

4. Evaluation methodology and organization of the mission

4.1 Collection methodology

Evaluation methods should be rigorous yet at all times proportionate and appropriate to the context of the project intervention and will be discussed with HI steering committee. The project works across various governorates and sites in Jordan, though all at day-trip distance from Amman, site visits will be arranged in line with the methods selected. Due to the current COVID-19 crisis and the rising number of positive cases in Jordan, parts of the evaluation process may have to be conducted online and the entire exercise may have to be done remotely. HI asks applicants to please provide a plan for conducting the evaluation remotely in case in person activities cannot be carried out.

The evaluation will be based on primary and secondary data, in particular documentation directly related to the project – proposal, narrative and financial reports, monitoring datasets, surveys, mid-term evaluation, financial information, other documents produced to analyze individual project components and inform decisions regarding the course of the activities. Other contextual and related information by other organizations will also be made available. Finally, key stakeholders involved in the project at different stages will be mobilized for the evaluation – in addition to the whole current project team, former senior management and experts, main representatives of partners and finally volunteers and beneficiaries.

HI values the contributions of the evaluator towards proposing appropriate, innovative, and robust methods of evaluation. Some basic methodological requirements, however, are that:

• The evaluator should try to make tools and evaluation processes of the sample as inclusive as possible for persons with disabilities as much as they possibly can.

• The evaluation must be a participative and interactive process.

• The evaluation should consist of a series of evaluation tasks that shall be implemented throughout the duration of the evaluation. It will allow to build evidence using a logical sequence of activities.

• The methodology should use qualitative data collection techniques and analysis.

• The proposed method should also describe how crosscutting issues of gender and age addressed and incorporated throughout the various stages of the evaluation.

4.2 Actors involved in the evaluation

Portfolio

Role

FCDO-Humanitarian Programme Manager

They are the project funders and periodically review its implementation giving key recommendations

HI Project Managers (Rehabilitation, EDEI, CBR and Inclusion)

Your day to day interlocutors regarding the evaluation process including field work, meetings, FGDs etc

NHF operational staff

Provide support for and information on camp related activities

Regional Technical Unit (Regional health and rehabilitation technical specialists, Regional community based rehabilitation specialists)

Responsible for giving technical guidance of the evaluation process including validating the evaluation methodologies and report

Operations Manager

Oversees and monitors all operation aspects of the evaluation alongside the technical team

4.3 Organization of the mission

The Evaluation steering committee will be responsible for validating the proposed evaluation methodology after the closure of the selection process. In addition, the steering committee and FCDO must approve the inception report and evaluation methodology prior to the commencement of any fieldwork or any other substantive work.

HI will review the proposed evaluation methods by the evaluator after the closure of the selection process.

The steering committee members are:

• Operation Manager;

• Regional MEAL Manager;

• Operation Logistics Manager;

• Regional Health and Rehabilitation Technical Specialist;

• Jordan Finance Manager;

The evaluation should consists of 8 phases:

1. Preparatory phase (briefing with HI team, document review, appreciation-review of the evaluation feasibility), sampling, preparation of data collection tools, logistics arrangements.

2. Kick – off meeting leading to the inception report presenting the proposed evaluation methodology includes the questionnaire completion and tools for the use of data collections.

3. Fieldwork – data collection with partners and beneficiaries.

4. Communication between the steering committee and the evaluator on the final report

5. Development of the final report based on the feedback of all the reviewers.

6. Organize workshop for debriefing and presentation of the results orally to the steering committee and Project Managers, includes present the step of filling the end of evaluation form as presented in the standard process.

7. Data analysis and presentation of preliminary findings (meeting with relevant stakeholders to present analysis, conclusions and recommendations).

8. Validation of the final report on the basis HI quality checklist and according to the steering committee decision.

5. Principles and values

5.1. Protection and Anti-Corruption Policy

The following are the HI institutional polices that should be adhered by the external evaluator and his/her team.

Code of Conduct**

*Protection of beneficiaries from sexual exploitation, abuse and harassment*

Child Protection Policy**

*Anti-fraud and anti-corruption policy*

5.2. Ethical measures

As part of each evaluation, HI is committed to upholding certain ethical measures. It is imperative that these measures are taken into account in the technical offer:

o Guarantee the safety of participants, partners and teams: the technical offer must specify the risk mitigation measures.

o Ensuring a person/community-centred approach: the technical offer must propose methods adapted to the needs of the target population (e.g. tools adapted for illiterate audiences / sign language / child-friendly materials, etc.).

o Obtain the free and informed consent of the participants: the technical proposal must explain how the evaluator will obtain the free and informed consent and/or assent of the participants.

o Ensure the security of personal and sensitive data throughout the activity: the technical offer must propose measures for the protection of personal data. **

*

*These *measures may be adapted during the completion of the inception report.

5.3. Participation of stakeholders and beneficiaries

Mention the involvement of stakeholders and beneficiaries in the evaluation:

o Partners are involved in the survey and/or qualitative data collection methods as they are the main direct service providers of the intervention, which they responsible to receive and provide the services to the beneficiaries.

o Consultation of beneficiaries in the construction of the tools should be considered and adhered by the evaluator.

o Consultation and involvement of beneficiaries in the data collection, analysis and reporting is recommended.

5.4. Others

It is essential that the process of data collection, as well as storage of data, be supported by careful ethical practice, including informed consent, anonymity and confidentiality, no-harm and protection of data and data storage. Informed consent needs to include awareness of the evaluation data collection process and that the evaluation report may be published and publicly disseminated. Extra precaution must be taken in involving project beneficiaries considering the sensitivity of the thematic issues tackled by this project. To protect the anonymity of communities, partners and stakeholders’ names or identifying features of evaluation participants (such as community position or role) will not be made public.

The evaluator should uphold and respect the following ethical principles:

• HI’s protection policies (child protection and protection from sexual exploitation, abuse and harassment …etc.)

• Integrity (respect of gender sensitivity issues, especially when performing interviews/focus groups, religion and beliefs, and local norms).

• Anonymity and confidentiality.

• Independence and objectivity.

• Veracity of information.

• Coordination spirit.

• Intellectual property of information generated during and by the evaluation (including report and annexes) will be transferred to HI and donor.

• Quality of report and respect for timelines. Should the quality of the report be manifestly below the contract.

6. Expected deliverables and proposed schedule

6.1. Deliverables

· An inception report of no more than 15 pages specifying the proposed methodology for answering evaluation questions and a detailed action plan for the evaluation process.

· A presentation document presenting the first results, conclusion and recommendation, to be presented to the steering committee.

· A final evaluation report of approximately 25 – 30 pages maximum and the following contents:

o Table of contents.

o Abbreviations list

o Executive summary (that can be used as stand- alone document).

o Brief on General Jordanian context at the design and implementation phase.

o Introduction that includes evaluation objectives, methodologies, the used techniques, and limitations of the evaluation where relevant.

o Presentation of the evaluation analysis and findings, covering the five focus areas (Efficiency, Change, Sustainability, Capacity, and Cooperation).

o Conclusion and recommendations with a clear relationship between them.

o Report annexes include, The Final Evaluation Term of Reference; The Data collection tools; list of the people met; list of documents and bibliography as well as composition evaluation form.

The final report should be integrated into the following template:

The quality of the final report will be reviewed by the Steering Committee of the evaluation using this checklist:

6.2. End-of-Evaluation Questionnaire

An end-of-evaluation questionnaire will be given to the evaluator and must be completed by him/her, a member of the Steering Committee and the person in charge of the evaluation.

6.3. Evaluation dates and schedule

It is expected that the evaluation process will take around 2,5 months for the period from 10^th December 2020 to 1st of March 2021. A detailed action plan will be submitted as part of the inception report. The final report should be validated no later than the 21^st February 2021, and the consolidated report should be finalized by 1^st of March 2021.**

7. Means

7.1 Expertise sought from the consultant(s)

The evaluation expert (or team of experts) who will undertake this assignment should have the following skills, experience and knowledge:

• Academic background in Disability, Health, Rehabilitation, Social Sciences or developmental studies with a minimum of a Master’s Degree in the relevant field.

• Experience in project evaluation and related methodologies with FCDO (ex-DFID)-funded projects is a plus.

• Demonstrated Experience in conducting participatory (qualitative and quantitative) evaluation techniques.

• A wide experience in all aspects of project cycle management.

• Experience working with persons with disabilities in general and/or children with disabilities is an asset.

• Practical knowledge of rights-based approaches and Inclusion.

• Strong analytical and report writing skills.

Qualified persons with disabilities are encouraged to apply!

7.2 Budget allocated to the evaluation

In the case of external evaluation: Statement of the budgetary modalities that the candidate must detail in the offer: the cost per day; the breakdown of the time spent per evaluator and per stage of work; the ancillary costs (services and additional documents).

The overall cost of the intervention including transport costs (international and local), logistics costs, accommodation, and translation costs; with proposals for payment modalities.

Caution: please note that all payment is conditional on the validation of the deliverables and not on the sending of these deliverables. By validation, we mean validation of the quality and under no circumstances of the appreciation of the project evaluated.

7.3. Available resources made available to the evaluation team

Document and resource of information:

· Project proposal

· Monitoring Box, (PM Box) which include all project activities with the timeframe. In addition, human recourse involved

· Memorandums of understanding and letters of Agreements.

· Quarter Donor Reports

· HI Operational and monitoring tools, including; Post-intervention monitoring, Rehabilitation Technical Assessments, EDEI technical assessments for PHCs and EI units, Rehabilitation Management System Scoring tool (RMS), training reports, pictures, attendance sheets, accessibility reports, CBR supervision tools, Sustainable Analysis Process (SAP) report

· Systematic Assessment of Rehabilitation Situation (STAR) report

· Supported partners operational and monitoring tools

· Mid Term evaluation Report