Terms of Reference for Evaluation
“Emergency Food Assistance for Conflict-Affected Families in Hajjah, Hodeida, Taiz, Aden, and Lahj Governorates, Yemen”
August 17, 2021
Type of evaluation
Final evaluation
Type of contract
International
Contract type
Consultancy
Tentative start date
October 5, 2021
Duration of contract
26 working days
Project duration
36-39 monthss
Project location
Hajjah, Hodeida, Taiz, Aden, Lahj – Yemen
Thematic area(s)
Food Assistance, Nutrition
Program objective
Improve the immediate food security of the most vulnerable, conflict-affected families in Hajjah, Hodeida, Taiz, Aden, and Lahj Governorates in Yemen.
Evaluation objective
Evaluate the effectiveness and relevance of food and nutrition assistance, service delivery mechanisms, implementation processes, and entitlements in achieving project goals, purposes, and targets
Program Background
Yemen is the worst humanitarian crisis in the world, characterized by large-scale internal displacement, food insecurity, lack of access to health services and water, disruption of livelihoods, and a high prevalence of malnutrition. SC responds to food insecurity and nutrition needs through a BHA-funded project which seeks to increase household access to and consumption of diverse, nutritious food and improve nutrition among pregnant and lactating women and children under five through training and infant and young child feeding (IYCF) activities.
Scope and Purpose of Evaluation
The evaluation will assess and measure the extent to which the project was able to improve the food and nutrition security of beneficiaries; the effectiveness of project implementation (beneficiary selection, food assistance delivery mechanisms, and feedback and response mechanism [FRM]); the effectiveness of using community nutrition voluteers (CNVs) in community-level training and IYCF service provision; and capture lessons learned, best practices, and recommendations. Project performance will be measured through the outcome indicators: Prevalence of households with moderate or severe hunger (Household Hunger Scale); Percentage of households with ‘acceptable’ FCS; and c) Average rCSI score.
Key Questions
Effectiveness
- To what extent did the project enable beneficiaries to improve their food and nutrition security through project outcomes, output indicators defined within the project logframe?
- To what extend were three food delivery mechanisms (Cash-in-Hand, Commodity Food Vouchers [CFVs], and Value Food Vouchers [VFVs]) effective in households’ food security improvement? Which among the three was most effective?
- How effective was the use of CNVs in providing community-level nutrition training and IYCF services? What worked well and what did not?
- How effective was the FRM in providing accountability and voice to beneficiaries? What worked well and what did not?
Relevance
- To what extend did the beneficiary selection process (targeting and verification) reach the most vulnerable households?
The consultant is encouraged to reference the below suggested methodology table in their technical proposal but may propose changes where deemed appropriate.
Evaluation Question
Key Data to be Collected
Suggested Methods
Existing Data Soruces
To what extent did the project enable beneficiaries to improve their food and nutrition security as defined by project outcome and output indicators established within the project logframe?
All endline values for outcome indicators established in the project logframe
Beneficiary perceptions regarding effectiveness of interventions on their household and community
Household survey
Key informant interviews (KIIs) with project stakeholders (CDCs, local leader, CNVs, etc)
KIIs with project technical and operations staff – and other NGO actors where possible
Desk Review
KIIs with key project staff, including programming and operations
KIIs with project beneficiaries
Focus group discussions (FGDs) with project beneficiaries
Project baseline report and data
Project IPTT
Distribution reports and data
Project proposal and quarterly reports
Impact of the Suspension Report
Detailed Implementation Plan
Beneficiary Feedback Database
FRM sensitization materials
Selection criteria documentation, including registration and verification forms
To what extent was the project able to implement interventions in a timely manner? What factors limited the timeliness of implementation and did others experience the same delays? What was the effect of any delays and how did the project adapt?
Key project staff perspectives on factors impacting intervention delays, impact on programming, and adaptations made
Which among the three modalities / delivery mechanisms (Cash-in-Hand, CFVs and VFVs) was the most effective? What challenges and successes did each modality face during the project? How could future projects improve the effectiveness and efficiency of each modality?
All endline values for outcome indicators established in the project logframe
Beneficiary and community leader perceptions regarding the relevance and efficiency of the food delivery mechanism
Project staff experiences with the design, implementation and delivery of cash and voucher assistance
How effective was the use of CNVs in providingcommunity-level nutrition training and IYCF services? What worked well and what did not?
Beneficiary perceptions, including knowledge gains, referrals, and/or reported behavior change due to CNV training and/or interventions
CNV and lead project staff perceptions of the project’s effectiveness and relevance
How effective was the FRM in providing accountability and voice to beneficiaries? What worked well and what did not?
Beneficiary awareness of, utilization of, and access to project FRM
Project staff perceptions of challenges and successes during sensitization, utilization and close-out of beneficiary feedback
To what extent did the beneficiary selection process (targeting and verification) reach the most vulnerable households?
Beneficiary perceptions regarding awareness of, understanding of, and agreement with eligibility criteria, selection process, and results
Community leaders’ experiences through beneficiary selection process
Programming and technical staff perceptions and experiences regarding the strengths and challenges of beneficiary selection process, including gaps and recommendations for future projects
Household Survey Sampling
The household survey will utilize one stage simple random sampling stratified by Northern and Southern Yemen. The simple random sample will be use the project’s list of beneficiaries. Sample size calculation is based on estimated proportion of beneficiaries who had ‘acceptable’ FCS (score FCS>42) and proportion of beneficiaries who had little or no hunger (score HHS<2). The household survey will include a minimum of 446 households (223 each in the North and South).
Evaluation Timeline
The evaluation is planned for 26 days between October 5 2021 and late November to mid-December 2021. The number of days, methodology, and elements will be finalized after discussion with the selected consultant. The timeline and Level of Effort may change depending on visas, travel to and within Yemen, local authority approvals, COVID-19, and other logistical issues.
Evaluation Management
The consultant will produce an evaluation plan and inception report to outline the proposed methodology, including data collection methods, sampling considerations, timeline, logistics, etc. This will include the final report outline, which must align with the the donor recommended outline: https://usaidlearninglab.org/library/evaluation-report-template. Before any evaluation work is performed, SC must review and approve the draft plan, after which the consultant will design context- and project-specific data collection tools for KIIs, household interviews, and FGDs based on agreed evaluation questions. SC must also approve data collection tools before data collection begins. SC Yemen will provide the consultant with trained enumerators to support data collection.
Application process
The following qualifications and skills are expected of the consultant:
- Minimum education of master’s degree in Emergency response, International Development, Social Science, Economics, Evaluation, or relevant field from a recognized university.
- At least 7 years’ experience in evaluation of programs, including cash, food security, and/or nutrition.
- Thorough understanding of mixed-methods data collection, including design and collection of household surveys.
- Strong skills in data analysis and familiarity with at least one data analysis software (SPSS, STATA).
- Ability to create compelling data visualizations to convey key findings.
- Excellent analytical, research methods and communication skills.
- Fluency in English is required and Arabic preferre).
- Experience working in the middle east and/or Yemen is a plus.
Roles and responsibilities:
- Submit inception report and finalize based on SC feedback and approval.
- Obtain SC approval of evaluation methodology and data collection tools.
- Notify SC of delays during the final evaluation.
- Supervise and ensure proper data collection and entry per methodology and protocol.
- Clean and analyze qualitative and quantitative raw data.
- Debrief preliminary findings to SC .
- Submt draft and final evaluation report to SC.
Expected outputs
- Inception report (max. 20 pages) detailing proposed methodology.
- Soft copies of final quantitative and qualitative tools
- Completed copies of survey questionnaires, consent forms, and qualitative notes.
- Soft copies of raw and clean compiled data sets.
- Final Report (in English) following the outline described above.
- PowerPoint presentation of findings (max. 15 slides) and delivery of findings to key stakeholders.
Interested candidates should submit an expression of interest including:
Technical proposal: briefly and clearly describe your understanding of the task, technical aspect of the proposal, and proposed methodology (evaluation strategy, sampling design, data collection methods, data processing & analysis, data quality control measures, and timelines or operational plan).
Previous Experience: previous experience conducting final evaluations for BHA-funded projects (including FFP, OFDA, or other USAID-funded grant/project), working in Yemen (if any), and/or in non-permissive environments with a fragile security situation.
Detailed curriculum vitae (CV): detailed CV with contact details (including at least two traceable references). If the consultant is a firm, provide CVs of key staff that will work on the evaluation.
Signed financial proposal/budget: tasks should be broken down into modules: Consultancy fees, Questionnaire development, Data processing & analysis, Communication, Reporting, and Miscellaneous (stationeries, printing, etc.).
Example of Previous Work: SC strongly suggests that applicants submit one copy of an evaluation report completed by the consultant/firm. The copy may be a limited sample of the report to protect client confidentiality but should demonstrate past experience and ability.
Applications should be sent to
with the subject “Emergency Food Assistance for Conflict-Affected Families in Hajjah, Hodeida, Taiz, Aden, and Lahj Governorates, Yemen – FFP”.
Applicants should reference the full TOR found here when submitting their full application:
https://drive.google.com/file/d/1s7WorurdsF1SMY0KYtqmKD9lzJWcRKjv/view?usp=sharing
To help us track our recruitment effort, please indicate on the application form /in your cover letter where (ngotenders.net) you saw this job posting.