OXFAM CONSULTANCY FOR PROACT-END OF PROJECT EVALUATION

Purpose of the evaluation:

The purpose of this evaluation is to assess project performance against the evaluation criteria: project design relevance, effectiveness, efficiency, impact, sustainability, M&E systems, other cross cutting issues, etc..

The End of Project (EoP) evaluation results will give the possibility to use the findings for:

o Assessing the contribution in terms of outcomes achieved by the project;

o Identify Learning on best practices and stories of change;

o Providing recommendations on focal areas such as sustainability of the outcomes and methodology for further replications amongst others;

o Accountability towards stakeholders at large (government, donor, farmer representative bodies, civil society and the private sector;

The users of the EoP evaluation are:

Project teams, partners and governments (state and national): they will use the results and recommendations derived from the EoP evaluation to improve the future programs for greater impact;

European Union: EU may use the results for accountability and other project design purposes.

The EoP has an additional purpose of drawing lessons and developing recommendations for Oxfam and the EU that may help for improving the selection, enhancing the design and implementation of similar future projects and activities in the country and on a global scale upon project completion. The EoP report should include examples of good practices for other projects in Northern Nigeria

The evaluation team should provide an analysis of the attainment of the main objective and specific objectives and project indicators set targets.. Through its assessments, the evaluation team should enable the Government, Oxfam and other stakeholders and donors to verify prospects for development impact and sustainability, providing an analysis of the attainment of SDG goals, project objectives, delivery and completion of project outputs/activities, and outcomes/impacts based on indicators.

3. Key questions to be answered by the evaluation

The key question of the terminal evaluation is whether the project has achieved its objective, i.e. to what extent has the project improve food/nutrition security and resilience of vulnerable households in disaster and conflicts affected communities in Adamawa and Kebbi States of northern Nigeria.

Based on the above-mentioned objectives, the specific key questions that the EoP evaluation will address should include but not limited to the following:

Project design

The evaluation will examine the extent to which the:

§ Project’s design is adequate to address the problems at hand;

§ A participatory project identification process was instrumental in selecting problem areas and national counterparts;

§ Project has a clear thematically focused development objective, the attainment of which can be determined by a set of verifiable indicators;

§ Project was formulated based on the logical framework (project results framework) approach;

§ Project was formulated with the participation of national counterpart and/or target beneficiaries;**

Relevance

1. To what extent did the project respond to the priorities and needs of the participants, key groups and Governments of Adamawa and Kebbi state and Federal Government of Nigeria?

2. How relevant was it to the target groups’ needs and priorities?

3. To what extent are the objectives of the project still valid? How can the project be replicated in other contexts?

**

Effectiveness (Attainment of project objectives and results)

1. . To what extent the objective of the project was achieved?

2. To what extent the expected accomplishments of the project were achieved? In particular, how did the project contribute to increasing the capacity of government to develop policies for enhancing the food security and resilience?

3. To what extent the planned activities contributed to achieving the objective and the expected accomplishments?

4. What were the challenges/obstacles to achieving the expected results?

5. What has prevented (if any) the achievement of the desired results?

Efficiency

1. Were the resources sufficient for achieving the results? Were the results commensurate with the resources?

2. Were the results achieved on time?

3. Were all activities organized efficiently and on time?

4. To what extent the resources were used economically? How could the use of resources have been improved?

5. Where there any alternatives to achieve the same results? If yes, which ones?

6. Was the project implemented in the most efficient way compared to alternatives? In particular, how do the costs and use of resources compare with other similar projects on food security and resilience in Nigeria?

Impact:

1. What are the positive and negative changes produced directly or indirectly, intended or unintended?

2. What has happened as a result of the program or project?

3. How many people have been impacted by the project efforts?

Sustainability

18.. Could the results be further sustained? In particular, to what extent will the benefits of the project continue after completion?

19.How is the stakeholders’ engagement likely to continue, be scaled up, replicated or institutionalized after funding ceases? In case, will resources and capacity be enough to ensure that institutions will take over and sustain the benefits?

1. What are the major factors that will influence the achievement or non-achievement of sustainability of the project?

What would be the specific roles of beneficiaries and Government in sustaining the project gains at scale.

The following four dimensions or aspects of risks to sustainability will be addressed:

a. Financial risk

b. Environmental risk

c. Institutional and governance risk

d. Socio-political risk

Cross-cutting issues (poverty, gender, environment)

1. Was the conception of the project differentiated by target groups and was a poverty analysis available

2. Did the project achieve poverty-reducing effects?

3. Did the project improve the living conditions of the target group (access to education, health, nutrition, water, environment)?

4. Do women and men equally benefit from the intervention?

5. What are the intended and unintended effects for men and women?

6. What actual or expected positive and negative effects are foreseeable on the environment?

Monitoring and evaluation system

MEAL Plan design. Did the project have a MEAL plan to monitor results and track progress towards achieving project objectives? The Evaluation will assess whether the project met the minimum requirements for the application of the Project MEAL plan (see Annex 3).

MEAL plan implementation. The evaluation should verify that an M&E system was in place and facilitated timely tracking of progress toward project objectives by collecting information on chosen indicators continually throughout the project implementation period; annual project reports were complete and accurate, with well-justified ratings; the information provided by the M&E system was used during the project to improve performance and to adapt to changing needs; and the project had an M&E system in place with proper training for parties responsible for M&E activities to ensure that data will continue to be collected and used after project closure. Were monitoring and self-evaluation carried out effectively, based on indicators for outputs, outcomes and impacts? Are there any annual work plans? Was any steering or advisory mechanism put in place? Did reporting and performance reviews take place regularly?

Budgeting and Funding for MEAL activities. In addition to incorporating information on funding for M&E while assessing M&E design, the evaluators will determine whether M&E was sufficiently budgeted for at the project planning stage and whether M&E was adequately funded and in a timely manner during implementation.

4. Scope and Evaluation approach and methodology

The end of project evaluation (EoP) will cover the whole duration of the project from its starting date

in April 2016 to the estimated completion date in April 2021. It will assess project performance against the evaluation criteria: relevance, effectiveness, efficiency, cross-cutting, coherence, sustainability and impact, etc.**

The Endline evaluation will be conducted in accordance with the OECD Evaluation Policy and Oxfam’s evaluation guideline

It will be carried out as an independent in-depth evaluation using a participatory approach whereby all key parties associated with the project are kept informed and regularly consulted throughout the evaluation. The evaluation team leader will liaise with the Oxfam on the conduct of the evaluation and methodological issues.

The evaluation team will be required to use different methods to ensure that data gathering and analysis deliver evidence-based qualitative and quantitative information, based on diverse sources: desk studies and literature review, statistical analysis, individual interviews, focus group meetings, surveys and direct observation. This approach will not only enable the evaluation to assess causality through quantitative means but also to provide reasons for why certain results were achieved or not and to triangulate information for higher reliability of findings. The concrete mixed methodological approach will be described in the inception report.

The evaluation team will develop interview guidelines. Field interviews can take place in the form of survey, key informant interview, focus-group discussions or one-to-one consultations, etc. The methodology will be based on the following:

1. A desk review of project documents including, but not limited to:

(a) The original project document, monitoring reports (such as progress and financial reports to EU and annual Project Implementation Review (PIR) reports), MEAL Tracking Tool, output reports (case studies, annual plans, etc.

(b) Other project-related material produced by the project.

1. Since the project document contains a project results framework, the evaluation team will assess performance against this framework. The validity of the theory of change will be re-examined through specific questions in the interviews and, possibly, through a survey of the various stakeholders such as government officials, and to the representatives from the private sector that the project have partnered with and have been assisted by the project.

2. Counter-factual information: Baseline, midline and background information for the benchmarks exist for this project.

3. Interviews with project management and technical support including staff and management at OXFAM in the country office and – if necessary – staff associated with the project’s financial administration and procurement.

4. Interviews with project partners including Government counterparts from the Adamawa and Kebbi states.

5. On-site observation of results achieved in demonstration projects, including interviews with project participants.

6. The inception report will provide details on the methodology to be used by the evaluation team and include an evaluation matrix.

The consultant will in consultation with Oxfam team determine the appropriate sample sizes considering the geographical areas, targeted groups, and the homogeneity of the target population.

5. Evaluation team composition

The evaluation team will be composed of a national evaluation consulting firm with experience working in Northern Nigeria. The evaluation team should be able to provide information relevant for follow-up studies, including evaluation verification on request to the European Union (EU) up to two years after completion of the evaluation. Members of the evaluation team must not have been directly involved in the design and/or implementation of the programme/projects. The PROACT Program Manager, Resilience and Private Sector Engagement and the PQMEAL Manager will support the evaluation team. The Head of Program and Country Director will be briefed on the evaluation and equally provide support to its conduct.

6.Time schedule and deliverables

The evaluation is scheduled to take place in the period from February to March 2021. The key deliverables include:

i. Inception report

ii. Inception meeting

iii. Data collection tools

iv. Progress report preliminary findings

v. Power point presentation of the main findings

vi. Presentation of findings at a validation meeting

vii. Final Report (maximum 40 pages, excluding executive summary table of contents and annexes (The format for the report will be shared with the consultant)

7. Qualification and skills of Evaluation firm

The required skills and competences for the evaluator are:

1. Essential

Ø At least first degree in any statistics, international development and any social sciences

Ø At least five (5) years’ proven experience in Evaluating developmental projects for International development agencies and/or INGO .

Ø A proven record in delivering effective and professional evaluation results in developmental projects;

Ø Experience assessing the impact and outcomes on gender equality in evaluations;

Ø Excellent communication, analysis, writing and report presentation skills in English;

Ø Capacity to carry out field research in Northern Nigeria.

2. Desirable

Ø Knowledge and awareness of food security and resilience and the experience in the Nigerian context;

Ø Knowledge of Oxfam and its work.

Ø Ability to conduct interviews and discussions in English and local language.

Ø Ability to analyze and synthesize data from different sources relating to the scope of the evaluation.

Ø Good eye for detail, adherence to logic and capacity for inductive reasoning.

Ø Strong presentation, facilitating, communication and team working skills.

Ø Good interpersonal skills, including the ability to conduct discussions with a diversity of people ranging from senior management to project participants.

Ø Willingness and ability to travel to the different projects sites in Adamawa and Kebbi states.

8. Evaluation and management responsibilities

8.1 Evaluator’s (Consultant’s) roles and responsibilities

The firm/Consultants will facilitate and an inception meeting in collaboration with Oxfam team and obtained entry point data/information and agree thereof the work plan and itinerary for the assignment.

In addition, the followings shall be carried out. **

i. Review all relevant documents for evaluation study;

ii. Develop evaluation study design which includes survey methodology, review and refine the data collection tools (household questionnaire, focus group discussion guides, interview protocol, data entry templates, etc.) , as appropriate, including a field manual for training, in Consultation with the Oxfam/DEC/CRUDAN technical team, integrate all Oxfam/DEC/CRUDAN feedback on the study methodology and tools;

iii. Coding of quantitative tools and training of the field team in the use and data collection procedures.

iv. Develop the field work schedule in consultation with the PROACT/PQ teams

v. Conduct training for the data collectors during field visits phase, finalize the Endline study schedule Supervise the data collection process, give advice and ensure the quality of the data

vi. Flight expenses for the Evaluator/consultant for his/her team to and from Adamawa and Kebbi States.

vii. Vehicle hire to support the evaluation study process for the consultant’s team

viii. Food and accommodation for the consultant and his/her team in Adamawa and Kebbi

ix. Recruitment and payment of the enumerators and translation of questionnaires, where necessary

x. Conduct all interviews

xi. Data analysis and report writing, draft the first reports and include Oxfam feedback and finalize the report for submission

xii. Hand over all the data collected to Project team

xiii. Update Program Quality and MEAL Manager and Program Manager, Resilience and Private Sector Engagement on the progress and any challenges from the field.

xiv. Where necessary, confirm and obtain relevant information from key backstopping personnel from Implementing Partners OXFAM in Abuja.

The application from consulting firms shall contain the following sections:

o Evaluator’s approach on conducting the EoP evaluation – Methodology (providing detailed information on suggested tools and analysis methods, etc);

o Research implementation plan (questionnaire piloting, implementation, data quality assurance, processing, analysis);

o Detailed work plan;

o Detailed financial proposal;

o Information about the individual consultant/consulting firm and proposed staff (company portfolio in case of consulting firm, containing information about similar projects, staff resumes; in case of individuals resumes and information about similar projects);

o Examples of previous similar work/report.

The application should also include the following documents on the firm/organization applying (in case of companies/organizations):

Ø Registration document of the legal entity;

Ø TIN

OXFAM only makes payment upon completion of tasks)

The bids will be evaluated using the following criteria:

a) Clearly methodological proposal;

b) Adequacy of the methodology to key evaluation questions and criteria defined in the ToR

c) Viability of the methodology in relation to the context

d) Well-structured work plan with phases and deadlines for the results, detailed timeline

e) Viability of the methodology in relation to the time available

f) Incorporation of a gender perspective in the proposal

g) Methodological knowledge and M&E experience

h) Knowledge and experience of developmental and humanitarian program evaluations

i) Experience of working in Nigeria

j) Adaptability of the proposal in relation to proposed data collection and analysis

k) Adaptability of proposal in relation to available funds

Application Submission: Applications MUST be submitted by e-mail to [email protected] in one PDF file using Combinepdf.org or any other packages. The deadline for submission of applications is 10 days from the date of advertisement.

Please note that OXFAM will deduct and remit to the Federal Inland Revenue (FIR) a % withholding Tax for freelancer or 10% for firms from the total consultancy fees in accordance with the Nigerian Tax Law.

Equal opportunity:

“OXFAM is an equal -opportunity- employer and do not discriminate on the basis of race, colour, religion, etc. Qualified women are strongly encouraged to apply”.

9. Project evaluation parameters

The evaluation team will rate the projects. The ratings for the parameters described in the evaluation will be presented in the form of a table with each of the categories rated separately and with brief justifications for the rating based on the findings of the main analysis. An overall rating for the project should also be given. The rating system to be applied is specified in annexes 1 and 2.