Analysis and Report Writing Consultant for Final Evaluation

Request for Proposal

Analysis and Report Writing Consultant for Final Evaluation

Strengthening Health Outcomes of Women and Children (SHOW)

1.0 Plan International Canada Overview

Founded in 1937, Plan International (http://plancanada.ca/) is one of the world’s oldest and largest international development agencies, working in partnership with millions of people around the world to end global poverty. Not for profit, independent and inclusive of all faiths and cultures, Plan International strives for a just world that advances children’s rights and equality for girls.

Plan International Canada Inc. (Plan Canada) has been a member of the Plan International (Plan) global federation since 1968. The Federation has 22 National offices, 4 Regional offices and delivers programs in 50 countries worldwide.

Plan International Canada is a registered charity with offices in Toronto and Ottawa whose primary focus is to raise funds through Individual, Corporate and Institutional funding.

1.1 OUR VISION

Our vision is of a world in which all children realize their full potential in societies that respect people’s rights and dignity.

1.2 OUR MISSION

Plan International Canada (“Plan Canada”) aims to achieve lasting improvements in the quality of life of children, families and communities in developing countries. We do this through a process of collaboration that unites people across cultures, adding meaning and value to their lives.

2.0 Overview of Request for Proposal

The Consultant will be the technical lead in the completion of the Final Evaluation of the Strengthening Health Outcomes of Women and Children (SHOW) project, implemented in Bangladesh, Ghana, Haiti, Nigeria and Senegal. A project baseline survey was conducted in 2016, followed by a midterm study in 2018-2019. Nearing the end of the project, Plan Canada is now seeking qualified consultant to lead quantitative analysis, qualitative analysis, and report writing for evaluations in four countries: Ghana, Haiti, Nigeria and Senegal.

The Consultant will be responsible for overseeing the following tasks: data cleaning and (re)structuring, as necessary in consultant with local data collection firms; analysis of all quantitative and qualitative data, including the production of data summary and analysis tables and coding of transcripts; and production of a concise results reports. The main objective of the consultancy is to produce one global evaluation report, which will include four individual country annexes and summary tables as well as the drafting of four country specific PPT results presentations to be presented to each country office.

3.0 Project Background

The Strengthening Health Outcomes for Women and Children (SHOW) is a six-year multi-country, gender-transformative project, funded by Global Affairs Canada. The initial project, valued at CAD$65M, started on January 20^th, 2016 and ended June 30^th 2020; to respond to the COVID-19 pandemic in Bangladesh, Ghana, Nigeria and Senegal, the project was extended to December 31^st, 2021. This study focuses on the outcomes and achievements of the initial project only.

The SHOW project had the ultimate outcome of contributing to the reduction of maternal and child mortality among vulnerable women and children, including adolescent girls, in targeted regions of Bangladesh, Ghana, Nigeria, Senegal and Haiti. The project works towards this aspirational outcome using a three-pronged approach, through the following intermediate outcomes:

§ Improved utilization of essential health services by women of reproductive age (WRA), adolescent girls, newborns and children under 5 living in poverty, with high vulnerability

§ Improved delivery of quality essential health services to WRA, adolescent girls, newborns and children under 5 living in poverty, with high vulnerability

§ Increased dissemination and use of data by project, beneficiary communities, health committees, service providers planners and decision makers

4.0 Evaluation Objectives

The overall objective of the four-country Final Evaluation is to conduct a robust and objective study in order to assess the project’s performance vis-à-vis expected outcomes within the project Performance Measurement Framework, and vis-à-vis a select number of DAC Criteria for Evaluating Development Assistance. More specifically the objectives of the study are to:

• Assess the impact of the SHOW project’s achievements vis-à-vis the project’s overall objectives/ intended outcomes, referencing baseline and midterm results
• Assess the project’s performance vis-a-vis the DAC Criteria for Evaluating Development Assistance, specifically: effectiveness, impact and sustainability

• Review best practices in project implementation, and subsequently generate specific recommendations for each country and/or across the project to guide future MNCH/SRHR program management and design

• Explore unexpected outcomes of the project activities, or successes that stem from the project expanding beyond its original scope

Plan International staff in Program Units and Country Offices, Plan Canada, implementing and technical partners, and the donor will be the major users of the evaluation results. In addition, key stakeholders in project countries such as government line ministries / departments, partner NGOs, local authorities, and communities are interested parties of the findings.

5.0 Scope of the Final Evaluation

5.1 Study Scope

In order to achieve the evaluation purpose and objectives, the Final Evaluation includes primary and secondary data collection to assess progress against all indicators included in the SHOW Performance Measurement Framework (PMF), as well as to respond to the Evaluation Criteria listed in Section 5.2 below (Effectiveness, Impact and Sustainability). Quantitative and qualitative data collection is currently ongoing in Ghana, Haiti Nigeria and Senegal, being led by four local data collection firms with oversight from Plan International M&E staff.

The endline evaluation will build on the results of the baseline in 2016 and midterm study in 2019. Indicators for expected outcomes in the SHOW project, shared across all countries, include:

Intermediate outcome indicators

% of women aged 15-49 who received antenatal care (ANC) by a skilled health provider at least 4 times during pregnancy (disaggregated by age)

% of children aged 12-23 months vaccinated against measles (dis. by sex)

% of live births attended by skilled health personnel (dis. by age)

% of mothers, and percentage of babies, who received postnatal care within two days of childbirth (dis. by age)

Quality of (MNCH/SRH) data collected

Frequency of data and results used for local decision making

Immediate outcome indicators

% of WRA (dis. by age) and their male family members who know at least two danger signs and related strategies along the continuum of care

% of WRA (dis. by age) and their male family members who know at least two key gender equality messages related to MNCH/SRHR

% of CHC leadership positions held by women

Average level of support provided by male family members for the utilization of MNCH/SRHR services by female family members (dis. by sex and by age for women)

% of women who are members of organized community groups (dis.by age and type of group)

% of CHC members that are female

Average level of satisfaction of WRA (dis. by age) and their male partners with the quality and responsiveness of MNCH/SRH services

% of Health Providers trained who can identify the 7 key signal functions for BEmONC according to government guidelines (dis. by sex)

% of health facilities that provide gender responsive and adolescent friendly referral services

Average level of satisfaction of female MNCH/SRH users (dis. by age) and their male family members with the MNCH/SRH referral system

% of health facilities that provide gender responsive and adolescent friendly MNCH/SRH services

% of health facilities that utilize environmentally safe waste disposal methods

% of facilities where records are maintained regularly

% of project areas with household level monitoring data collection # of times government stakeholders, health providers, community health committee members (dis. by sex) receive data from the project

§ Average level of capacity of government stakeholders, health providers, and community health committee members (dis. by sex) on the data continuum

Questions exploring the following topics related to gender equality are also expected to be included in the Final Evaluation, with analysis by age and sex of respondents:

Marital status of respondents

Type of support provided by male partner to WRA (dis.by age) during, before and after childbirth

Male partners’ level of and type of support for exclusive breastfeeding

Distribution of decision-making between WRA (dis.by age) and their partners

Attitudes towards intimate partner violence (IPV) and gender-based violence (GBV)

Women’s skills/abilities and opportunities related to making community-level decisions

Male partners’ level of support towards their female partners’ (dis. by age) participation in community groups/committees, assumption of leadership roles in those fora, as well as in community level decision making

Level of decision-making/participation of women in committees, and barriers (including household level, community and committee level)

Distribution of productive and reproductive labour between WRA and their partners

Women’s and adolescent girls’ (relative to men and boys’) level of access to and control over resources and opportunities (income/employment; education; information (including health information); mobile phones, radios or other communication technology; health services; mobility and transport; etc.

Additionally, questions exploring the following topics on sexual and reproductive health, including family planning, are also expected to be included in the Final Evaluation, with analysis by age and sex of respondents as appropriate:

Knowledge of SRH, including menstrual health, STIs and HIV/AIDS

Knowledge of benefits of delaying and spacing pregnancies

Knowledge and awareness of modern family planning methods

Ever use of modern family planning methods

Current use of modern family planning methods

Access to family planning services, information and supplies

Finally, the Final Evaluation is expected to include bivariate and multivariate analysis of expected outcomes by poverty and vulnerability status of respondents, using the same indices applied through household surveys at baseline and midterm in the SHOW project:

The Progress out of Poverty Index (PPI) developed by the Grameen Foundation, available for each of the five SHOW implementing countries

A Vulnerability Index that categorizes respondents into three levels of vulnerability based on a number of socio-economic criteria

5.2 Evaluation Criteria

The following key questions will guide the Final Evaluation’s assessment of the project against the DAC Criteria for Evaluating Development Assistance:

§ Effectiveness: The extent to which the project attained its outcomes.

o To what extent were the outcomes achieved?

o How did the project contribute to the achievement of these outcomes?

o How have the project’s implementation strategies, tools, unique partnerships and innovations contributed to project results?

o What were the major factors influencing the achievement or non-achievement of the outcomes?

§ Impact: *The positive and negative changes produced by the project, directly or indirectly, intended or unintended.**

o What has happened as a result of the SHOW project – either as intended or unintended, positive or negative?

o According to beneficiaries, what difference has the project made in their lives?

§ Sustainability: The extent to which the benefits (outputs, outcomes) of the project are likely to continue after donor funding has been withdrawn.

o What is the likelihood of continuation and sustainability of project outcomes and benefits after completion of the project?

§ What commitments (financial, human resources, etc.) have been made by stakeholders to maintain or improve results?

§ How will improvements in stakeholder knowledge, attitudes, capacities, etc., if observed, contribute to maintaining results?

§ To what extent is support available from the external environment to maintain or improve results?

o What are the major external factors that may influence, positively or negatively, the sustainability of the project results?

o To what extent has the project’s design, implementation, stakeholder management, etc. contributed to the sustainability of project results?

An assessment responding to these key evaluation questions is expected for each of the following four (4) SHOW country projects: Ghana, Haiti, Nigeria and Senegal.

6.0 Evaluation Methodology

The Final Evaluation is a summative evaluation, employing a mixed methods approach.

6.1 Data Collection Methods

As one of the specific objectives of the evaluation is to assess the effectiveness of the project, the Consultant(s) will assess the achievement of outcomes by comparing results to project baseline (and midterm) studies. As in the baseline and midterm studies, the Final Evaluation includes the core components of:

§ A household survey, wherein primary respondents are WRA (15 – 49 years of age) with a live birth between January 2019 and March 2020. In households that meet this inclusion criteria, a questionnaire will also be administered to male partners / family members who were present during the most recent pregnancy.

§ A health facility assessment including both interviews with key informants (health facility staff) on facility staffing and capacity, service provision, and data management; and observations of facility infrastructure and supplies.

§ Key informant interviews (KIIs) and Focus Group Discussions (FGDs) with project beneficiaries and key project stakeholders. **

All data collection will be conducted by local consultants situated in each SHOW implementing country.

6.2 Sampling Strategy

The Final Evaluation will aim to reach a representative sample for all quantitative methods in SHOW intervention areas, to support pre-post analysis of all outcome indicators.

6.2.1. Household Survey

The household survey will be conducted using a random multi-stage cluster sampling method to ensure cost-effectiveness and efficient survey implementation, as was done at baseline and midterm. Questionnaires for all respondent groups have been consistently applied across baseline, midterm, and endline studies.

Sample size estimates for the household survey in Table 1 were determined using a power-based calculation, with an additional 10 percent sample of households has been included in the estimate to address non-respondents or incomplete questionnaires.

Table 1: Sample Size Estimations, Household Survey

Ghana 768 WRA 408 male partners 1,176

Haiti 500 WRA 401 male partners 901

Nigeria 800 WRA 400 male partners 1,200

Senegal 1,094 WRA 406 male partners 1,500

To distribute the sample of WRA by age group (15-19, 20-49), the following proportional allocation is applied to the WRA sample. No specific strategy is expected to be applied to ensure proportionate distribution of male and female index children.

Table 2: Population proportion of WRA, 15-19

Ghana 19%

Haiti 16%

Nigeria 22%

Senegal 11%

In order to maintain a level of effort appropriate for male partners as a secondary respondent at sampled households, we estimate a sample size equal to that collected at baseline and midterm (400) for interviews with male partners.

6.2.2. Health Facility Questionnaire

A random sample of health facilities is expected to be drawn from the same clusters where the household survey is administered, to ensure cost-effectiveness and efficient survey implementation. Sample size estimates for the health facility questionnaire were determined using a precision-based calculation.

Table 3: Sample Size Estimation, Health Facilities

Ghana 68

Haiti 11

Nigeria 155

Senegal 144

In some countries, local consultants will distribute the sample by facility type, as SHOW project activities (including health care provider training, equipment distribution, etc.) vary by facility type/level (i.e. community clinic, dispensary, health centre, hospital, etc.).

6.2.3. Qualitative Data Collection

Focus group discussions (FGD)s will be carried out with the following groups:

Table 4: Total number of Focus Group Discussions

Ghana 14

Haiti 10

Nigeria 12

Senegal 15

Key Informant Interviews (KII)s will be carried out with the following groups:

Table 5: Total Key Informant Interviews

Ghana 20

Haiti 11

Nigeria 16

Senegal 16

7.0 Purpose and Objectives of the Consultancy

The Consultant(s) will serve as lead analyst and author for the global evaluation report, including four country annexes and tables, as well as country specific results presentation. The Consultant(s) will be responsible for analysis of all data.

Specifically, the roles and responsibilities of the Consultant(s) are to:

§ Verify and validate cleaned data, in coordination with local data collection firms

§ Process and analyze all raw data;

§ Sharing of frequency and statistical tables for quantitative data

§ Sharing of coded qualitative data

§ Designing analysis framework for findings

§ Validate preliminary findings with project stakeholders through four country specific PPT results presentations to be shared with each of the four country offices (Senegal and Haiti presentation to be in French and presented to team in French, Nigeria and Ghana to be in English and presented to team in English);

§ Write draft and final evaluation global report, including annexes (Nigeria and Ghana in English, Senegal and Haiti in French) incorporating feedback from Plan International

For responsibilities listed above, the Consultant(s) will work in collaboration with Plan International Country Office staff, Plan Canada and the local data collection firms as appropriate; in particular, support from Country Office staff will include validating preliminary findings.

8.0 Deliverables and Timelines

8.1 Key Deliverables

The key deliverables expected from the Consultant(s) for this assignment are as follows:

§ Cleaned and finalized quantitative and qualitative data

§ Frequency and statistical tables for quantitative data

§ Coded qualitative data

§ Analysis of all quantitative and qualitative data in accordance with the evaluation objectives listed above, for four countries, including critical analysis of the data through statistical treatment and triangulation with other sources.

§ Four Country Specific Results Presentations and sharing of the findings in English and French

§ Draft and final Evaluation Report,[9]** including one Global Evaluation Report In English and Annexes and Tables for each of the four assigned SHOW implementing countries, in English and French

8.2 Timelines

The consultancy is expected to commence in April 2021, and final reports must be submitted to and approved by Plan Canada by September 1^st, 2021.

The Consultant(s) selected to complete this assignment will be responsible for the deliverables specified in the table below:

Table 6: SHOW Final Evaluation Consultancy – Deliverables and Timelines

Deliverable

Timeline

Contract Awarded April 2021

Orientation for Consultant(s) and communication and preparation on data table outputs with Country Study Leads

April 2021

Finalized data cleaning and verification May 2021

Frequency Tables and statistical tables June 2021

Samples of completed qualitative analysis (i.e. coded transcripts) July 2021

Sharing of analysis framework July 2021

Draft four Country Specific Results Presentations July 2021

Present data to four country offices July 2021

Draft Global Evaluation Report and four Annexes and table for each of the four assigned SHOW implementing countries August 2021

Final Global Evaluation Report, including four country specific Annexes and Tables, incorporating feedback from Plan International Canada and Country Offices September 1, 2021

9.0 Qualifications of the Consultant

Plan International is looking for an experienced Consultant(s) with experience conducting evaluations for MNCH/SRHR programs in developing country contexts. The Consultant(s) should have the following qualifications:

§ Master’s degree or higher in International Development, Public Health, Gender Studies and/or other Social Sciences, Statistics or related fields (CVs required);

§ Minimum 10 years experience in administering studies, producing quality evaluation reports, preferably for international non-profit organizations and/or multilateral agencies;

§ Extensive experience in assessing health outcomes in developing contexts

§ National and/or international experience assessing gender equality outcomes

§ Demonstrated experience in quantitative and qualitative analysis

§ Knowledge and experience with MNCH and SRHR issues (including adolescent health), policies and service systems, particularly in development country contexts

§ Familiarity with the social-cultural contexts of each of the four assigned SHOW project countries and any related cultural, political, or religious sensitivities relevant to the completion of this assignment is preferred

§ Ability to read, write and present findings in French

§ Excellent writing and communication skills in English (Sample of work and references required) is mandatory

11.0 Ethics Approval and Disclosure/Ownership of Information

All ownership and copyright for final data collected is held by Plan International. It is understood and agreed that the Consultant(s) shall, during and after the effective period of the contract, treat as confidential and not divulge, unless authorized in writing by Plan International, any information obtained in the course of the performance of the contract. Information will be made available for the consultants on a need‑to‑know basis.

12.0 Child Protection

Plan International is committed to actively safeguarding children from harm and ensuring children’s rights to protection are fully realized. Plan International takes seriously the commitment to promote child safe practices and protect children from harm, abuse, neglect and any form of exploitation as they come into contact with Plan International supported interventions. In addition, we will take positive action to prevent child abusers from becoming involved with Plan International in any way and take stringent measures against any Plan International Staff and/or Associate who abuses a child. Our decisions and actions in response to child protection concerns will be guided by the principle of ‘the best interests of the child’.

As such, the study must ensure confidentiality and anonymity of participants. The Consultant(s) are required to provide a statement within their proposal on how they will ensure ethics and child protection in the analysis process in line with Plan International’s Safeguarding Children and Young People Policy. This must also include consideration of any risks related to the study and how these will be mitigated, including but not limited to:

§ Safeguards to protect the confidentiality of those participating in the study

§ Data protection and secure maintenance procedures for personal information

13.0 Supervision and Management of the Assignment

A communications protocol will be provided to the Consultant(s) at the beginning of the assignment to ensure that all Plan International stakeholders can remain appropriately informed throughout the study.

The Consultant(s) will be directly accountable to the Plan International Canada Director of Monitoring, Evaluation and Research.

Annex 1: Terms and Conditions of the Request for Proposal

The submission of a response to this RFP constitutes that the Consultant(s) and authorized staff have read, understood and accepted the proceeding terms and conditions as well as all other provision of this RFP. The information contained in this RFP (or accumulated through other written or verbal communication) is confidential. It is for proposal purposes only and is not to be disclosed or used for any other purpose.

In consideration of Plan providing such information to the Consultant(s) and as a condition to the review of this RFP, the Consultant(s) agrees that they will:

• Use all information and material disclosed exclusively for the purpose of responding to the RFP and will not use such information or materials to obtain any other commercial, trading, financial or other advantage or for any other purpose.
• Maintain as confidential all information and materials relating to the RFP that they may acquire in any manner and make copies of such information only to the extent that the same is strictly required for the purpose.
• Not disclose whether directly or indirectly any information or materials relating to the RFP (or any part thereof) except to their own personnel and professional advisors and then only to the extent strictly required for the purpose and under conditions of confidentiality.
• Not make any announcement, press release or other public statement in connection with the RFP without the prior written consent of Plan.
• On request by Plan at any time, deliver to Plan as soon as practicable all information and materials relating to the RFP in your possession, custody or control which contain details of or incorporate the whole or any part of the RFP.
• Be prohibited from discussing the RFP with any other Consultant(s) without first obtaining the prior written consent of Plan.
• Hold in strict confidence any information received in response to this RFP. This information will not be disclosed to any party, other than within Plan and their engaged consultant, without the express written consent of the Consultant(s).
• Acknowledge that Plan undertakes not to publicise any information obtained during this RFP process, either generally or to any other Consultant(s)s involved in the RFP. Additionally, there will be no obligation on the part of Plan to share any of the results or conclusions of the RFP process with any Consultant(s).
• Acknowledge that Plan reserves the right to change any aspect of, or cease, the RFP and any subsequent negotiation process at any time.
• Notify Plan if during the course of its evaluation the Consultant(s) becomes aware of an error in this RFP or in the information or data provided. Plan reserves the right to share information on errors with other bidding Consultant(s)s.
• Acknowledge that all intellectual property rights in this RFP and all materials provided by Plan or its professional advisers and consultants in connection with this RFP are and shall remain the property of Plan and its professional advisers and consultants.
• Agree to comply with Plan’s child protection policies. This includes acknowledgement of reading Plan’s procedures. For any employee or contractor who will provide services at any of Plan’s locations, a criminal reference check and vulnerable sector screen will be conducted by Plan. Plan may at its own discretion deem any employee or contractor unsuitable for providing these services. The Consultant(s) will have an opportunity to replace any such employees and contractors with another that meets Plan’s requirements.
• To the extent that the Consultant(s) has access to the facilities or computer resources of Plan, the Consultant(s)’s employees or contractors agree to comply at all times with the applicable core values, rules and regulations regarding safety, security, use and conduct as detailed in Plan’s Code of Conduct.