RFP: End line Evaluation of the pilot Pneumococcal vaccines Advance Market Commitment (AMC) 339 views

Background

The Pilot AMC for Pneumococcal Conjugate Vaccine (PCV) Advance Market Commitments (AMC) for vaccines aim to encourage the development and production of affordable vaccines tailored to the needs of developing countries. Through a forward-looking binding contract from donors and international agencies guaranteeing a viable market for target vaccines, AMCs encourage vaccine makers to develop or build manufacturing capacity for urgently needed vaccines.

In June 2009, the Governments of Italy, the United Kingdom, Canada, the Russian Federation, Norway and the Bill and Melinda Gates Foundation, collectively pledged a total of US$ 1.5 billion to fund a pilot AMC against pneumococcal disease. The pilot AMC for pneumococcal vaccines completed its tenth year of implementation in 2018, having been launched in 2007 with an overall aim of reducing morbidity and mortality from pneumococcal disease by accelerating the development, availability, and uptake of pneumococcal conjugate vaccines (PCVs). Specifically, the objectives of this pilot were to: Accelerate the development of pneumococcal vaccines that meet developing country needs (e.g. serotype composition and vaccine presentation) as specified in the Target Product Profile (TPP).

Bring forward the availability of effective pneumococcal vaccines for developing countries by guaranteeing the initial purchase price for a specific quantity of the new vaccines that represents value for money and incentivizes manufacturers to invest in scaling-up production capacity to meet developing country vaccine demand.

Accelerate vaccine uptake by ensuring predictable vaccine pricing for countries and manufacturers, including binding commitments by participating companies to supply the vaccines at low, long-term, and sustainable prices after the AMC finances are depleted.

Pilot the effectiveness of the AMC mechanism as an incentive for needed vaccines and to learn lessons for possible future AMCs.

Following a Monitoring and Evaluability assessment study conducted in 2008, a monitoring and evaluation framework with four components was adopted:

A baseline study to determine the point of comparison for future M&E, including the development of counterfactuals. The baseline study was completed in 2010 and included

an assessment of potential counterfactuals, proposed indicators, and industry and country-level interviews to characterise the PCV market environment prior to the pilot AMC launch.

Annual monitoring of both the AMC and the complementary activities required to support the public health goal of the AMC. Annual Reports have been completed covering the period June 2009-December 2019 here with information on the status of implementation and selected process and outcome indicators. A process and design evaluation two years after the launch of the AMC to assess whether the AMC mechanism was working as expected and to obtain information on the AMC design issues. The process and design evaluation was completed in 2013 and assessed how this pilot AMC’s design and implementation features have enabled or constrained progress towards intended AMC-specific objectives, and identified future challenges here.

Outcome and impact evaluations every four years after the signing of the first AMC Supply Agreement to assess the extent to which the AMC has achieved its goal and objectives; The Gavi Secretariat commissioned the first Pneumococcal AMC pilot outcomes and impact evaluation in 2015.

Other evaluation activities relevant to the AMC include a 2011 analysis of lessons learned in relation to the AMC design and process that provides important background on design-decisions taken. An overall results framework outlining the theory of change and performance indicators for Gavi support for pneumococcal conjugate vaccine were also developed.

All AMC background documents and reports can be found on the Gavi website at: https://www.gavi.org/investing-gavi/innovative-financing/pneumococcal-amc

 

Objectives

This RFP is for the second and end line outcome and impact evaluation. The main objectives of this evaluation are:

To explore the extent to which the pilot AMC has achieved its overarching goal of reducing morbidity and mortality from pneumococcal disease through the mechanisms proposed on its theory of change, as well as its four specific objectives.

To explore the effectiveness and efficiency of the AMC design and implementation to the extent that these processes contribute to explaining the extent that outcomes have been achieved and document lessons learned to improve the design of potential future AMCs.

The results of this end line evaluation of the pilot AMC mechanism will be critical for both learning and accountability to all AMC stakeholders. The recommendations based on lessons learned will also inform potential AMCs or other relevant mechanisms.

Scope

The evaluation will be retrospective, covering the entire period of the pilot AMC mechanism implementation 2009-2020 and will build upon the M&E work that has already been done, including the outcome and impact study undertaken in 2015.

As proposed in the AMC M&E framework, the second outcomes and impact evaluation will focus on the achievement of AMC outcomes and will assess causal linkages between the AMC intervention and results achieved through comparisons with appropriate counterfactuals. The second outcomes and impact evaluation should be informed by, but is not limited to, the proposed counterfactuals in the baseline study. The evaluation should propose updates to these counterfactuals, or development of new counterfactuals as needed, with justification.

The evaluation should explore changes—positive and negative, intended and unintended—generated by the AMC. As such, it is expected that the evaluation review the overall status of key performance indicators related to the AMC outcomes and impact included in the Results Framework for Pneumococcal Conjugate Vaccine (Annex 4).

Using a theory-based approach, issues related to AMC design processes, design elements, and implementation should be explored as explanatory factors for observed and estimated changes in outcome and impact measures. As such, the evaluation should build upon rather than replicate work of the process and design evaluation completed in 2013.


EVALUATION QUESTIONS

Impact

  • To what extent has the AMC contributed to the reduction of morbidity and mortality from pneumococcal disease in Gavi eligible countries?
  • During implementation of the project and at the end of current or future agreements with manufacturers?
  • To what extent has the AMC contributed to equitable reduction of morbidity and mortality from pneumococcal disease in Gavi-eligible countries? (Please refer to the equity dimensions represented in the Gavi indicators frameworks for the 2016-2020 period) To what extent has the AMC contributed to indirect and broader effects (e.g. herd effects, equity, child mortality, broader social and economic effects)?
  • What are the identified mechanisms for the proposed impact? How reliable are estimations of impact in terms of other potential factors?

Outcome

To what extent can the causal path from the AMC Theory of Change be validated?

  • • What were the main contributors (and relative weight of different components) to the long-term outcome of the pilot AMC according to the available data?
  • • To what extent have internal and external factors (e.g. changes to Gavi policies, certain elements of AMC design, the role of the Bank) impacted the execution and results of the pilot AMC?

Lessons learnt

  • To what extent is the pilot AMC model still relevant given the changes in the market environment?
  • Whether and how changes in the pilot model would have contributed to more effective execution and better results?
  • What are the key lessons learned from this pilot that could be used to inform development of similar initiatives in the future?
  • What are the learnings at country level in terms of budgeting and financing?
  • Are there specific lessons that could inform the use of AMC-type mechanisms to meet critical goals for vaccine development, production and rollout related to vaccines against emerging and re-emerging pandemic and epidemic prone diseases like Ebola and Covid-19?
  • To what extent is the AMC pilot replicable? Based on the validation of the AMC ToC and causal pathways, are there aspects of the model that are seen to be most critical if replicating for other vaccines?
  • To what extent does the AMC pilot compare with other similar mechanisms and what could have been the implication if other mechanisms were tried instead of the AMC approach?

Bidders may propose additional evaluation questions to the above list of evaluation questions as part of their proposals, with justification.

METHODOLOGY

In order to respond to the above questions and provide a high-quality report, bidders are expected to employ a range of evaluation methods and to pursue innovation where possible. Firms bidding on the evaluation are strongly encouraged to propose innovative methodological approaches in response to the evaluation questions. They should also describe their strategy for safeguarding the quality and credibility of the evaluation, including their proposed verification strategy and methods for determining and estimating contribution with justification, and strengths and limitations of counterfactuals. As noted above, the evaluation design should demonstrate how it will build upon the previous studies.

Bidders should describe their approach and considerations to overcome the limitations highlighted in the first outcomes and impact evaluation: Challenges inherent in isolating the influence of the AMC from all the other concurrent factors

Limitations to understanding the true and precise mortality and morbidity impact of immunisation

The bidder should include the feasibility of/approach to undertaking a cost-benefit analysis of the AMC in their proposal. At a minimum, bidders should elaborate on the following methods and data collection and suggest additional methods as part of their proposal: Counterfactuals:

The incremental contribution of the AMC to any observed and estimated changes in outcome and impact measures will be assessed through counterfactual analysis. The bidder should consider a wide range of counterfactuals including but not limited to counterfactuals recommended in previous AMC evaluations. If counterfactuals defined in earlier evaluations are deemed no longer to be suitable, alternatives can be proposed with justification. Criteria for selection of counterfactuals should be stipulated with a description of strengths and limitations.

Impact:

  • Comprehensive assessment of existing empirical evidence and model-based estimates of disease burden and vaccination impact, including but not limited to: Empirical evidence of changes in pneumococcal disease burden and impact of PCV vaccination
  • Modelled vaccination impact estimates from 2009-2020.

Outcome:

  • Progress toward outcomes should be assessed using historical and projected vaccine supply and demand estimates and forecasts, including but not limited to: Vaccine supply and demand estimates from 2009 to-date
  • Projected vaccine supply forecasts through 2020
  • Projected Gavi demand and coverage forecasts through 2020
  • Actual vaccine uptake trend in AMC eligible countries
  • Competition and new manufacturers entering the market


Review of literature and available data:

This evaluation should capitalise ongoing and past findings. The evaluation should build on the findings and recommendations of all previous AMC assessments and all AMC annual reports available on the Gavi website. The role of the pilot AMC should be interpreted in the context of overall national government and donor support for pneumococcal conjugate vaccine in Gavi-eligible countries, as outlined in the Gavi pneumococcal vaccine Results Framework.

The evaluation should leverage other ongoing complementary work including; pneumococcal impact studies, Gavi transition assessments etc. Broader immunisation and health data should also be utilised. This includes, but should not be limited to: WHO-UNICEF and Institute of Health Metrics and Evaluation estimates of national immunization coverage;

Dates of introduction of new vaccines available at both WHO and Gavi


Interviews:

Relevant AMC stakeholders (including initial design team) members should be interviewed to ascertain their views and perception on whether the AMC has successfully achieved its objectives and contributed to positive outcomes and impact. Bidders should provide an indication of scope and scale of stakeholder interviews in the proposal.

Finally, to ensure credibility, the AMC Outcome Evaluation should be conducted in accordance with the following principles: 1) independence and impartiality; 2) involvement of stakeholders(including external participants (countries, manufacturers, CSOs, experts in different AMC fields like economic theory, innovative financing, manufacturing etc.

; 3) transparency; and, 4) reference to international norms and definitions such as the OECD Development Assistance Committee (DAC) principles.

Gavi is committed to learning and adapting based on findings and recommendations from evaluations. Bidders should build a learning approach into the proposal to ensure that opportunities for learning throughout the evaluation, as well as from the findings, lessons learned, and recommendations are maximised. As such, bidders are expected to explicitly describe their communication, learning and engagement modes to foster utility of evaluation results.

 

Confirmation of Intent/Confidentiality

Please transmit your intent to participate using and signing the document in Annex 1. Confirmations of intent should be submitted by email to the below mentioned contacts.

Acceptable means of transmission include computer file with digital signature.

For Contractual RFP & Contract Terms & Conditions, Proposal Format, etc.

  • Contact Person: Romain Nicolas, Procurement Manager
  • Phone: +41 22 909 71 68
  • Email: [email protected]

For Technical RFP Deliverable Specifications & Requirements

  • Contact Person: Esther Saville Head, Evaluations
  • Phone: +41 22 909 6796
  • Email: [email protected]
  • To help us track our procurement effort, please indicate in your email  where (ngotenders.net) you saw this tender/procurement notice. 


Deadline: October 12, 2020

Share this tender

Created in 2000, Gavi is an international organisation – a global Vaccine Alliance, bringing together public and private sectors with the shared goal of creating equal access to new and underused vaccines for children living in the world’s poorest countries.

Connect with us

Newsletter

Subscribe our newsletter for quick response

Contact Info