International Consultant for Project Final Evaluation

Timor-Leste is half of a tropical island on the Australian continental shelf characterized by a rugged mountain chain running east-west that divides the country into a generally warmer north coast and a milder south coast. The capital city, Dili, is in the north.

Oé-Cusse is a district of East Timor. It is a coastal exclave in the western part of the island of Timor, separated from the rest of East Timor by West Timor, which is part of the province of Nusa Tenggara Timur, Indonesia, and which surrounds Oé-Cusse in all directions except the north, where it borders the Savu Sea. The capital of the district is Pante Macassar, also called Oé-Cusse Town.

Oé-Cusse has unique development challenges as it houses the nation’s youngest population and has the nation’s lowest fundamental indicators in terms of human development. In 2014 the Timor-Leste government established the Special Zones of Social Market Economy (ZEESM TL) and the Special Administrative Region of Oé-Cusse Ambeno (RAEOA) as a regional authority to administer Oé-Cusse. RAEOA-ZEESM TL’s people-centered approach involves simultaneous investment in the education, health, and economic sectors.

Across Timor-Leste, young people make up 63.2 percent of the total unemployment rate, and the youth literacy rate is 84.4 percent. Within Oé-Cusse, Passabe records the lowest literacy rate of 50.3 percent. The population of Oé-Cusse is younger than the rest of the country, with 42% of its population under 15 years (General Directorate of Statistics Census, 2010-2015).

In Oé-Cusse, 19,942 students are enrolled in primary schools, 1672 in secondary, 340 in technical, and 59 in University. There are currently 607 primary school teachers across 74 primary schools, 75 secondary school teachers across 10 high schools, 18 technical schools’ teachers at the existing vocational training institute, in addition to a branch of a private university (DIT) established as a satellite campus of DIT located in Dili.

The status of education in Oé-Cusse is affected by a lack of skills education in high schools, insufficient educational materials in the national language (Tetum), and access to ICT learning and equipment. These challenges pose a significant barrier to achieving the region’s development goals.

About the Project

The Project “Leveraging ICT to Improve Education and Skills in Timor-Leste” was launched in 2019 in partnership with the government’s Region Authority of Oé-Cusse Ambeno (RAEOA), funded by the UN-India Development Partnership fund. The total project budget is $ 959,129.00 (UN-India’s contribution is $ 881,193.20 and the government contribution is $77,935.80).

Since 2019, the project has achieved several results to improve primary and secondary education by providing children and young people in Oé-Cusse with information and communications (ICT) skills and knowledge to prepare them better to address the challenges and opportunities in the local economy.

The project’s first objective is to “improve education quality in Oé-Cusse Region by promoting computer skills to 1007 children at primary school and 4370 young students attending secondary school through the implementation of laboratories for IT innovation and experimentation”. To achieve this, the project has installed 10 laboratories: 4 in-house computer labs; 4 robotic labs, and 2 mobile computer labs. Additionally, the project has developed curriculum and learning materials in Tetum, Portuguese and English. The project actively develops students’ and teachers’ capacities to interact with and use ICT technology by providing access to ICT infrastructure and facilitating educational lessons. Students can apply the knowledge learned through the training in their everyday life to support their schooling and identify potential academic or occupational pathways.

The project’s second objective is to “enhance child and young people’s knowledge on financial education and environmental awareness through the development and implementation of two computer-based educative interactive games in Tetum and Portuguese language.” The games will be installed on all procured computers, educate students on environmental problems and solutions, and introduce students to the skills and knowledge they need to make informed financial decisions, promoting entrepreneurship early on. Playing the games will directly impact learning outcomes and prepare students to address the local economy’s challenges and opportunities. 

A governance mechanism (project board) has been established to provide overall guidance and oversight to implement the project. The project has been directly implemented by UNDP with extensive involvement and collaboration with the Secretary of Education at RAEOA ZEESM TL and UNICEF Timor-Leste. Coordination with the National Ministry of Education has been promoted through the support of UNICEF. (Details are in the project documents)

UNDP Timor-Leste is looking for an International Consultant to perform the Final Evaluation of the Leveraging ICT to Improve Education and Skills in Timor-Leste Project.

The evaluation’s overall purpose is to assess the processes and achievements made to draw lessons from the project implementation. More specifically, the evaluation is intended to be forward-looking, which will capture effectively lessons learned and provide information on the nature, extent, and, where possible, the effect of the project to the ICT Education in Oé-Cusse as well as the ICT education policies of the RAEOA’s Government, and Ministry of Education, Youth and Sports in Timor-Leste. The emphasis on capturing learning will be critical to understanding what has and what has not worked as a guide for future planning.

The specific objectives of this final evaluation are:

• To review progress towards the project’s objectives and outcomes,
• To assess the likelihood of the project delivering its intended outputs,
• To identify strengths and weaknesses in design and implementation,
• To identify risks and countermeasures,
• To identify challenges, lessons learned, and good practices.

The consultant will report to the CTA and Programme Manager of the Special Administrative Region Authority of Oé-Cusse Ambeno (RAEOA – ZEESM – TL) Project. The scope of work for consultancy will include, but may not be limited to:

• Review all relevant sources of information, such as the project document, logical framework, monthly reports, annual progress reports, files, the national curriculum and legal documents, any other materials that the evaluator considers useful for this evidence-based assessment, including the Regional Strategic Plan of Oé-Cusse and Project work plan for 2019, 2020 and 2021.
• Develop an evaluation matrix with evaluation criteria, the related evaluation questions (and, where needed, sub-questions), the data sources required to answer the questions, the data collection, and data analysis methods.
• Familiarize themselves with current standings of local development in O Oé-Cusse within the National Development Strategy.
• Frame the evaluation effort using the five criteria of relevance, effectiveness, efficiency, impact, and sustainability. The evaluation must provide evidence-based information that is credible, reliable, and useful. The evaluator is expected to follow a participatory and consultative approach ensuring close engagement with the government, the local organizations, the ICT project team, UNDP senior management team, final beneficiaries, and other key stakeholders.
• Prepare a draft evaluation report providing descriptive overviews, laying out the facts and lessons learned, providing conclusions and recommendations, and finalizing the report upon the feedback from the UNDP, project team, and government counterparts.

The incumbent will be tasked to conduct the evaluation as per Monitoring, Reporting, and Evaluation Systems of the UNDP M&E tools, focusing on five possible areas of evaluation (evaluation criteria): relevance, effectiveness, efficiency, impact, and sustainability. Specific areas of evaluation and evaluation questions are to be developed during the inception phase.

• Relevance: compliance with the goals and objectives of the project, including contribution to the Regional Strategic Plan of Oé-Cusse 2019-2023 and ICT curricula reform with the public needs and Government priorities
• Effectiveness: compliance with the results achieved due to the project implementation with the planned work plans for 2019-2021 and direct and indirect beneficiaries’ needs.
• Efficiency: the achievement of results by as less spending as possible (the ratio of outcomes and the resources needed for their achievement should be defined)
• Impact: unforeseen and projected influence, including COVID19.
• Sustainability: long-term influence and impact of the strategy and capacity development of local governmental partners.

Essential:

• Master’s Degree in Information and Communication Technology, Computer Science, social science, Education, Monitoring and Evaluation or related fields (compulsory)
• At least 10 years of experience working on education, education reform, pedagogy, knowledge management, communications in developing countries.
• Previous work experience evaluating projects related to Education, including ICT in education,
• Experience in South/South-East Asia and/or Timor-Leste
• Strong interpersonal, oral, and written communication skills in English is imperative;
• Ability to work effectively with Microsoft Office and proven reporting experience.

Desirable:

• Working knowledge of Tetum and/or Portuguese is an advantage.

Interested applicants are mandatory to apply online through https://jobs.undp.org/cj_view_job.cfm?job_id=97295. Supporting documents/filled templates to be submitted along with the applications are following, and those templates can be download from https://procurement-notices.undp.org/view_notice.cfm?notice_id=75865.

• UN P11 Form
• Individual Contract Offerors Letter to UNDP Confirming Interest and Availability
• Individual Contract Reference Check (at least from two referees)
• CV

Any request for clarification must be sent in writing to [email protected].