Data Engineer

Job Description

A Data Engineer works closely with the cross-functional product team and is responsible for enabling the team to ensure effective, fast, and secure data accessibility and usage..

The CHT uses a primary CouchDB database which replicates to a PostgreSQL data warehouse which stores the data in a JSON column and uses materialized views to allow easy access. As the CHT has grown in number and scale of projects this architecture is no longer able to scale. We need a Data Engineer to help us update and productise our replication, warehousing, and visualization architecture.

Ongoing work will include supporting the solution and empowering developers to build and reuse data visualisations.

Responsibilities

Data engineering (70%)

• Develop data solutions which enable visualization of health information

• Implement standardized schemas (like FHIR) to make integration more consistent

• Normalize data from schemaless NoSQL documents into a relational schema

• Develop tests and assertions to improve data quality

• Automate collection of anonymised impact data into a data warehouse

• Deliver scalable solutions to support large deployments

Data analysis (20%)

• Work with the Research team to assist in data analysis for scientific investigations

• Analyze datasets and communicate results within the Medic organization

Other team activities (10%)

• Work with the Senior Director of Technology and development team to guide platform development and technical strategy with regards to data

• Work with development team to improve overall data competency

Required Competencies (rank these in order of priority – these are your non-negotiables)

• Strong working knowledge of SQL, especially aggregate functions/features.

• Proficiency in PostgreSQL or similar relational database

• Can develop data management solutions

• Can implement data workflows to enable data standardization and normalization

• Can develop schemas and queries that scale

• Comfortable with using scripting languages for data manipulation

• Thrives working as part of a distributed team with a flexible schedule

• Enjoys working remotely with opportunities to travel to project sites (e.g. Nepal and Kenya) and work with a globally distributed team

• Wants to help build solutions that improve lives in a real and significant way

Nice to have:

• A Bachelor’s degree or equivalent

• Can visualise data to help administrators be more effective **