Data Engineer

We are seeking a highly skilled Data Engineer with strong expertise in data reporting, ETL pipeline development, and workflow orchestration. The ideal candidate should have hands-on experience working with MySQL, MongoDB, Apache Airflow, and modern data engineering practices to build scalable and reliable data solutions.

Key Responsibilities

• Design, develop, and maintain ETL/ELT pipelines for data ingestion, transformation, and reporting.

• Build and optimise data pipelines to process structured and semi-structured data from multiple sources.

• Develop complex SQL queries, stored procedures, and reporting datasets using MySQL.

• Work extensively with MongoDB for data extraction, transformation, and reporting requirements.

• Design and manage Apache Airflow DAGs for scheduling, monitoring, and orchestrating data workflows.

• Ensure data quality, integrity, and consistency across various systems.

• Collaborate with business stakeholders to understand reporting requirements and deliver actionable insights.

• Monitor pipeline performance and troubleshoot production issues.

• Implement best practices around data governance, security, and documentation.

• Optimise database queries and data processing workflows for performance and scalability.

Requirements

Primary Skills

• Strong experience in MySQL database development and reporting.

• Good hands-on experience with MongoDB.

• Strong expertise in Apache Airflow for workflow orchestration and scheduling.

• Experience building ETL/ELT pipelines and data integration solutions.

• Advanced SQL query writing and performance optimisation.

• Experience working with large datasets and data warehousing concepts.

Technical Skills

• Python for data engineering and automation.

• Data modelling and schema design.

• REST API integration and data ingestion.

• Linux/Unix command line proficiency.

• Version control using Git.

Preferred Skills

• Experience with cloud platforms (AWS, Azure, or GCP).

• Knowledge of data warehouses such as Snowflake, Redshift, BigQuery, or Synapse.

• Experience with Docker and containerised deployments.

• Understanding of data lake architectures.

• Exposure to BI tools such as Power BI, Tableau, Metabase, or Superset.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or related field.

• 3–8 years of experience in Data Engineering or related roles.

• Strong analytical and problem-solving skills.

• Excellent communication and stakeholder management skills.

Nice to Have

• Experience working in a startup or fast-paced environment.

• Knowledge of real-time data processing frameworks.

• Experience with CI/CD pipelines and DevOps practices.

Success Criteria

• Ability to independently design and implement scalable data pipelines.

• Deliver reliable reporting solutions with high data accuracy.

• Ensure efficient workflow orchestration and monitoring using Apache Airflow.

  • Collaborate effectively with business and engineering teams to meet reporting and analytics needs.