Coffeebeans.io - Lead Data Engineer

- Design, develop, and maintain efficient, reliable, and scalable data pipelines using modern data engineer tools and best practices.

- Build and optimise data flows between various internal and external data sources and destination.

- Implement and maintain data quality checks, monitoring, and alerting systems.

- Write efficient and maintainable code following best practices and coding standards.

- Participate in code reviews and contribute to technical documentation.

- Troubleshoot and resolve data pipeline issues and performance bottlenecks.

Required Technical Skills :

- Strong proficiency in SQL and data modelling.

- Hands-on experience with Python or Java programming.

- Proven experience with Apache Spark for large-scale data processing.

- Experience with Apache Airflow for workflow orchestration.

- Expertise in working with modern data platforms and warehouses, particularly Databricks and/or Snowflake (good to have Snowflake / databricks certification).

- Experience working with at least one major cloud provider (AWS, GCP, or Azure).

- Strong understanding of data engineering concepts, ETL processes, and data warehousing principles.