Lead Data Engineer
drmartens
Job Description
- Develop and maintain scalable data pipelines to support operational and analytical needs.
- Collaborate with data scientists, analysts, and business teams to understand data requirements and deliver solutions that align with organizational goals.
- Optimize and maintain cloud-based data infrastructure (e.g., Snowflake, Azure) for performance and cost-efficiency.
- Ensure the integrity, reliability, and security of data through robust testing and validation practices.
- Support the implementation of data governance practices, working closely with the Data QA Specialist and Data Governance Lead.
- Monitor and troubleshoot data pipeline performance issues, proactively resolving bottlenecks and incidents.
- Contribute to the design and implementation of data models and schemas that meet business requirements.
- Stay updated on emerging technologies and best practices, recommending improvements to existing processes and tools.
THE STUFF THAT SETS YOU APART
Must-Have Experience:
- 6+ years of experience in data engineering or a similar role.
- Hands-on experience building and maintaining data pipelines, ETL processes, and integrations.
- Proficiency in programming languages commonly used in data engineering (e.g., Python, SQL, Scala).
- Experience with cloud data platforms such as Snowflake or Azure.
- Solid understanding of data modelling principles and database management systems.
Technical Skills:
- Knowledge of big data frameworks and processing tools (e.g., Snowflake, dbt, Apache Spark, Hadoop).
- Familiarity with DevOps practices, including CI/CD pipelines and version control systems (e.g., Git).
- Understanding of data governance principles, quality frameworks, and security best practices.
- Experience with containerization and orchestration tools (e.g., Docker, Kubernetes) is a plus.