Data Engineer II
darwinbox
Job Description
We are seeking a Data Engineer II to join our Enterprise Data team. In this role, you will design and maintain data pipelines for structured and semi-structured data, apply transformation logic, and manage storage solutions like Azure Data Lake and Delta Lake. You will deliver analytics-ready datasets, ensure governance and security standards, optimize performance for cost and scalability, and collaborate with cross-functional teams.
In this role, you will build and maintain ingestion pipelines for both structured and semi‑structured data while applying transformation logic across the bronze, silver, and gold layers. You will manage data storage and optimize the use of Azure Data Lake and Delta Lake, ensuring the delivery of curated datasets for analytics and reporting. Additionally, you’ll implement governance and security standards using Unity Catalog, optimize Spark and Databricks workflows for performance and cost, and collaborate closely with data scientists and analysts to meet evolving business needs.
We are seeking a Data Engineer II to join our Enterprise Data team. In this role, you will design and maintain data pipelines for structured and semi-structured data, apply transformation logic, and manage storage solutions like Azure Data Lake and Delta Lake. You will deliver analytics-ready datasets, ensure governance and security standards, optimize performance for cost and scalability, and collaborate with cross-functional teams.
In this role, you will build and maintain ingestion pipelines for both structured and semi‑structured data while applying transformation logic across the bronze, silver, and gold layers. You will manage data storage and optimize the use of Azure Data Lake and Delta Lake, ensuring the delivery of curated datasets for analytics and reporting. Additionally, you’ll implement governance and security standards using Unity Catalog, optimize Spark and Databricks workflows for performance and cost, and collaborate closely with data scientists and analysts to meet evolving business needs.