DevOps Engineer II
darwinbox
Job Description
- Design, build, and maintain public cloud infrastructure services including compute, networking, storage, and security patterns used by data platforms and data product teams.
- Develop and maintain Infrastructure‑as‑Code (IaC) using Azure DevOps, Terraform, as well as Python, and ARM or equivalent tooling utilized in IaC code.
- Build and maintain automation frameworks and self‑service catalogs enabling application and data engineering teams to provision cloud resources independently.
- Ensure infrastructure provisioning adheres to IT general controls, security requirements, and governance‑as‑code standards.
- Identify emerging cloud services and evaluate their fit for resiliency, cost optimization, and performance needs.
- Design, implement, and optimize automated CI/CD/CT pipelines for Azure‑based data engineering workloads.
- Integrate data processing workloads into standardized pipelines supporting data ingestion, transformation, quality checks, deployment, and operational readiness testing.
- Automate monitoring, logging, alerting, and observability across the data platform and its pipelines.
- Collaborate with data engineering teams to improve pipeline reliability, reduce deployment friction, and embed DevOps best practices.
- Own maintenance and upgrades of DevOps tooling, including Azure DevOps, orchestration frameworks, and testing platforms.
- Provide hands‑on technical coaching across teams to accelerate adoption of DevOps, cloud engineering, and automation best practices.
- Serve as a subject-matter expert on Azure services relevant to data engineering: Databricks, Azure Data Lake Storage, Synapse, Confluent Cloud, Fivetran, SQL Database, and others.
- Partner with Site Reliability Engineers (SRE’s) to develop reusable patterns, templates, and standards for CI/CD/CT observability pipelines.
- Produce clear documentation, training materials, and operational runbooks.
- Act as a testing evangelist—supporting TDD/ATDD/BDD, automated unit/integration testing, and deployment readiness validation.