Senior /Staff Data & Cloud Platform Engineer
micron
Job Description
-
Data Pipeline Engineering: Build scalable ingestion pipelines for structured, semi-structured, and engineering data sources.
-
Workflow Orchestration: Develop reliable batch, scheduled, and event-driven workflows using Apache Airflow and cloud-native services.
-
ETL / ELT Frameworks: Implement reusable pipeline patterns, transformations, incremental ingestion, and data quality checks.
-
Schema Management: Implement schema registry, schema versioning, schema evolution, and compatibility controls.
-
Delta Detection: Build change detection and incremental refresh mechanisms for efficient large-scale data synchronization.
-
Entity Extraction: Extract and normalize key identifiers such as Lot, Die, Wafer, Test, Flow, Product, and Step.
-
Cloud Landing Zones: Set up secure AWS, GCP, and on-prem landing targets for validation and analytics pipelines.
-
Infrastructure Automation: Provision infrastructure using Terraform, CI/CD, and Infrastructure-as-Code practices.
-
Platform Operations: Support containerized workloads using Kubernetes, Docker, monitoring, logging, and operational controls.
-
Security & Cost Optimization: Implement IAM, RBAC, hybrid connectivity, access controls, and cost optimization for TB-scale data.
-
Cross-Functional Delivery: Partner with IT, TPG AI, SMAI, Data Science, Product Engineering, and platform teams.