Senior Data Engineer

1. Lakehouse‑Based Data Engineering (AWS & GCP)

• Design and implement Lakehouse architectures using object storage and open table formats (e.g., Apache Iceberg) to support ACID transactions, schema evolution, and time‑travel.
• Build and maintain batch and streaming data pipelines on:
• AWS (e.g., S3, Glue, Kinesis, Spark)
• GCP (e.g., GCS, Dataproc, Dataflow, Pub/Sub)
• Implement medallion architecture patterns (raw / bronze → silver → gold) consistently across clouds.

2. Data Product Development

• Develop curated, governed, and consumption‑ready data products aligned to business domains.
• Partner with Data Product Owners and stakeholders to translate requirements into robust technical implementations.
• Ensure data products are discoverable, reusable, and well‑documented, supporting analytics and downstream AI/ML use cases.

3. Data Modeling & Quality

• Design and maintain analytical data models optimized for BI, reporting, and advanced analytics.
• Implement data quality checks, validation rules, and monitoring within pipelines.
• Manage schema evolution and ensure adherence to enterprise data standards across AWS and GCP.

4. DataOps, CI/CD & Automation

• Apply DataOps and DevOps best practices by implementing CI/CD pipelines for data pipelines and data products.
• Enable automated testing, deployment, and rollback across cloud environments.
• Collaborate with platform and DevOps teams to improve observability, reliability, and operational maturity.

5. Performance, Reliability & Cost Optimization

• Optimize pipelines for performance, scalability, and cost efficiency on both AWS and GCP.
• Monitor and tune compute and storage usage in collaboration with platform and FinOps teams.
• Troubleshoot and resolve complex production issues across multi‑cloud environments.

6. Technical Leadership & Collaboration

• Act as a senior technical contributor and mentor for junior and mid‑level data engineers.
• Participate in design and code reviews to maintain high engineering standards.
• Contribute to reusable frameworks, templates, and best practices for data engineering.