Senior Data Engineer — Enterprise Data Platform

• Design, implement, and maintain data pipelines and storage layers following a medallion architecture (landing → bronze → silver), operating as the enterprise tenant within our multi-tenant data platform and unified enterprise metastore
• Develop and operate pull-based data ingestion flows using Azure Data Factory and custom Python-based API connectors to land data from diverse enterprise source systems
• Build and maintain transformation pipelines with strong focus on data quality, lineage, and observability from landing zone through to consumption-ready layers
• Implement data security features including PII masking, tenant isolation, and retention policies in line with enterprise compliance requirements
• Collaborate with data product teams and enterprise stakeholders to drive self-service adoption and best practices
• Establish monitoring, alerting, and cost tracking integrations for pipeline operations
• Contribute to infrastructure as code and CI/CD practices for platform deployment and management

Skills & Requirements

What We're Looking For

• 5+ years of experience building scalable, maintainable, and self-service data solutions
• Strong background in batch and streaming pipeline design, orchestration, and schema management
• Hands-on experience with Databricks (Unity Catalog, Delta Lake), Apache Spark, and Python/SQL
• Experience with Azure cloud services, particularly Azure Data Factory, Azure Data Lake Storage (ADLS), and related data integration tooling
• Deep understanding of data quality validation and monitoring frameworks
• Familiarity with medallion architecture patterns and layered data processing
• Excellent collaboration skills across engineering, product, and enterprise stakeholder teams
• Proven ability to write clean, maintainable, and well-structured code
• Strong problem-solving mindset
  • Demonstrated experience with AI-assisted coding: spec-driven development, systematic validation of AI-generated output (functional and technical), and a controlled-adopter approach — AI amplifies productivity, the human owns every line that ships.