Aptean - Lead Data Engineer - ETL/Data Warehousing

 Design and build robust, scalable data ingestion pipelines using Microsoft Fabric (Pipelines, Dataflows, Notebooks) to integrate data from Business Applications and external APIs.

- Perform deep source system analysis to define ingestion strategies that ensure data reliability, consistency, and observability, while applying metadata-driven design for automation.

- Develop and maintain Delta Tables using the medallion architecture (bronze/silver/gold) to systematically cleanse, enrich, and standardize data for downstream consumption.

- Implement comprehensive data quality checks (nulls, duplicates, schema drift, outliers, SCD types) and ensure data integrity across all transformation layers in the Lakehouse.

- Apply governance practices including schema versioning, data lineage tracking, role-based access control (RBAC), and audit trails to ensure compliance, traceability, and secure data access.

- Build semantic models and define business-aligned KPIs to support self-service analytics and dashboarding in Power BI and other BI platforms.

- Structure the gold layer and semantic model to support AI/ML use cases, ensuring datasets are enriched, contextualized, and optimized for AI agent consumption.

- Develop and maintain AI-ready run flows and access patterns to enable seamless integration between the Lakehouse and AI agents for tasks such as prediction, summarization, and decision automation.

- Implement DevOps best practices for pipeline versioning, testing, deployment, and monitoring; proactively detect and resolve data integration and processing issues.

- Collaborate cross-functionally with analysts, data scientists, and business users to ensure the data platform supports evolving needs for analytics, operational reporting, and AI innovation.