Senior Azure Data Engineer
persistent
Job Description
- Design, develop, deploy, and maintain enterprise-grade ETL/ELT pipelines using Azure Data Factory (ADF).
- Build scalable data integration solutions to ingest data from structured, semi-structured, cloud, and on-premises sources.
- Develop and manage ADF pipelines, datasets, linked services, triggers, integration runtimes, and parameterized workflows.
- Implement incremental loading, Change Data Capture (CDC), full-load processing, and data synchronization solutions.
- Design reusable, scalable, and maintainable data integration frameworks.
- Configure data orchestration, transformation workflows, dependency management, and error handling mechanisms.
- Monitor ADF workflows and troubleshoot pipeline failures, performance bottlenecks, and operational issues.
- Implement logging, auditing, monitoring, retry policies, and exception management frameworks.
- Design and develop database solutions using Azure SQL Database.
- Develop and optimize complex SQL queries, stored procedures, views, functions, and database objects.
- Perform database performance tuning, indexing optimization, execution plan analysis, and query optimization.
- Develop data validation, reconciliation, and quality control mechanisms for enterprise data platforms.
- Build and maintain Azure Functions for serverless integrations, event-driven processing, and automation use cases.
- Integrate Azure Functions with Azure Data Factory, Azure Storage, REST APIs, and other Azure services.
- Develop and consume REST APIs for system integration and enterprise data exchange.
- Support deployment automation, CI/CD pipelines, release management, and operational readiness initiatives.
- Participate in architecture reviews, technical governance, and cloud modernization programs.
- Mentor junior engineers and drive engineering best practices across delivery teams.