Azure Data Engineer

Key Responsibilities 

• Design, develop, and maintain scalable data pipelines for batch and real-time data processing using Azure (preferred) or AWS cloud platforms. 

• Build and optimize ETL processes to ensure high-quality, secure, and efficient data flow across systems. 

• Collaborate with cross-functional teams to translate business requirements into robust data models and solutions. 

• Implement data quality, data governance, and data security standards throughout the data lifecycle. 

• Develop and maintain documentation, including design and mapping documents. 

• Conduct code reviews, unit testing, and peer reviews to ensure code quality and compliance. 

• Troubleshoot, monitor and resolve data pipeline and infrastructure issues to minimize business impact. 

• Stay updated with new technologies and evaluate their organizational impact. 

Required Qualifications 

• 5–7 years of experience in data engineering, with a strong background in building and managing large-scale data pipelines. 

• Hands-on experience with Azure Data Services (Data Factory, Data Lake, Synapse, Databricks, etc.) is preferred. Experience with AWS data services is also acceptable. 

• Proficiency in SQL, Python, or Scala for data processing and transformation. 

• Experience with data warehousing (e.g., Snowflake, SQL Server, MongoDB) and real-time databases. 

• Strong understanding of data architecture, data ingestion, curation, and consumption patterns. 

• Familiarity with data quality management, metadata management, data lineage, and data security best practices. 

• Excellent communication skills and ability to work collaboratively with global teams.