Lead Engineer 1

Responsibilities

• 5 to 10 Years of IT & Azure Data engineering technologies experience
• Prior experience in ETL, data pipelines, data flow techniques using Azure Data Services
• Working experience in Python, Py Spark, Azure Data Factory, Azure Data Lake Gen2, Databricks, Azure Synapse and file formats like JSON & Parquet.
• Experience in creating ADF Pipelines to source and process data sets.
• Experience in creating Databricks notebooks to cleanse, transform and enrich data sets.
• Development experience in orchestration of pipelines
• Good understanding about SQL, Databases, Datawarehouse systems preferably Teradata
• Experience in deployment and monitoring techniques.
• Working experience with Azure DevOps CI/CD pipelines to deploy Azure resources.
• Experience in handling operations/Integration with source repository
• Must have good knowledge on Datawarehouse concepts and Datawarehouse modelling.
• Working knowledge of SNOW including resolving incidents, handling Change requests /Service requests, reporting on metrics to provide insights.
• Collaborate with the project team to understand tasks to model tables using data warehouse best practices and develop data pipelines to ensure the efficient delivery of data.
• Strong expertise in performance tuning and optimization of data processing systems.
• Proficient in Azure Data Factory, Azure Databricks, Azure SQL Database, and other Azure data services.
• Develop and enforce best practices for data management, including data governance and security.
• Work closely with cross-functional teams to understand data requirements and deliver solutions that meet business needs.
• Proficient in implementing DataOps framework.

Qualifications

• Azure data factory
• Azure Databricks
• Azure Synapse
• PySpark/SQL
• ADLS
• Azure DevOps with CI/CD implementation.

Nice-to-Have Skill Sets:

• Business Intelligence tools (preferred—Power BI)
• DP-203 Certified.