Lead I - Data Engineering

Primary (Must-Have) Skills

• Azure Databricks
• PySpark
• Python
• Microsoft Fabric
• Azure Data Factory (ADF)
• Azure Synapse Analytics
• Informatica
• ETL/Data Engineering
• SQL
• Data Quality Frameworks
• DataOps
• CI/CD Pipeline implementation
• Solution Design & Architecture
• Strong communication and stakeholder management skills

Secondary Skills

• Apache Spark
• Azure Cloud Services
• Data Modeling
• Performance Optimization
• Azure DevOps
• Git
• Agile/Scrum methodology
• JavaScript (basic knowledge is acceptable)

Roles & Responsibilities

• Design, develop, and optimize scalable data pipelines using Azure Databricks, PySpark, and Python.
• Build and maintain ETL/ELT workflows using Azure Data Factory, Synapse, Informatica, and Microsoft Fabric.
• Develop robust data engineering solutions to support business intelligence and analytics initiatives.
• Ensure data quality, integrity, governance, and monitoring across data platforms.
• Implement DataOps best practices and automate deployments using CI/CD pipelines.
• Translate functional and business requirements into technical solutions and development tasks.
• Collaborate with business stakeholders, product owners, architects, and cross-functional teams.
• Troubleshoot production issues, optimize performance, and ensure high availability of data solutions.
• Participate in solution design discussions, technical reviews, and estimation activities.
• Follow coding standards, documentation practices, and industry best practices.

Good-to-Have Skills

• Domain experience in Manufacturing or Retail
• Experience with Azure DevOps
• Knowledge of Delta Lake and Lakehouse architecture
• Experience with Data Warehousing concepts
• Exposure to Power BI
• Experience with cloud migration projects
• Certification in Azure Data Engineering or Databricks
• Knowledge of AI/ML data preparation pipelines

Required Soft Skills

• Excellent communication and interpersonal skills
• Strong analytical and problem-solving abilities
• Client-facing experience
• Ability to work independently as well as in a team
  • Strong requirement analysis and solutioning skills