Technical Lead - Data Engineering

Key Responsibilities

•  Strategic Delivery Management: Lead the end-to-end lifecycle of Data/AI projects, translating business goals into actionable technical roadmaps and AI strategies.
•  Architectural Excellence: Design and implement Medallion Data Lake architectures on Azure, ensuring seamless integration of structured and unstructured data.
• Hands-on Optimization: Act as the subject matter expert in PySpark and Databricks, directly overseeing the optimization of high-performance data pipelines and ETL/ELT patterns.
• AI Integration: Collaborate with Data Scientists to integrate models into production environments, ensuring smooth model operations and deployment.
• Governance & Security: Enforce strict data security and compliance standards.
• Technical Leadership: Guide a cross-functional squad of engineers and scientists through complex technical roadblocks, fostering a culture of excellence and continuous learning.

Technical Skills & Qualifications:

Core Tech Stack (Must-Have)

•     Languages: Expert-level PySpark, Databricks, Python, SQL.
•     Expertise in Azure (Azure Data Factory, Databricks, Azure Storage).
•     Data Architecture: Mastery of Data Modeling, Data Warehousing, and Medallion (Bronze/Silver/Gold) patterns.
•    Good understanding on Data Ingestion Frameworks and practices.
•   Databases: Snowflake, Azure Datalake , Lakehouse {delta lake}
•   Understanding of LLM framework will be a plus.

 AI & Machine Learning {Big Plus}:

•     Proficiency in ML/DL frameworks and the development-to-deployment lifecycle.
•     Strong experience integrating AI models with Spark-based data processing.
•   Experience on integration with LLM models and storage will be plus.

Integration & Infrastructure:

•      Strong understanding of Cloud Computing and Azur services.

Soft Skills & Leadership:

• Communication: Ability to articulate complex technical concepts to both engineering teams and non-technical stakeholders.
• Problem-Solving: A strategic thinker with a proven track record of resolving architectural bottlenecks.
• Collaborative Leadership: Ability to motivate teams and drive initiatives forward within a fast-paced corporate environment.
• Industry Awareness: Proactive engagement with emerging trends in Data Engineering, Generative AI, and Cloud Architecture.
• Continuous Learning.

Preferred Experience:

•     Prior experience in a lead or co-lead role for Data or AI product development . 
  •   Proven expertise in handling large-scale migrations to Azure-native data stacks.