Lead Software Engineer - Data Engineering

Leadership & Delivery
• Lead and mentor a team of data engineers and platform developers
• Drive Agile execution and ensure predictable, high-quality delivery
• Establish engineering best practices, code quality, and CI/CD standards
Data Platform & Architecture
• Architect scalable and secure data platforms on AWS
• Design robust data ingestion frameworks for batch and near real-time pipelines
• Define best practices in data modeling, governance, and metadata management
Data Engineering & Ingestion
• Lead design and development of scalable ingestion pipelines (structured and unstructured data)
• Build and optimize Snowflake-based data platforms for performance and cost
• Enable ingestion of diverse sources (databases, APIs, files, streaming data)
Cloud & Platform Engineering
• Leverage AWS services (S3, Glue, Lambda, EMR, Redshift, etc.) for end-to-end pipelines
• Implement CI/CD pipelines using Azure DevOps / Jenkins
• Ensure system scalability, resiliency, and operational readiness
Software Engineering Excellence
• Enforce software engineering principles (modular design, code quality, testing, version control)
• Drive automation and continuous improvement
• Promote reusable frameworks for ingestion and transformation
Stakeholder Collaboration
• Partner with product managers, SMEs, and business stakeholders
• Translate business needs into scalable data solutions

What You Have:

• 10+ years of experience in Data Engineering / Data Platform roles
• Strong experience in AWS data ecosystem (S3, Glue, Lambda, EMR, Redshift)
• Deep expertise in Snowflake (architecture, optimization, data modeling)
• Strong programming skills in Python and SQL
• Extensive experience with data ingestion pipelines and ETL/ELT frameworks
• Exposure to real-time streaming (Kafka, Spark Streaming)
• Experience with CI/CD tools (GitHub, Jenkins, AWS CloudFormation etc.)
• Solid understanding of distributed systems and scalable architectures
• Strong foundation in software engineering principles (Git, testing, design patterns) Experienced in working with Agile teams
• Collaborate with Data Science and AI teams to operationalize ML models and analytics workflows.
• Promote integration of AI capabilities into data engineering pipelines (e.g., GenAI, MCP, ATA).
• Support real-time analytics and edge AI use cases in manufacturing environments.
• Use AI extensively in building and testing Data Ingestion and Data pipeline
  • This position requires candidate to work a 5-day -a -week schedule in the office