Software Development Engineer III

• Architect, design, and build highly scalable, secure, and resilient Data Engineering platforms using Databricks, Apache Spark, and Scala.
• Lead the design and implementation of large-scale distributed systems, ensuring high availability, performance, and operational excellence.
• Define and drive the technical roadmap for data platforms, lakehouse architectures, and enterprise data products.
• Design and implement advanced ETL/ELT pipelines capable of processing large-scale batch and real-time workloads.
• Build and govern engineering standards, coding guidelines, architecture principles, and operational best practices across teams.
• Lead end-to-end feature development, from requirement gathering and solution design to implementation, deployment, and production support.
• Collaborate with Product Managers, Architects, Platform Teams, and Business Stakeholders to translate business goals into scalable technical solutions.
• Drive technical excellence through code reviews, architecture reviews, performance optimization, and engineering mentorship.
• Design and implement observability frameworks including monitoring, logging, tracing, alerting, and reliability engineering practices.
• Establish and maintain robust CI/CD pipelines and DevOps practices for data and AI platforms.
• Lead the adoption and implementation of Agentic AI and Enterprise AI solutions across business use cases.
• Design, develop, and orchestrate multi-agent AI systems utilizing modern agent frameworks and orchestration platforms.
• Define and implement LLM evaluation frameworks including quality, reliability, accuracy, safety, performance, and cost optimization metrics.
• Guide teams in selecting the right architecture patterns, technology stack, and engineering practices for long-term success.
• Mentor and develop engineers, enabling them to grow technically and professionally while fostering a culture of ownership and innovation.
• Continuously evaluate emerging technologies, AI advancements, and industry best practices to drive innovation and strategic differentiation.

You will need

• 9+ years of professional software engineering experience with a strong focus on Data Engineering and distributed systems.
• Expert-level experience with Databricks, Apache Spark, and Scala/Python.
• Deep expertise in designing and building scalable, high-throughput, low-latency data platforms and distributed systems.
• Strong understanding of Data Structures, Algorithms, System Design, and Software Engineering principles.
• Proven experience architecting and implementing large-scale ETL/ELT pipelines in enterprise environments.
• Strong expertise in modern Data Lake, Lakehouse, and Data Platform architectures.
• Hands-on experience implementing CI/CD pipelines using tools such as GitHub Actions, Azure DevOps, Jenkins, or similar platforms.
• Experience building observability solutions including monitoring, logging, tracing, alerting, and reliability engineering practices.
• Strong knowledge of data governance, security, privacy, compliance, and data quality frameworks.
• Experience gathering product requirements and converting business needs into scalable technical solutions.
• Demonstrated ability to lead technical initiatives, influence architecture decisions, and drive execution across multiple teams.
• Strong stakeholder management and communication skills, with the ability to work effectively across engineering, product, and business teams.
  • Proven track record of mentoring engineers, setting engineering standards, and leading high-performing teams.