Team Lead, Data Platform
augury
Job Description
A Day In Your Life
- Lead and grow a team of ~5 Software Engineers in India, providing technical direction, coaching, and hands-on contribution (~30–50%).
- Own the technical vision and roadmap for DIH’s data platform in India, in close partnership with DIH leadership in Israel.
- Design and evolve scalable, data-centric backend services and pipelines that support Digital Twin and AI/agentic use cases.
- Set a high engineering bar for the team: clean code, testing, observability, and strong ownership culture.
Technical Leadership & Architecture
- Own the architecture and long-term evolution of DIH data services and pipelines built in India.
- Define and enforce data modeling standards (raw / modeled / aggregate layers, e.g., Bronze/Silver/Gold) across DIH systems.
- Make key architectural decisions across storage, lakehouse, and streaming systems (e.g., Databricks, Kafka), including trade-offs in cost, latency, and correctness.
- Lead design reviews and guide engineers through complex distributed systems and data pipeline decisions.
Production Data Systems
- Build end-to-end data pipelines from raw industrial events to validated, modeled, and aggregated datasets powering products and AI agents.
- Ensure pipelines are incremental, idempotent, and resilient to duplicates, late events, and schema changes.
- Design time-based aggregations and event-time correctness (watermarks, backfills, reprocessing).
- Model key industrial relationships (machines, sensors, factories, work orders, tenants) to support analytics and AI/graph-style queries.
- Ensure systems are observable and reliable in production (metrics, logging, tracing, data quality checks).
Streaming, Scale & Reliability
- Build systems that scale to high-volume industrial data workloads (hundreds of TB, millions of events/hour).
- Design streaming and event-driven systems with proper consumer patterns (idempotency, replay, lag monitoring).
- Make pragmatic trade-offs across partitioning, MERGE vs append-only, backfills, and cost vs performance.
- Ensure multi-tenant isolation and correctness across schemas, keys, and access patterns.
AI-Native & Cross-Functional Collaboration
- Partner with DIH and AI/ML teams to make data models and pipelines agent-ready (structured schemas, deterministic outputs, tool-friendly queries).
- Help design data interfaces and tools used by AI agents to answer complex industrial questions with traceability and correctness.
- Collaborate with Product and Engineering to translate requirements into scalable data architectures.
- Represent the India team in cross-functional and cross-region forums.
Team Leadership & Culture
- Hire, onboard, and develop strong data engineers in India.
- Coach engineers on system design, data modeling, testing, and production ownership.
- Build a culture of accountability, engineering excellence, and continuous improvement.
- Drive effective distributed collaboration across India and Israel teams.