Data Engineering Lead

dataforpublicgood

Bangalore 5 Years Exp Posted 37d ago

Lead a team of data engineers, oversee resource allocation, and drive technical upskilling in AI-driven development and automation.
Act as the primary technical liaison for high-level government officials and agency heads to define data protocols and strategic integration roadmaps.
Evaluate APIs and datasets to design robust data models (JSON-LD, GeoJSON). Architect ETL modules that are containerized and ready for deployment.
Oversee the development of Python-based ETL modules for data ingestion via REST APIs and streaming protocols (AMQP, MQTT).
Own the integration delivery lifecycle using Agile methodologies, ensuring high-quality, on-time delivery of data flows.
Monitor data flows to identify and resolve integration errors, ensuring high availability and performance.
Create and maintain documentation for integration procedures, data maps, and technical specifications.
- Enforce software development best practices, ensuring rigorous unit/functional testing and maintaining the integrity of the Data Catalogue.