Lead Infrastructure Engineer

KEY ACCOUNTABILITIES

• Experience on configuring, administering and maintaining Google Cloud Composer and Tidal Enterprise Scheduler platforms.
• Planning the installation, upgrades & configurations of above products in Google Cloud Platform.
• Experience on configuring, managing, troubleshooting & optimizing cloud composer (Airflow).
• Design, implement, and manage infrastructure to support scheduling workflows and orchestration pipelines.
• Monitor platform performance, ensuring high availability, scalability, and reliability of Composer and Tidal environments.
• Troubleshoot and resolve technical issues related to Tidal, Composer, and supporting infrastructure tools.
• Perform platform upgrades, patches, and routine maintenance to ensure security and stability.
• Setting up CI/CD pipelines for owned solutions.
• Setting up Monitoring dashboard and implement observability in all our owned services.
• Design and implement Disaster Recovery strategies for owned tech stacks.
• Identify, design, and implement internal process improvements by automating manual processes, optimizing data delivery for greater scalability, etc.
• Work with stakeholders including the Product, Developers, Data & Analytics teams and support their infrastructure needs.
• Comply with audit and other reporting requirements.
• Lead cultural change for adoption of new technologies.
• Own the technical/product road map for Integration.
• Translation of strategy and architecture into a highly available and secure technical implementation, in accordance with best practices
• Continuous support and collaboration with other data engineers, developers, data and analytics teams and projects.
• Maintain comprehensive documentation of configurations, workflows, processes, and best practices.

MINIMUM QUALIFICATIONS

• Education – Full time graduation from an accredited university (Mandatory- Note: This is the minimum education criteria which cannot be altered).
• Workflow Management: Google Cloud Composer & Airflow jobs
• Cloud: Google cloud platform (GCP).
• Experience managing Google Cloud Composer, Tidal Scheduler from a platform/infrastructure perspective.
• Terraform
• Scripting Languages: Python, Bash
• Experience with other integration tools like Kafka, or MuleSoft.
• Knowledge of CI/CD pipelines and related tools (e.g., Jenkins, GitHub Actions).
• Ability to manage multiple priorities in a dynamic environment.
• Strong aptitude to learn and passion for problem solving.
• Excellent communication skills in coordinating with different stakeholders.

PREFERRED QUALIFICATIONS

• Strong knowledge of infrastructure components, including networking, storage, and compute in cloud environments.
• Familiarity with Agile methodologies
• Familiarity with modern Software Engineering principles
• Workflow Management: Google Cloud Composer & Airflow jobs
• Cloud: Google cloud platform (GCP)
• Experience managing Google Cloud Composer, Tidal Scheduler from a platform/infrastructure perspective.
• Hands-on experience with monitoring and logging tools for performance and issue resolution.