Data Engineering

vanguardjobs

Hyderabad 8 Years Exp Posted 1h ago

Job Description

The Manager, Data Engineering is responsible for leading a team of data engineers to design, develop, and optimize scalable data pipelines and infrastructure. This role focuses on enabling high-performance data workflows, ensuring seamless data integration and driving automation to enhance data accessibility.

Core Responsibilities

  • Partners with internal and external clients to gain an expert understanding of business functions and informational needs. Works closely with other technical and data analytics experts across the business and external vendors to implement data solutions.

  • Hires, evaluates, and supervises crew. Provides guidance and training as necessary to develop crew. Sets performance standards, reviews performance, and makes informed compensation decisions in accordance with all applicable Human Resources policies and procedures.

  • Manages and optimizes the automation of key ETL (Extract / Transform / Load) and quality assurance processes for large volumes of data.

  • Consults with department or function heads on the implementation, transition, and operating plans. Executes and adjusts plans for team of Data Engineers. Develops a clear understanding of Data Analytics and Management work across Vanguard businesses and how they connect.

  • Manages the planning, prioritization, and execution of data production projects and processes.

  • Manages and troubleshoots the design and development of software and processes to support large scale, efficient data pipelines. Integrates complex and large scale data from a variety of sources for business partners to generate insight and make decisions. Supports dashboarding efforts, data science and engagement projects, and production support as needed.

  • Translates business specifications into design specifications and code. Responsible for writing complex programs, ad hoc queries, and reports. Ensures that all code is well structured, includes sufficient documentation, and is easy to maintain and reuse.

  • Develops and exercises cross-functional data delivery to support business needs. Manages data integrity and quality standards.

  • Educates and develops junior data engineers on the team while applying quality control to their work. Develops data engineering standards and contributes expertise to other data expert teams across Vanguard.

  • Participates in special projects and performs other duties as assigned.

Qualifications and Skills

  • Minimum 8 years of relevant work experience, with at least 5 years in a leadership role managing data engineering teams and large-scale data infrastructure.
  • Bachelor's degree (B.E./B.Tech) in Computer Science, IT, or a related field or Diploma in Data Science/Data Engineering.
  • Strong people leadership experience (teams with more than five people would be advantageous)
  • Strong Data/DataOps Engineering experience
  • Experience in medium to large business that consume large amounts of data
  • Excellent communication and storytelling skills; Ability to articulate complex problems in a simple manner.
  • First-hand experience with Python/Pyspark in building end-to-end data pipelines
  • Understanding DBA concepts; Prior experience working in/with DevOps or Data Engineers teams; Comfortable with DevOps/SRE thinking.
  • Knowledge and experience in designing, implementing, deploying and configuring AWS resources;
  • Experience with SCRUM-Agile mindset and implementation tools such as Jira, Azure DevOps
  • DataLake, Lakehouse & EDW methodologies; Data analysis, data modelling, data integration, data warehousing and database design; AWS Stack
  • Understanding of GIT, post/pre-scripting and CI/CD workflow under TDD using Jira/Azure DevOps
  • Support multiple environments (Dev, UAT, Prod) and a strong understanding of SDLC concepts.
    • Degree qualified in a relevant commercial, analytical or mathematical discipline or equivalent experience