Data Engineer

advanceautoparts

Hyderabad 4 Years Exp Posted 2h ago

Job Description

  • Help in the migration and modernization of data platforms, moving applications and pipelines to Google Cloud-based solutions.

  • Architect and maintain cloud-based data infrastructure leveraging AWS or GCP services.

  • Ensure data security and governance, enforcing compliance with industry standards and regulations.

  • Develop and promote best practices for data modeling, processing, and analytics.

  • Mentor and guide a team of data engineers, fostering a culture of innovation and technical excellence.

  • Manage and scale data pipelines from internal and external data sources to support new product launches and ensure high data quality.

  • Develop automation and monitoring frameworks to capture key metrics and operational KPIs for pipeline performance.

  • Collaborate with internal teams, including data science and product teams, to drive solutioning and proof-of-concept (PoC) discussions.

  • Develop and optimize procedures to transition data into production. Define and manage SLAs for data products and operational processes.

  • Research and apply state-of-the-art methodologies in data and Platform engineering.

  • Create and maintain technical documentation for sharing knowledge.

  • Develop reusable packages and libraries to enhance development efficiency.

  • Lead and drive the development and optimization of scalable data architectures and pipelines.

    • Develop real-time and batch data processing solutions, integrating structured and unstructured data sources.