Data Engineer

bayer

Bengaluru 6 years experience Posted 3h ago

Job Description

  • Apply cloud and big-data engineering skills to build, test, and maintain data pipelines and data-product components using tools such as Spark, dbt, Airflow, or equivalent cloud-native services.
  • Execute ingestion, transformation, and loading jobs that reliably move data from operational systems, IoT devices, or clinical sources into data platforms at agreed SLAs.
  • Troubleshoot pipeline failures, data-quality anomalies, and performance bottlenecks, diagnosing root causes and implementing fixes without service disruption.
  • Validate that implemented pipelines meet data contracts, schema standards, and access-control policies defined by data architects and governance teams.
  • Optimize pipeline performance and cloud infrastructure costs by profiling job runtimes, partitioning strategies, and resource configurations.
  • Document pipeline designs, data lineage, and operational runbooks in shared repositories accessible to data consumers and support teams.
  • Coach less-experience colleagues on data-engineering patterns, testing practices, and cloud-platform tooling used in the assigned domain.

 

WHO YOU ARE:

Required -

  • Bachelor’s degree in Computer Science, Data Engineering, Information Technology, or a related field, 6+ years of experience
  • Proficiency in SQL, Python, PySpark, Spark, or another Programming language, such as Java, Scala or R
  • Proven experience building highly scalable, performant, resilient, and cost-optimized data pipelines
  • Experience with modern data processing frameworks, such as Snowflake, Databricks, or AWS SageMaker Unified Studio
  • Hands-on experience with core AWS services, including Amazon S3, IAM, Lake Formation, AWS Glue, Lambda, and related services required to build, operate, and secure end-to-end data solutions
  • Strong understanding of relational databases and modern lakehouse table formats (Apache Iceberg, Delta Lake) 
  • Expertise in design, build, maintain and document conceptual, logical, and physical data models 
  • Sound understanding of data governance principles, access control, data cataloging, lineage, encryption, auditability, and compliance requirements. Ability to implement secure cloud environments and role-based access controls using IAM and Lake Formation.

Preferred –

  • Familiarity with the concept of Retrieval-Augmented Generation (RAG) data pipelines
  • Familiarity with data architecture patterns, such as data hub, mesh, or fabric
  • Experience with RESTful APIs and data integration
  • Familiarit with the Data Product concept and hands-on implications
  • Relevant certifications (e.g., GCP Certified, AWS Certified, Azure Certified)
  • Demonstrated experience with Terraform for infrastructure automation, together with Git, GitHub, GitHub Actions, and CI/CD practices.
  • Experience with GxP and computerized system validation

 

Similar Openings for You