Data Engineer
bayer
Job Description
- Apply cloud and big-data engineering skills to build, test, and maintain data pipelines and data-product components using tools such as Spark, dbt, Airflow, or equivalent cloud-native services.
- Execute ingestion, transformation, and loading jobs that reliably move data from operational systems, IoT devices, or clinical sources into data platforms at agreed SLAs.
- Troubleshoot pipeline failures, data-quality anomalies, and performance bottlenecks, diagnosing root causes and implementing fixes without service disruption.
- Validate that implemented pipelines meet data contracts, schema standards, and access-control policies defined by data architects and governance teams.
- Optimize pipeline performance and cloud infrastructure costs by profiling job runtimes, partitioning strategies, and resource configurations.
- Document pipeline designs, data lineage, and operational runbooks in shared repositories accessible to data consumers and support teams.
- Coach less-experience colleagues on data-engineering patterns, testing practices, and cloud-platform tooling used in the assigned domain.
WHO YOU ARE:
Required -
- Bachelor’s degree in Computer Science, Data Engineering, Information Technology, or a related field, 6+ years of experience
- Proficiency in SQL, Python, PySpark, Spark, or another Programming language, such as Java, Scala or R
- Proven experience building highly scalable, performant, resilient, and cost-optimized data pipelines
- Experience with modern data processing frameworks, such as Snowflake, Databricks, or AWS SageMaker Unified Studio
- Hands-on experience with core AWS services, including Amazon S3, IAM, Lake Formation, AWS Glue, Lambda, and related services required to build, operate, and secure end-to-end data solutions
- Strong understanding of relational databases and modern lakehouse table formats (Apache Iceberg, Delta Lake)
- Expertise in design, build, maintain and document conceptual, logical, and physical data models
- Sound understanding of data governance principles, access control, data cataloging, lineage, encryption, auditability, and compliance requirements. Ability to implement secure cloud environments and role-based access controls using IAM and Lake Formation.
Preferred –
- Familiarity with the concept of Retrieval-Augmented Generation (RAG) data pipelines
- Familiarity with data architecture patterns, such as data hub, mesh, or fabric
- Experience with RESTful APIs and data integration
- Familiarit with the Data Product concept and hands-on implications
- Relevant certifications (e.g., GCP Certified, AWS Certified, Azure Certified)
- Demonstrated experience with Terraform for infrastructure automation, together with Git, GitHub, GitHub Actions, and CI/CD practices.
- Experience with GxP and computerized system validation