Data Engineer

• Design, implement, and maintain secure and scalable data ingestion pipelines from a wide variety of source systems, including SAP, SalesForce, SharePoint, APIs, and (legacy) manufacturing platforms.

• Build and enhance metadata-driven services that enable discoverability, access governance, and operational transparency of enterprise data.

• Serve as a technical expert and cross-functional enabler for structured and unstructured data acquisition, quality, and compliance.

• Establish and maintain holistic data quality management, monitoring and reporting.

• Contributes to a global data engineering team delivering to all major business domains.

• Drives ingestion and metadata service implementation for over 100 enterprise data sources.

• Collaborates across business IT, cybersecurity, infrastructure, and architecture teams to ensure secure and sustainable delivery.

Main Tasks:

• Build and maintain Python- or Scala-based extraction services (e.g., Debezium Server, custom APIs, rclone)

• Implement CDC, delta, and event-based patterns.

• Push-based HTTP, Kerberos-authenticated DLT delivery.

• Establish, operate and troubleshoot extraction from SAP using tools like Theobald Extract Universal.o

• Integrate with systems such as Salesforce, SharePoint, and other API- or file-based endpoints.   

• Establish and maintain a business-friendly, web-accessible data catalog application, with dataset profiles, metadata, and usability features.

• Integrate dataset discoverability, preview/exploration options, and lineage information using Unity Catalog as a backend metadata system.

• Design and implement structured access request workflows including request submission, approval chains, audit trail, and enablement triggers.

• Perform design reviews with Cybersecurity.

• Ensure documentation and compliance for all interfaces and data ingress points.

• Manage audit and traceability requirements.

• Collaborate closely with IT and business users to translate requirements into scalable technical patterns.

• Serve as technical escalation point for complex source integration. 

• Define and implement a multi-layered data quality framework, including unit-level, integration-level, and cross-pipeline validation rules.

• Establish centralized and version-controlled storage of DQ rules, with integration into orchestration and CI/CD pipelines.

• Implement automatic DQ monitoring with severity levels (Critical, High, Medium, Low) and enable flagging, filtering, and quarantining logic at relevant stages of the pipeline.

  • Collaborate with source system owners and business stakeholders to define meaningful and actionable DQ thresholds.