Data Engineer
bahwancybertek
Job Description
- Agent Operation and Auditing: Interact with the multi-agent system, put the plan.md under scrutiny, and thoroughly review the generated code bundle (detecting bottlenecks such as data skew, excessive shuffles, or incorrect data typing).
- Parity Testing and Reconciliation: Design and run data parity validations certifying that the Databricks output matches the legacy source load 100% (volume, values, and schemas).
- Exception Management and Escalation: Independently resolve minor code warnings and promptly escalate NEEDS-HUMAN events, pattern changes, or semantic inconsistencies to the Architect.
- Code Quality and Security: Remediate findings from SAST tools (e.g., Fluid Attacks), dependency vulnerabilities, and non-conformities before opening the PR.
- Delivery Cycle (Git Flow): Manage feat/TICKET-slug branches, ensure strict compliance with local pre-checks, and lead the resolution of REJECTED tickets or defects raised in QA/UAT.
- Transition Stability: Sustain operations, rapid anomaly remediation, and pipeline telemetry during the post-deployment hypercare window until formal handover to support.