Data Engineer

- Design scalable batch and streaming architectures
- Develop advanced Airflow DAGs with dynamic task mapping
- Build robust REST API ingestion pipelines
- Implement Kafka producers/consumers
- Write and optimize SQL/PLSQL queries and data models for transformations
- Ensure data quality, monitoring, and reliability
- Error handling, fault tolerance. Data triaging
- Ensure idempotent and incremental data loads
- Api rate limits and retries at scale

Required qualifications to be successful in this role:

- 7-10+ years of experience in Data Engineering
- Strong Python programming
- Expertise in Apache Airflow, DAG Coding, Kafka
- Experience with REST API integrations
- Advanced SQL and PL/SQL knowledge
- Experience handling large-scale data pipelines
- Cloud Platforms (AWS/GCP/Azure)