Data Engineer
d4insight
Job Description
- Design and develop real-time streaming pipelines and enterprise data solutions.
- Implement and support Apache Kafka / Confluent Kafka platforms and integrations.
- Develop streaming applications using Apache Flink, Apache Beam, and PySpark Streaming.
- Optimize streaming applications for performance, scalability, and reliability.
Data Architecture & Warehousing
- Design end-to-end data flow architectures across enterprise systems.
- Work on Data Vault, Data Lake, Data Warehouse, and Data Mart implementations.
- Support enterprise data modeling including conceptual, logical, and physical data models.
- Manage metadata, data lineage, governance, and data quality initiatives.
Platform & Infrastructure
- Support deployments in Cloudera Data Platform (CDP) and Kubernetes environments.
- Recommend suitable relational and non-relational database solutions.
- Contribute to enterprise data platform modernization initiatives.