Big Data Developer

Build and optimize big data solutions on AWS services such as S3, EMR, Glue, Athena, and EKS/Kubernetes.

Develop, schedule, and monitor workflow orchestration pipelines using Apache Airflow.

Execute and manage Spark jobs on Kubernetes/EKS environments ensuring performance, scalability, and reliability.

Implement and maintain data lake architectures leveraging Apache Iceberg for efficient data management and governance.

Collaborate with cross-functional teams including Data Architects, Analysts, and Business stakeholders to understand data requirements and deliver robust solutions.

Optimize Spark workloads, query performance, and resource utilization for large-scale datasets.

Ensure data quality, security, consistency, and adherence to best practices across data platforms.

Troubleshoot production issues, perform root cause analysis, and provide timely resolutions.

Contribute to CI/CD implementation, automation, and infrastructure improvements for data engineering platforms.

Work with Hadoop ecosystem components such as YARN, HDFS, and Hive for data storage and processing when required.

• Participate in code reviews, technical discussions, and knowledge-sharing sessions within the team.