Big Data Developer

griddynamics

Hyderabad 6 Years Exp Posted 14d ago

Resposibilities

Build and optimize big data solutions on AWS services such as S3, EMR, Glue, Athena, and EKS/Kubernetes.
Develop, schedule, and monitor workflow orchestration pipelines using Apache Airflow.
Execute and manage Spark jobs on Kubernetes/EKS environments ensuring performance, scalability, and reliability.
Implement and maintain data lake architectures leveraging Apache Iceberg for efficient data management and governance.
Collaborate with cross-functional teams including Data Architects, Analysts, and Business stakeholders to understand data requirements and deliver robust solutions.
Optimize Spark workloads, query performance, and resource utilization for large-scale datasets.
Ensure data quality, security, consistency, and adherence to best practices across data platforms.
Troubleshoot production issues, perform root cause analysis, and provide timely resolutions.
Contribute to CI/CD implementation, automation, and infrastructure improvements for data engineering platforms.
Work with Hadoop ecosystem components such as YARN, HDFS, and Hive for data storage and processing when required.
Participate in code reviews, technical discussions, and knowledge-sharing sessions within the team.

Qualifications

Desired data engineering skills:

We offer