Senior Associate Data Engineering L2

• Design and deliver scalable data engineering solutions to address complex business requirements.

• Build and optimize data ingestion pipelines for batch and real-time processing from multiple data sources.

• Implement robust data processing, storage, and computation frameworks ensuring performance and scalability.

• Drive architecture discussions and ensure the overall health and efficiency of data solutions.

• Develop analytics, aggregation, and search capabilities to enable data-driven decision-making.

• Collaborate with cross-functional teams to integrate data platforms with business applications.

• Implement cloud-based data solutions leveraging AWS, Azure, or GCP platforms. 

Qualifications

Your Skills & Experience:

• 6+ years of IT experience, with 3+ years in data engineering or related technologies.

• Hands-on experience with Big Data technologies and cloud platform Azure (Multi-cloud is added advantage)

• Strong programming expertise in Scala, or Python

• Experience with Hadoop ecosystem tools such as Spark, Hive, Kafka, Flink, NiFi, and Airflow.

• Working knowledge of NoSQL and MPP data platforms like MongoDB, Cassandra, Redshift, or BigQuery.

• Understanding of data integration, data wrangling, analytics pipelines, and cloud data services. 

• Strong understanding of IAM, data security, and cloud-based data platform services.

Additional Information

Set Yourself Apart With: 

• Experience with ETL tools such as Informatica or Talend and relational databases.

• Knowledge of data governance tools and processes (Collibra, Alation, lineage, cataloging).

• Exposure to messaging frameworks like ActiveMQ, RabbitMQ, and microservices architecture.

• Experience in performance tuning and optimization of data pipelines

• CI/CD practices including cloud infrastructure provisioning and automated deployment pipelines.

• Relevant cloud or big data certifications along with Gen AI experience.