Data Engineer

Key Responsibilities

• Design, build, and maintain scalable data pipelines (ETL/ELT)

• Work with large datasets including structured and semi-structured data (CSV, JSON, etc.)

• Develop and optimize data ingestion and transformation workflows

• Build efficient distributed data processing solutions

• Integrate data from multiple sources using REST APIs and cloud services

• Ensure data quality, reliability, and performance across pipelines

Required Skills

• Strong proficiency in SQL (complex queries, joins, aggregations)

• 7–10 years (minimum 4+ years in Data Engineering)

• Hands-on experience in Python for data processing

• Strong knowledge of PySpark

• Experience with Databricks

• Solid understanding of data engineering concepts and architectures

• Experience building ETL/ELT pipelines

• Familiarity with cloud data platforms (preferably Azure)

• Experience with Azure Data Lake / Blob Storage

• Understanding of distributed data processing systems

Cloud & Integration Skills

• Azure fundamentals and data services

• REST APIs for data ingestion and integration

• Cloud-based data pipeline implementation

Good to Have

• Experience with Azure Data Factory (ADF)

• Workflow orchestration tools

  • Data pipeline scheduling and automation