Senior Data Engineer

Key Skills Required

• Strong SQL (complex queries, joins, aggregations)

• Python for data manipulation and processing

• Azure Databricks

• PySpark

• Data Engineering fundamentals

• ETL / ELT pipeline design and development

• Data pipeline development

• REST API integration

• Basic understanding of Azure / cloud platforms

• Azure Data Factory (ADF) is good to have.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for batch and real-time processing

• Build and optimize ETL/ELT workflows using Azure Databricks and PySpark

• Work with large datasets to ensure data quality, reliability, and performance

• Develop efficient SQL queries for data extraction, transformation, and analysis

• Integrate data from multiple sources using REST APIs and other ingestion methods

• Collaborate with data scientists, analysts, and business teams to deliver data solutions

• Implement and support data engineering best practices across the platform

• Monitor, troubleshoot, and optimize data pipelines for performance and cost efficiency

• Ensure secure and compliant handling of data across cloud environments (Azure)

• Support continuous improvement of data architecture and engineering processes