Data Engineer

Develop and maintain reliable, scalable, and efficient ETL/ELT data pipelines using technologies such as Azure Databricks, PySpark, Python/Scala, and SQL .

Design and implement data ingestion and transformation solutions for a variety of data sources, including relational, event-based, and unstructured data.

Build and maintain scalable Data Lake and Lakehouse solutions and optimize data processing and storage performance.

Develop physical data models and implement data storage architectures in accordance with established design and engineering guidelines.

Implement data quality checks, monitoring, alerting, and troubleshooting mechanisms to identify and resolve data quality and data integrity issues.

Analyze complex data elements, data flows, dependencies, and relationships to contribute to conceptual, logical, and physical data models.

Develop and operate large-scale data storage and processing solutions using distributed and cloud-based technologies.

Work with Azure services such as Azure Data Lake Storage (ADLS), Event Hubs, and Azure Functions to support scalable data solutions.

Implement and support data governance practices, including metadata management, data access, retention, and availability.

Participate in testing, validation, troubleshooting, and continuous improvement of data pipelines and solutions.

Collaborate with business stakeholders, analysts, data scientists, and IT teams to understand requirements and deliver analytics and data solutions.

Apply Agile development practices, including Scrum, Kanban, DevOps, and continuous improvement , to deliver data-driven solutions.

Document technical solutions, processes, data flows, and system dependencies to support knowledge sharing and effective solution maintenance.

• Apply appropriate engineering, security, governance, and compliance practices throughout the data development lifecycle.