SSE Data Engineer

- Design, develop, and maintain scalable data engineering solutions using Microsoft Azure services.
- Build and optimize ETL/ELT pipelines using Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
- Develop distributed data processing solutions using Apache Spark, PySpark, and Python.
- Implement and manage data lake architectures using Azure Data Lake Storage (ADLS) Gen2.
- Write complex SQL queries and perform performance tuning for large-scale analytical workloads.
- Design and implement dimensional data models and data warehouse solutions.
- Develop and maintain batch and real-time data processing pipelines using Azure Event Hubs and Azure Functions.
- Work with modern data formats and lakehouse technologies, including Delta Lake, Apache Iceberg, and Apache Hudi.
- Implement data pipeline orchestration using Azure Data Factory, Apache Airflow, Databricks Workflows, or Microsoft Fabric pipelines.
- Follow DevOps practices, including version control, CI/CD, and Infrastructure as Code using Terraform or Bicep.
- Implement data governance, metadata management, monitoring, and security using Azure Monitor, Azure Key Vault, and Microsoft Purview.
- Collaborate with architects, business stakeholders, and cross-functional teams to deliver reliable and high-quality data solutions.
- Troubleshoot data pipeline issues, optimize performance, and ensure data quality, security, and reliability.