Data Engineer
quest-global
Job Description
Data Engineering & Development
• Design, develop, and maintain scalable data pipelines using Python and SQL.
• Build robust ETL/ELT frameworks for structured and unstructured data processing.
• Develop data ingestion solutions from APIs, databases, cloud storage, and external
file sources.
• Perform data cleansing, transformation, enrichment, validation, and quality
assurance activities.
• Design reusable and maintainable data processing components.
Cloud & Big Data Platforms
• Develop and support cloud-native data solutions on AWS and/or Azure.
• Build and optimize data workflows using services such as AWS Glue, EMR, Lambda,
S3, RDS or Azure Data Factory, Databricks, Synapse, and ADLS.
• Work with large-scale datasets and distributed data processing frameworks.
Data Warehouse & Analytics
• Design and maintain data models, data marts, and analytical datasets.
• Support enterprise reporting and self-service analytics initiatives.
• Optimize database performance and query execution.
• Collaborate with Power BI teams on semantic models and reporting datasets.
Data Quality & Operations
• Implement data validation frameworks and quality controls.
• Develop monitoring, alerting, logging, and operational support processes.
• Troubleshoot production data pipeline failures and performance issues.
• Support ongoing maintenance and improvements of enterprise data platforms.
Engineering Excellence
• Follow coding standards, peer reviews, and software engineering best practices.
• Implement CI/CD pipelines for data engineering workloads.
• Maintain source control repositories using Git.
• Create technical documentation and knowledge-sharing materials.