Data Engineer
texplorers
Job Description
- Data Pipeline Development: Design, develop, and maintain scalable data pipelines using Azure Synapse, Databricks & Apache Spark (PySpark).
- Data Integration: Integrate data from various sources, ensuring data quality and consistency.
- Performance Optimization: Optimize data processing workflows for performance and cost-efficiency.
- Collaboration: Work with data architects, analysts, and product owners to understand data requirements and deliver solutions.
- Monitoring and Troubleshooting: Monitor data pipelines and troubleshoot issues to ensure data integrity.
Documentation: Document data workflows, processes, and best practices.
- Monitoring and Troubleshooting: Monitor data pipelines and troubleshoot issues to ensure data integrity.