Data Engineering Consultant

• Data Pipeline Development: Design, develop, and maintain efficient and scalable data pipelines using Python and Azure technologies
• Data Integration: Integrate data from various sources, ensuring data quality and consistency
• Azure Cloud Services: Utilize Azure cloud services for data storage, processing, and analytics, including Azure Data Factory, Azure Databricks, and Azure SQL Database
• CI/CD Implementation: Implement and manage CI/CD pipelines to automate the deployment of data solutions
• Data Modeling: Develop and maintain data models to support business intelligence and analytics needs
• Performance Optimization: Optimize data processing and storage solutions for performance and cost-efficiency
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs
• Documentation: Create and maintain comprehensive documentation for data pipelines, processes, and solutions
• Troubleshooting: Identify and resolve data-related issues and bottlenecks
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Builder Responsibilities:
Design, develop, and deploy AI-powered solutions using no-code, low-code, and advanced platforms, translating business needs into scalable applications that enhance products, workflows, and decision-making.

Required Qualifications:

• Bachelor's or Master's degree in Computer Science, Information Technology, or equivalent in related field
• 7+ years of experience in data engineering, with a strong focus on Python, Azure, ADB,ADF, CI/CD, and deployment
• Technical Skills:
  • Extensive experience with Azure cloud services (Azure Data Factory, Azure Databricks, Azure SQL Database, etc)
  • Experience with containerization and orchestration tools (e.g., Docker, Kubernetes)
  • Knowledge of SQL and NoSQL databases
  • Familiarity with data warehousing concepts and technologies
  • Solid understanding of CI/CD principles and tools (e.g., Jenkins, GitLab CI/CD)
    • Proficiency in Python for data processing and automation