Senior Data Engineer

• Data Integration and Management: Lead initiatives focused on integrating, transforming, and managing large datasets from diverse sources, ensuring optimal availability and performance.
• Advanced SQL Queries: Analyze complex queries to resolve data issues; write ETL code to meet project requirements.
• Azure Data Factory: Leverage Azure Data Factory to orchestrate data workflows and pipelines, implementing best practices for cloud-based integration and processing.
• Azure Pipelines Deployment: Manage CI/CD pipelines using Azure Pipelines, automating and streamlining deployment processes.
• Kubernetes Deployment: Oversee deployment and management of data solutions within Kubernetes environments, ensuring application scalability, reliability, and security.
• Python and Shell Scripting: Develop custom scripts in Python and shell for tasks related to data processing, automation, and system integration.
• Project Leadership: Lead multiple projects simultaneously, coordinating among development, QA, and business teams; provide technical leadership and mentorship to junior team members.
• Requirement Analysis: Work with stakeholders to understand business requirements and design and execute builds to fulfill those needs.
• Configuration and Execution: Configure systems according to project specifications and execute data builds efficiently.
• API Configuration: Understand and configure APIs using Python code.

About you

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• 7 or more years of experience in data engineering with a strong background in technical leadership.
• Proven expertise in SQL, data warehousing, and cloud environments (preferably Microsoft Azure).
• Hands-on experience with Azure Data Factory, Azure pipelines, Kubernetes, Python, shell scripting, and Snowflake or similar technologies.
• Experience handling extensive datasets and developing ETL code to import data into a datamart.
• Demonstrated ability to collaborate with cross-functional teams in a dynamic setting.
• Strong problem-solving and analytical capabilities.
• Retail domain experience is a plus.
  • Excellent communication and interpersonal skills.