Data Engineer-Staff Software Engineer

What you'll do

• Partner with Product Managers, and Solution Architects to define and analyze customer’s technical requirements.
• Provides technical input to Statement of Work for External Service Providers
• Ability to break down product management stories into detailed design specifications for medium to complex business problems.
• Experience with building data pipeline from several business applications
• Ability to collaborate with business stakeholders to identify and meet data requirements.
• Guides team members on the use of design patterns and adherence to code quality metrics
• Uses software code management tools and processes to ensure code traceability and good governance.
• Conducts code reviews with other team members (either internal or external service providers)
• Makes continuous improvement to our coding and test practices across applications.
• Works in DevOps team which uses continuous development, integration and testing with automated releases to lower and higher environments with application monitoring plus supporting the product.
• Experience performing root cause analysis on all data and processes to answer specific questions and identify opportunities for improvement.
• Participates in recommending emerging technologies and application solutions.

Who you are

• Candidate with 9-11 years of experience in a Data Engineer role.
• Graduate degree in Computer Science or equivalent
• High Problem-Solving Skills with analytical thinking and logical reasoning
• Having experience working on Spark platform, Spark SQL, Python and common Python libraries.
• Working experience on cloud-based environments such as Databricks on Azure, ADF, PySpark etc.
• Hands on experience writing notebooks using Python/Spark
• Hands on experience in reporting tools like PowerBI, MicroStrategy.
• Well versed with relational databases, nonrelational databases, data streams, and file stores
• Experience with source control systems such as GitHub and Jenkins build and continuous integration tools.
• Automation using Linux shell & Python scripting.