Lead Platform Engineer

Technology Leadership:

• Work independently to and lead design, development and evolution of big-data ecosystem and data lake architectures to enable data processing, storage, and retrieval, including the integration of structured data sources from enterprise transactional systems and trusted third parties.

• Define and manage cloud infrastructure services through automation, CI/CD and best practices. 

• Troubleshoot data pipelines and cloud related issues.

Architecture Review

• Communicate with stakeholders to understand requirements and propose cost effective and performant architecture design.

• Present platform architecture, roadmap, plans, status, and risk to various stakeholders, including senior engineers.

• Champion DevOps best practices and ensure the data engineering teams follow all the recommended architecture guidelines while delivering data products.

Platform Engineering

• Champion scalable and cost-effective data and advanced analytics platforms, evangelize best practices for building cutting-edge data and analytics ecosystem.

• Work closely with senior engineers and various stakeholders to understand requirements, create roadmaps, keep track of key milestones, and take accountability for the delivery of new platforms developments.

Continuous improvement

• Develop and enable processes that allow for continuous improvement through structured feedback, error/variation detection, performance measurements, data security evaluations, and data quality measures. 

Project Execution & Management

• Demonstrate a combination of business focus, strong analytical and problem-solving skills, and programming knowledge to be able to quickly iterate through PoC in the discovery phase of the project to a enterprise scale production implementation of a GxP solution 

Team Management

• Work with vendors and technology partners to select and implement new data solutions and provide operational support for data solutions already deployed.

• Work with a team of data architects and data engineers that design, develop, and maintain data pipelines, data warehouses, and other data infrastructure.

Collaboration & communication

• Collaborate and work closely with executives, stakeholders and business teams to effectively communicate architecture strategy & clearly articulate the business value

Required Skills & Qualifications:

• BS or MS in engineering, sciences, or equivalent relevant experience required.

• 6+ years of experience in building and managing data and analytics applications is required.

• Demonstrated expertise on Azure, especially on data services like Databricks, ADF etc.

• Proficiency in Python programming and Data engineering stacks

• Suitable candidate be able to demonstrate strong experience in the following areas:

  • Data Engineering –

    • Proficient on modern Data engineering stack such as Data bricks, Delta Lake, Spark

    • Hands-on experience working with Azure Functions, logic apps,

    • Power BI, Power Apps, Azure Security controls.

    • Working knowledge of devops pipelines using tools like GIT and Azure DevOps Pipelines

    • Good understanding of working with structured as well as unstructured data

  • Project Management & Communication Skills:

    • Strong project management understanding with experience in executing projects using Agile delivery framework.

    • Sound knowledge of project management tools such as ADO, Jira, MS Projects.

    • Strong analytical, organizational, interpersonal, and time management skills 

    • Excellent communication (written and verbal) and interpersonal skills