AI Engineering - Databricks

Responsibilities

Responsibilities: • Design and build modern data pipelines and data streams. • Move/Transform data across layers (Bronze, Silver, Gold) using ADF, Python, and PySpark. • Develop and maintain data pipelines and data streams. • Work with stakeholders to understand their data needs and provide solutions. • Collaborate with other teams to ensure data quality and consistency. • Develop and maintain data models and data dictionaries. • Develop and maintain ETL processes. • Develop and maintain data quality checks. • Develop and maintain data governance policies and procedures. • Develop and maintain data security policies and procedures. • Provide technical guidance and mentorship to junior data engineers