Senior Data Engineer

- Design, develop, and optimize data pipelines using Databricks and Apache Airflow.

- Implement PySpark-based transformations and processing in Databricks for handling large-scale data.

- Develop and maintain SQL-based data pipelines, ensuring performance tuning and optimization.

- Create Python scripts for automation, data transformation, and API-based data ingestion.

- Work with Airflow DAGs to schedule and orchestrate data workflows efficiently.

- Optimize data lake and data warehouse performance for scalability and reliability.

- Integrate data pipelines with cloud platforms (AWS, Azure, or GCP) and various data storage solutions.

- Ensure adherence to data security, governance, and compliance standards.