Senior Data Engineer
unitedhealthgroup
Job Description
- Develop, enhance, and maintain scalable data pipelines to support enterprise analytics and reporting needs
- Design and optimize data processing workflows using PySpark, Python, and Snowflake, following data warehousing and data modeling best practices
- Support and optimize Databricks workloads for batch and streaming use cases
- Manage cloud storage and data movement using Azure Blob Storage and AZ Copy
- Develop AI tools to minimize the manual workload
- Build scalable ETL/ELT pipelines in Databricks using PySpark, Spark SQL, Delta Live Tables, and workflows
- Perform activities which include platform stability, incident resolution, and operational support
- Identify and remediate security vulnerabilities, ensuring compliance with enterprise and healthcare data security standards
- Execute tools, framework, and software version upgrades to maintain platform currency and reliability
- Create, manage, and monitor Azure Data Factory (ADF) pipelines across development, test, and production environments
- Collaborate with cross functional teams to troubleshoot issues, improve performance, and deliver reliable data solutions
- Contribute to documentation, knowledge transfer, and continuous improvement of the data platform
- Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements