Azure Databricks + Pyspark
cognizant
Job Description
- Design and implement scalable data architectures using Databricks SQL and PySpark to support business objectives and data-driven decision-making.
- Collaborate with stakeholders to understand requirements related to Medicare and Medicaid Claims and translate them into technical solutions.
- Develop efficient data pipelines and workflows to process large datasets ensuring accuracy and reliability.
- Optimize performance of data solutions by identifying bottlenecks and implementing best practices for data processing.
- Provide technical guidance to team members on Databricks SQL and PySpark fostering knowledge sharing and skill development.
- Ensure compliance with data governance policies and security standards while handling sensitive Medicare and Medicaid Claims data.
- Conduct regular reviews of data solutions to identify opportunities for improvement and implement enhancements.
- Work closely with business analysts and domain experts to ensure alignment between technical solutions and business needs.
- Monitor and troubleshoot data systems to ensure seamless operation and minimal downtime.
- Document all technical processes workflows and solutions to maintain transparency and facilitate future scalability.
- Stay updated with the latest advancements in Databricks SQL PySpark and healthcare data management to incorporate innovative approaches.
- Collaborate in a hybrid work model ensuring effective communication and coordination with remote and on-site team members.
- Contribute to the companys mission by delivering impactful data solutions that improve healthcare outcomes and operational efficiency.