Senior Data Engineer

Data Processing & Pipelining – Data Requirements, Collection, and Infrastructure:

• Analyzes business requirements

• Analyzes data sources to ensure successful data extraction and pipeline development.

• Designs data models for optimal data processing and reporting performance.

• Designs and implements data pipelines for optimal data processing.

• Conducts comprehensive testing of deliverables to ensure quality, accuracy, and reliability.

• Identifies, analyzes, and troubleshoots data issues, such as mismatches, refresh errors, and inconsistencies.

• Analyzes deployment statistics, customer feedback, and platform enhancements to identify and drive product improvements.

Data Processing & Pipelining – Data Governance: 

• Implements data governance policies and procedures for data handling (e.g., data retention) to manage data consistency, integrity, accuracy, and reliability throughout the data lifecycle.

• Redacts Personally Identifiable Information (PII) and Protected Health Information (PHI) data to ensure compliance with data privacy and security standards.

• Follows data security measures to protect data from unauthorized access, use, disclosure, alteration, or destruction.

• Ensures data compliance with relevant laws, regulation, and industry standards.

Data Processing & Pipelining – Data Validation & Quality Assurance: 

• Implements rigorous data validation and integrity checks, identifying and addressing any data quality issues that could impact data pipeline and model performance.

• Designs and implements automation of data validation and governance.

Data Pipeline and Solutions Engineering – Pipeline Design: 

• Independently designs, develops, and optimizes automated and scalable data pipeline architectures to build data products.

• Implements appropriate data storage solutions to store the processed data to be used in a scalable and optimized way for access and analysis.

• Manages the flow of data pipeline and storage day-to-day operations.

Data Pipeline and Solutions Engineering – Data Solutions Engineering: 

• Works independently and collaboratively in an agile development environment with other engineers to develop, maintain, and debug data solutions that are scalable, efficient, cost effective, and reliable.

Core Responsibilities

Planning & Execution:

• Independently manages work, monitoring timelines and deliverables to ensure projects or initiatives stay on track and meet requirements.

• Proactively prioritizes work and adapts to resource or timeline shifts, suggesting adjustments to maintain project efficiency.

Collaboration & Partnership:

• Collaborates across teams to align on expectations and achieve shared objectives.

• Builds and maintains a comprehensive understanding of business, stakeholder, and/or customer needs to build and support effective partnerships.

• Actively listens to diverse perspectives and asks questions to ensure understanding of others.

Problem Solving:

• Independently identifies and addresses standard and non-standard issues in accordance with standard practices, escalating more complex issues as appropriate.

• Analyzes data and/or information from multiple sources to troubleshoot standard and non-standard errors.

• Contributes to knowledge sharing and best practices.

Continuous Learning:

• Embraces continuous learning by actively seeking to build knowledge and new skills and/or tools and staying current with industry trends and best practices.

• Seeks out and leverages feedback and training to improve skills.

• Contributes to a culture of continuous learning and knowledge sharing with team members.

Continuous Improvement:

• Develops ideas and recommends updates to increase the efficiency and effectiveness of processes, protocols, and workflows within a team.

  • Seeks input from team members on alternative approaches and methods