QA Engineer

Ensure data quality, reliability, and correctness across large scale Databricks batch pipelines through automated validation and testing practices.

• Own end to end data quality validation for Databricks pipelines across Bronze, Silver, and Gold layers
• Validate data completeness, accuracy, consistency, timeliness, and business rule correctness using Databricks SQL and Delta tables
• Perform dataset, record, schema, and transformation level validations including source to target mapping and derived field checks
• Design, develop, and maintain reusable and scalable data validation frameworks using PySpark and Python on Databricks
• Automate integration, regression, and end to end testing for notebooks, workflows, and scheduled jobs
• Define and implement test data management strategies including data refresh, masking, and environment isolation
• Manage test cases, execution, and defect tracking using Xray and Jira while maintaining test documentation and validation evidence
  • Collaborate with engineering, platform, and product teams to enforce data quality standards and communicate risks and release readiness