Senior Data Engineer

Bangalore NM Years Exp Posted 9d ago

Design, build, and maintain scalable data pipelines and curated data assets on Databricks to support reporting, analytics, executive dashboards, self-service BI, and commercial performance management.
Develop reliable ETL/ELT processes to ingest, transform, validate, and publish data from multiple sources using advanced SQL, Python, and PySpark.
Leverage Databricks notebooks, Unity Catalog, job scheduling, and performance optimization tools to ensure efficient and reliable data delivery.
Create reusable, maintainable, and well-documented pipelines aligned with enterprise data engineering standards.
Monitor, troubleshoot, and optimize pipeline performance, reliability, and availability of BI-ready datasets.

Build trusted, reusable datasets, data marts, reporting tables, and views for Power BI, Tableau, and other analytics platforms.
Partner with BI teams to translate reporting requirements, KPIs, and business logic into scalable data solutions.
Design business-ready datasets that enable executive reporting, operational analytics, commercial scorecards, and self-service BI.
Create reusable data products that accelerate BI delivery and ensure accurate reporting outcomes.

Design and maintain dimensional models, including fact and dimension tables, star schemas, reporting views, and semantic layers.
Translate business requirements into scalable data structures supporting KPI tracking, trend analysis, drill-down reporting, and self-service analytics.
Collaborate with BI developers to optimize data models for performance, usability, consistency, and maintainability.
Document data structures, business rules, dependencies, assumptions, and refresh processes.

Implement data quality controls, validation routines, reconciliations, and monitoring processes to ensure trusted reporting.
Investigate and resolve data quality issues, pipeline failures, performance bottlenecks, and reporting discrepancies.
Partner with stakeholders to validate outputs and address root causes of data issues.
Support data observability and promote accurate, transparent, and business-ready data assets.

Apply Git and version control best practices to manage code, notebooks, scripts, and reusable assets.
Follow development standards for branching, code reviews, release management, and documentation.
Write clean, modular, and maintainable SQL, Python, and PySpark code.
Contribute to reusable frameworks, templates, and engineering standards that improve quality, consistency, and team efficiency.

What your background should look like:

Master’s degree in Computer Science, Data Engineering, Data Science, Information Systems, Engineering, Business Analytics, or a related discipline.
Databricks Data Engineer Associate, Databricks Data Engineer Professional, Databricks Lakehouse Fundamentals, or other relevant Databricks certifications.
Relevant AWS data, analytics, or cloud certifications.
Experience working with Amazon Redshift, including legacy asset support, query analysis, data migration, or reporting-layer modernization.
Experience with Databricks Unity Catalog, notebook-based development, job scheduling, and performance optimization.
Experience coaching, mentoring, or nurturing junior data engineers, analysts, or BI developers.
Experience working with commercial, sales, distribution, pricing, marketing, customer care, inventory, POS, or supply chain data.
- Familiarity with Power BI data consumption patterns, dashboard performance needs, dataset design, and self-service BI enablement.