Data Architect

hirist

Kolkata 8 Years Exp Posted 1h ago

Job Description

 8+ years in data engineering or data platforms, including 4+ years hands-on with Databricks in production.

- 2+ years as architect or technical lead, owning design decisions for a platform or program.

- Led at least 2 end-to-end migrations to Databricks from legacy platforms (e.g. Hadoop, Teradata, Oracle, Informatica, Synapse, Snowflake), each at TB scale or above.

- Led the design and delivery of at least 2 end-to-end Databricks lakehouse implementations, covering medallion architecture, data modeling, batch and streaming, Unity Catalog governance, and non-functional requirements.

- Owned technical delivery of at least 1 Databricks program (team of 4+, 6+ months): scope, estimates, technical risks and timelines.

- Authored architecture artifacts used by delivery teams and customers: design documents, architecture decision records (ADRs), reference architectures and C4/solution diagrams.

- Production experience with Delta Lake, Unity Catalog, Lakeflow/Delta Live Tables or Workflows, and Structured Streaming.

- Advanced SQL and production-grade PySpark (testing, packaging, code review), Scala a plus.

- Deep Apache Spark expertise (execution model, shuffle, Catalyst/Tungsten, memory, AQE, Structured Streaming state) applied to architecture decisions.

- Deep expertise in one cloud (AWS, Azure or GCP) as it applies to Databricks, plus hands-on project experience in a second.

- Implemented CI/CD and infrastructure as code for Databricks in production (Databricks Asset Bundles, Terraform, GitHub Actions or Azure DevOps).

- Databricks Certified Data Engineer Professional.

- Consulting / customer-facing experience, working with external clients across a variety of industry markets.

Similar Openings for You