Senior Data Engineer
synoptek
Job Description
Key Responsibilities:
- Own the end-to-end design and evolution of the Medallion Architecture - defining ingestion patterns for Bronze, transformation and data quality standards for Silver, and data product delivery conventions for Gold, ensuring consistency across all platforms and teams
- Design, develop, and maintain scalable, high-performance data pipelines following Medallion Architecture and ETL/ELT workflows for batch and real-time data ingestion
- Develop and optimize Snowflake schemas, views, and data models aligned with the Medallion pattern to support analytical workloads and self-service reporting
- Build and optimize data models, tables, and views within Snowflake aligned with the Medallion pattern to support analytical and operational use cases
- Lead the design and implementation of real-time streaming data solutions to enable low-latency data availability across the organization
- Develop and manage Databricks workflows, notebooks, and Delta Lake pipelines for large-scale data processing
- Establish and maintain data governance practices across all Medallion tiers — including Unity Catalog policies, Snowflake RBAC, and data lineage tracking
- Mentor and technically guide junior and mid-level data engineers through code reviews, design reviews, pairing sessions, and documentation
- Collaborate with business stakeholders to understand requirements and deliver reliable, well-documented data products
- Apply CI/CD best practices and manage code through Git version control.
- Monitor, troubleshoot, and optimize data pipeline performance, and ensuring data quality
- Collaborate with business stakeholders to understand requirements and deliver reliable, well-documented data products
- Willingness to participate in developing the AI agents
Qualifications
Experience:
- 6 to 8 years of professional experience in data engineering or a related field
- Proven track record working with cloud-native data platforms (Snowflake, Databricks, or equivalent)
- Deep, production-proven expertise in designing and operating Medallion Architecture (Bronze/Silver/Gold) at scale using Databricks Delta Lake and/or Snowflake multi-layer schemas
- Extensive experience with Snowflake — advanced data modelling, performance optimisation, dynamic data masking, and row-level security
- Strong expertise in Databricks — cluster management, Delta Live Tables, and Unity Catalog
- Experience with real-time and streaming data architectures in production environments
- Knowledge of data orchestration tools such as Azure Data Factory
- Hands-on experience designing and building ETL/ELT pipelines for large-scale data environments
- Familiarity with data warehousing concepts, dimensional modeling, and data lake architecture
- Experience working in Agile/Scrum development environments
- Exposure to CI/CD pipelines and DevOps practices for data workflows
- Proficiency in PySpark and/or Python for data transformation and pipeline development
- Strong SQL skills including query optimization, window functions, and complex joins
- Experience with cloud platforms such as Azure, AWS, or GCP
Skills/Attributes
- Synoptek core DNA behaviors:
- Clarity: Possesses excellent communication skills, makes a concentrated effort to speak the customer’s language. Ability to field questions with concise, well-constructed responses
- OwnIT: Shows integrity, innovation, and accountability in completing daily assignments
- Results: Solutions focused and driven to resolve conflict quickly and precisely. Proactively looks for opportunities to contribute to the company’s business goals
- Growth: Willing to learn and ask questions. Constantly looking for new ways to improve yourself. Ability to adapt and grow in a fast-paced environment
- Team: Embraces both customers and colleagues as team members. Ability to be flexible, respectful, engaged and collaborative