Senior Data Engineer, WW FBA Central Analytics

amazon

Bengaluru, India 7 Years Exp Posted 1d ago

Job Description

- Architect and implement a scalable, cost-optimized S3-based Data Lakehouse that unifies structured and unstructured data from disparate sources across 8 WW FBA metrics domains.
- Lead the strategic migration from Redshift-centric architecture to a flexible lakehouse model, targeting query performance improvement from 60–300 seconds to under 10 seconds.
- Establish metadata management with automated data classification and lineage tracking.
- Design and enforce standardized data ingestion patterns with built-in quality controls and validation gates.
- Architect a centralized metrics repository that becomes the single source of truth for all FBA metrics across various time grains.
- Implement robust data quality frameworks with staging-first policies and automated validation pipelines.
- Design extensible metrics schemas that support complex analytical queries while optimizing for AI retrieval patterns, including multi-dimensional drill-down across Time → Geography → Category.
- Develop intelligent orchestration for metrics generation workflows with comprehensive audit trails.
- Lead the design of semantic data models that balance analytical performance with AI retrieval requirements for LLM-powered insight generation.
- Implement cross-domain federated query capabilities with sophisticated query optimization techniques.
- Architect vector database infrastructure capable of managing large-scale embeddings with consistent low-latency retrieval.
- Integrate schema definitions through MCP service calls to enable automated, AI-accessible data contracts.
- Build and own monitoring and alerting frameworks for all data pipelines, ensuring proactive failure detection and rapid resolution.
- Establish runbooks, schema change management processes, and data quality SLAs that move the team from reactive data consumers to proactive insight generators.

Basic Qualifications

- Experience with SQL
- Experience mentoring team members on best practices
- 7+ years of data engineering experience
- Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
- 5+ years of data warehouse technical architectures, data modeling, infrastructure components, ETL/ ELT and reporting/analytic tools and environments, data structures and hands-on SQL coding experience
- Experience with programming/scripting (Batch, VB, PowerShell, Java, C#, Chef, Perl, Ruby and/or PHP), or experience in any Bigdata architecture and experience that includes strong analytical skills, attention to detail, and effective communication abilities
- Experience including, building and maintaining data flows and pipelines
- 4+ years of working with Data & AI related technologies, including, but not limited to, AI/ML, GenAI, Analytics, Database, and/or Storage experience

Preferred Qualifications

- Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
- Experience operating large data warehouses
- Experience with training and deploying machine learning systems to solve large-scale optimizations, or experience with data infrastructures: relational analytic DBMS, Elastic-Search, and Big Data EMR/EC2/Glue/Lambda
- Experience in data mining, ETL, etc. and using databases in a business environment with large-scale, complex datasets
- Experience in machine learning, data mining, information retrieval, statistics or natural language processing, or experience in developing and deploying LLMs in production on GPUs, Neuron, TPU or other AI acceleration hardware
- Experience in building analytic or scientific data products or solutions
- Experience in any Bigdata architecture, or experience in Redshift and experience in managing firewalls
- Experience leading technical initiatives and key deliverables
- Experience leading large teams, with demonstrable ability to hire, develop, and manage high-performing technical teams

Similar Openings for You