Senior Data Engineer

LLM-in-the-pipeline architecture — the patterns for extraction, enrichment, entity resolution, and semantic validation over millions of records a day: structured outputs, retries, human-review fallbacks, and the abstractions others reuse

Eval harnesses — design the labelled sets, scorers, judge calibration, and regression suites that gate every prompt or model change; own precision/recall targets per check

The rule-vs-LLM standard — deterministic checks (dbt tests, data contracts, SQL) wherever structure allows; LLMs only where semantic judgement is needed; codified so the team can apply it without you

Cost and drift — token budgets per pipeline, model routing (cheap models for classification, stronger ones for hard cases), drift detection on vendor updates, and the token math behind the ceilings

Observability — every LLM call logged with prompt version, model, cost, latency, and decision, alongside pipeline alerting that surfaces problems before they cascade

Warehouse and transformation architecture — dbt layers that encode real business logic; Snowflake performance, clustering, materialisation, and cost over billions of rows

Orchestration and infrastructure — Airflow patterns for recovery and cost-aware scheduling; AWS infrastructure as code

Matching and deduplication — embeddings and retrieval patterns for company and people entity resolution across 13 markets

• Cross-functional voice — technical voice for the platform in architecture decisions with sourcing, data quality, product, and analytics