Associate, AI Data Scientist
blackrock
Job Description
Work forward-deployed with researchers, data specialists and product teams to understand workflows, frame ambiguous problems, define measurable outcomes and iterate rapidly from evidence to production.
· Research, design and deliver agentic workflows spanning planning, retrieval, tool use, structured extraction, reasoning, verification, exception handling and human escalation, using single-agent or multi-agent patterns where each is demonstrably appropriate.
· Engineer advanced agent patterns, including bounded iterative loops, reflection and critique, planner-executor and evaluator-optimizer patterns, with explicit stop conditions, budgets and safeguards that prevent unproductive or uncontrolled execution.
· Build agent and model harnesses that standardise tool interfaces, context assembly, state and memory handling, policy enforcement, tracing, replay and failure recovery.
· Determine when an agentic architecture, retrieval workflow, specialised model or deterministic service is the best fit, and validate the decision against representative tasks and simpler baselines.
· Design safe caching and reuse across retrieval, prompts, model responses, tool results and intermediate agent state, with clear cache keys, invalidation, freshness, access-control and provenance requirements.
· Build systematic evaluation harnesses using representative datasets, explicit quality dimensions, adversarial and edge-case testing, regression suites, human feedback and monitored production outcomes.
· Research and benchmark open-source SLMs and LLMs for task fit, considering quality, inference performance, cost, licensing, security, deployment constraints and operational supportability; apply fit-for-purpose routing and batching where they improve efficiency.
· Develop model-adaptation strategies using prompt and context optimisation, supervised fine-tuning, parameter-efficient fine-tuning or other suitable techniques, supported by governed dataset creation, experiment tracking, reproducibility and controlled promotion criteria.
· Operate the lifecycle for trained and fine-tuned open-source models, including data and model versioning, lineage, experiment and artifact management, evaluation gates, registry and release controls, deployment, performance and drift monitoring, rollback, retraining triggers and retirement.
· Build production-quality Python components that integrate models with enterprise data, retrieval services, APIs, workflow orchestration and human-review experiences.
· Improve grounding and trust through attribution, provenance, uncertainty handling, validation checks, access controls and clear separation of source evidence from model-generated conclusions.
· Own delivered reliability across testing, observability, latency, cost and maintainability, and diagnose model, data, prompt and orchestration failures in production.
· Document findings, reusable patterns and trade-offs; contribute to design reviews; and communicate results clearly to research, product, engineering, risk and senior stakeholders.
· Partner with engineering, product, risk, privacy, information security, legal and compliance colleagues to support responsible, secure and accountable use of AI.