Core AI Engineer
sycamore
Job Description
- Build provider-independent agent loops, state management, streaming, context handling, recovery, and graceful degradation.
- Develop the learning data plane around agents: structured trajectories, feedback and outcome signals, offline datasets, lineage, privacy controls, and reliable links between an agent version and its behavior.
- Create evaluation systems for task completion, tool use, long-horizon behavior, safety, latency, cost, accessibility, and business outcomes.
- Build offline analysis and improvement loops that can surface failure clusters, grow golden datasets, and propose changes to prompts, tools, skills, context, retrieval, memory, policies, and model routing.
- Design experiment and versioning systems for comparing changes through replay, shadow traffic, canaries, or controlled rollouts, with clear promotion and rollback criteria.
- Explore where approaches such as preference learning, reward modeling, reinforcement learning, or continual learning can improve agent behavior—and recognize when a simpler evaluation or systems change is the better answer.
- Build detectors that identify regressions and opportunities across quality, reliability, cost, latency, and user adoption without mistaking noisy proxy metrics for real progress.
- Design typed tool interfaces and protocol-based execution across internal capabilities and customer-authorized services.
- Build connectors and credential flows with explicit read/write modes, permissions, auditability, and predictable failure behavior.
- Develop memory extraction and retrieval while enforcing tenant, user, and project visibility boundaries.
- Build durable orchestration for long-running tasks, checkpoints, approvals, handoffs, timers, and human-in-the-loop interactions.
- Publish reliable APIs, event-driven interfaces, reusable libraries, and pluggable improvement strategies that work across different agent categories and enterprise deployments.
- Build and refine React product surfaces for agent configuration, live execution, traces, evaluations, approvals, and improvement workflows.
- Own full-stack product slices across UI state, streaming, typed APIs, backend services, and persistence, with attention to accessibility, responsiveness, and failure recovery.
- Translate powerful platform primitives into product interactions that enterprise users can understand and trust.
- Debug difficult failures across models, prompts, tools, state, data, experiments, APIs, and distributed services.
- Join customer conversations when a repeated requirement or production outcome reveals a missing horizontal capability.