Senior AI/ML Engineer

• Build and deploy production LLM agents that automate detection, triage, and remediation of platform and application issues
• Design and tune RAG pipelines over runbooks, incident history, logs, code, and internal documentation, including chunking strategy, embedding selection, vector store design, hybrid and semantic search, and reranking
• Implement agent orchestration: tool/function calling, multi-step planning, multi-agent handoffs, memory, and state management
• Integrate agents with observability and security platforms (alerting, logging, metrics, vulnerability and incident data) through APIs and MCP-style tool interfaces
• Build the supporting layer agents depend on: tool wrappers, guardrails, human-in-the-loop approval paths, and fallback behavior
• Own model and prompt engineering, including model selection, prompt and context optimization, output structuring, and cost/latency tuning; fine-tune or apply parameter-efficient techniques where warranted
• Build evaluation pipelines and define quality metrics for agent and RAG behavior (retrieval precision and recall, groundedness, hallucination and false-positive rate, escalation rate, MTTR impact) and iterate based on results
• Ship agent capabilities into EVA so developers consume them inside their existing workflow
• Partner with SRE, InfoSec, and platform teams to scope which classes of issues are safe to automate and which require human review

• Contribute reusable patterns, components, and documentation for agent development across the Optum AI DLC

• Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements

Required Qualifications:

• Bachelor's degree in Computer Science, Data Science, Machine Learning, or a related field, or equivalent experience
• 4+ years of software or machine learning engineering experience
• 2+ years hands-on building with LLMs, including prompt engineering, tool/function calling, and agent or multi-step orchestration
• Demonstrated experience designing and shipping RAG systems in production, including embeddings, vector databases (for example Azure AI Search, Pinecone, Milvus, pgvector, or FAISS), and retrieval evaluation
• Experience integrating third-party APIs and building backend services that run in production
• Experience with cloud platforms (Azure, AWS, or GCP), containers, and CI/CD pipelines
• Experience building LLM evaluation and testing workflows rather than relying on manual spot checks
• Proficiency in Python, with working knowledge of agent frameworks (for example LangGraph, LangChain, LlamaIndex, Semantic Kernel, or CrewAI)
  • Proven ability to work across teams and translate operational pain points into technical requirements