Senior AI/ML Specialist
oraclecloud
Job Description
LLM Fine-Tuning & Training Pipelines
- Implement and maintain supervised fine-tuning (SFT) and RLHF pipelines for Cyncly domain-specific models, leveraging proprietary product catalogue, documentation, and support data.
- Apply LoRA/QLoRA fine-tuning via Hugging Face PEFT; manage model checkpointing, hyperparameter tuning, and training run tracking. Work with quantised models (GGUF, AWQ, GPTQ) to optimise inference on GPU.
- Maintain reproducible experiments, model versioning, and artefact management using MLflow or Weights & Biases; collaborate with Data Engineering to curate and version domain datasets with PII scrubbing and data-quality checks.
Embeddings Infrastructure & Semantic Search
- Develop and optimise embedding pipelines using models such as BGE-M3 and E5-large; manage vector database ingestion, indexing strategies, and retrieval quality for Cyncly's product catalogues and documentation.
- Design, benchmark, and maintain vector database deployments (Qdrant, Weaviate, or equivalent), including index configuration, filtering strategies, and scalability planning.
- Implement retrieval quality metrics (MRR, NDCG, recall@k), run A/B evaluations of embedding model upgrades, and build hybrid search pipelines combining dense vector retrieval with BM25 sparse search to maximise product discovery accuracy.
RAG Architecture & Use Case Integration
- Design and implement RAG layers for Cyncly's five LLM use cases, ensuring context relevance, faithfulness, and latency targets; integrate MCP connectors to expose internal data sources and product APIs to the LLM inference layer.
- Build and maintain the developer copilot integration using Continue.dev and custom language-server extensions; implement prompt injection defenses, output filtering, and guardrail layers across all inference paths.
- Work closely with Product and Engineering squads to expose AI capabilities through REST/OpenAI-compatible APIs and integrate them into Cyncly's product surfaces.
Model Evaluation & Quality Assurance
- Write model evaluation harnesses and regression test suites; track and report quality metrics (BLEU, ROUGE, faithfulness score, hallucination rate, answer relevance) per deployed model and use case.
- Partner with the AI/ML QA team to design human evaluation workflows and validate model outputs before production release; maintain a model leaderboard and evaluation artefacts for all active models enabling data-driven decisions on upgrades and rollbacks.
- Implement continuous evaluation pipelines triggered on model updates, ensuring quality regressions are detected before reaching production.
Inference Infrastructure & Observability
- Deploy and manage LLM inference using vLLM or Text Generation Inference (TGI) on Cyncly's on-premises GPU infrastructure, targeting production-grade throughput and latency SLAs.
- Instrument inference latency, throughput (tokens/sec), GPU utilisation, and cost metrics; surface results to operational dashboards for the AI leadership team; participate in on-call rotation and incident response for model-serving issues.
- Contribute to the containerisation and orchestration of AI workloads using Docker and Kubernetes, ensuring deployments are reproducible, scalable, and integrated with Cyncly's CI/CD pipelines.
Knowledge Graphs & Advanced Data Pipelines
- Contribute to Cyncly's product knowledge graph (Neo4j) — entity-extraction pipelines, schema design, and graph query integration with RAG; build NLP pipelines to enrich product and document knowledge.
- Explore and prototype agentic AI patterns (tool use, multi-step reasoning) for complex information retrieval and action execution use cases.
- Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Machine Learning, or equivalent practical experience.
- 5–8 years of hands-on software and/or AI/ML engineering experience, with at least 2 years working directly with large language models or transformer-based systems.
- Strong Python programming skills; proficiency with PyTorch or JAX training ecosystems, and Hugging Face Transformers, PEFT (LoRA, QLoRA), and model evaluation frameworks.
- Working knowledge of