AI Engineer
workable
Job Description
- Design, prototype & deploy retrieval‑augmented generation systems: Architect scalable RAG pipelines that combine vector search, hybrid retrieval, re‑ranking and contextual compression techniques. Build and integrate vector search systems (e.g. Milvus, pgvector, FAISS, Weaviate) for high‑recall retrieval across structured and unstructured data.
- Develop hybrid retrieval and knowledge‑driven pipelines: Design hybrid retrieval systems that blend semantic, symbolic and graph‑based methods. Create custom chunking and encoding strategies to store operational knowledge in vector databases and knowledge graphs.
- Build knowledge graphs & integrate them into retrieval workflows: Architect knowledge graphs (Neo4j, RDF, custom schemas) and integrate them into retrieval workflows to support reasoning and decision‑making.
- Optimise data pipelines and embeddings: Build and optimise data pipelines that convert incoming documents into high‑quality embeddings for AI retrieval. Tune chunk sizes, indexing frequencies and embedding strategies to enhance recall, factual accuracy and efficiency.
- Implement hybrid search & metadata filtering: Combine semantic and keyword search to improve precision and efficiency. Experiment with metadata filtering techniques to surface the most relevant context for AI reasoning agents.
- Evaluate & monitor system performance: Evaluate end‑to‑end retrieval performance using classical IR metrics (precision, recall) and LLM‑specific evaluations (factuality, coherence, task success). Monitor retrieval logs and adjust embedding configurations to maintain relevance and mitigate hallucinations.
- Compare & fine‑tune LLMs: Compare performance of different LLMs (e.g. GPT‑4, Claude, Llama) across embedding structures and refine tuning strategies. Implement quantisation, distillation and optimisation techniques to meet latency, throughput and cost targets.
- Collaborate & enable teams: Work cross‑functionally with product managers, data engineers and domain experts to translate product goals into scalable AI solutions. Conduct workshops and enablement sessions to enhance AI literacy across internal teams.
- Ensure quality & compliance: Participate in rigorous code reviews and implement testing frameworks to ensure reliability, security and compliance. Continuously monitor model accuracy and safety, and uphold data governance and ethical guidelines.