vault / agents / rag.md

Retrieval

Agents & LLMs 67 words 2 outgoing 1 backlinks

Retrieval-augmented generation finds documents, then writes an answer from them. Keep the two scores separate: recall asks whether the expected sources reached the context, groundedness asks whether the answer stayed inside them. A wrong answer with good recall is a writing problem; poor recall is a search problem, and the fixes differ. Check the expected-source set itself before trusting either number.

Read the full example: related post.

LINKS IN THIS NOTE

[[ Evaluating an agent ]] A fluent answer proves nothing, so evaluate the loop rather than the vibe. Compare relevance, retrieval recall, groundedness and latency against a baseline, and read examples next to the scores: grounded, useful and found-the-right-document are three different failures. Offline golden... [[ Feedback triage ]] A thumbs-down starts an investigation, not a diagnosis. The workflow replays the question, collects evidence, separates a content gap from a configuration issue, and routes to an owner; routing is ordinary code, the model only proposes the likely cause. Keep...