The concept
RAG stands for retrieval-augmented generation. Before the model answers, the system looks up the relevant passages in your own documents and hands them to the model along with the question.
Why: a model only knows what it saw in pretraining (Day 18). It has never read your internal procedures. Asked anyway, it produces something plausible: a hallucination (Day 22).
RAG is an open-book exam instead of answering from memory.
Walkthrough: the wire cutoff question
Without RAG
- The model has never seen this bank's wire policy.
- It answers "Domestic wires must be submitted by 5:00 PM." Sounds confident. Could be wrong.
- Repeated to a customer, a payment may miss the real cutoff.
With RAG, in 3 steps
1. Retrieve
The question is matched against the bank's policy library in a vector database (Day 30). Say the top hit is "Wire Transfer Operations Procedure, section 3: Cutoff times", plus the current holiday calendar.
2. Augment
The prompt (Day 23) gets the question, those passages, and a system-prompt rule (Day 25): "Answer only from the provided sources. Cite the document and section. If the answer isn't there, say so."
3. Generate
Imagine the procedure says the cutoff is 3:30 PM local time. The model replies:
"Domestic wire requests must be submitted by 3:30 PM local time to go out today. Source: Wire Transfer Operations Procedure, section 3."
The employee can click through and check. And when the policy changes, you update the document, not the model. No retraining.
RAG reduces guessing; it doesn't remove it. If retrieval pulls the wrong section or an outdated version, the answer is wrong with a citation attached. Retrieval quality is where most of the work goes.
Where you see it at work
- Banking: procedure assistants for branch and call-center staff.
- Insurance: answering "is this covered?" from the actual policy wording.
- Healthcare: staff questions answered from approved clinical and admin protocols.
- Retail: support bots grounded in current return and shipping policies.
- Business analysis: querying large sets of requirements and process docs.
In one sentence
RAG retrieves the right passages from your own documents and gives them to the model, so answers are grounded, current and traceable to a source.