RAG: Retrieval-Augmented Generation Explained

Overview

This series explains the original RAG design: a query retrieves top-K passages from an external text index, while BART uses those candidates, its learned parameters, and marginalization to generate an answer. It follows corpus preparation, DPR retrieval, token-level conditioning, RAG-Sequence and RAG-Token, weak evidence supervision, decoding, and the paper’s experiments. The results show task-specific gains and more direct memory revision and inspection, but not guaranteed truth: coverage, ranking, context use, source quality, and claim support remain separate questions.

Learning outcomes

RAG separates explicit, replaceable text from BART’s learned language ability; retrieval changes available evidence, not the generator’s parameters.
Top-K retrieval and marginalization keep candidate passages in play, but a high score or visible passage is not verification.
RAG-Sequence favors document-level coherence, while RAG-Token favors multi-passage composition; the better choice depends on answer structure and failure cost.
The paper reports task-specific gains, yet corpus coverage, retrieval misses, generator additions, source quality, and metric limits prevent a universal factuality guarantee.

Keywords

RAGRetrievalGenerationBARTDPRMarginalizationGroundingProvenanceFAISSWikipedia

Chapters

RAG’s Two Memories

Why external text improves access without proving claims

7 min · Why external text improves access without proving claims

7 min

RAG External Memory Explained

How chunks, vectors, and top-K search shape evidence

9 min · How chunks, vectors, and top-K search shape evidence

9 min

RAG: From Query to Sentence

How candidate passages shape each next token

9 min · How candidate passages shape each next token

9 min

RAG: Who Knows What?

Trace evidence from query to generated claim

7 min · Trace evidence from query to generated claim

7 min

RAG-Sequence vs RAG-Token

Coherence, composition, and decoding trade-offs

10 min · Coherence, composition, and decoding trade-offs

10 min

How RAG Learns Evidence

Weak supervision, fixed indexes, and test-time cost

8 min · Weak supervision, fixed indexes, and test-time cost

8 min

RAG Results: What the Experiments Show

Metrics, retrieval ablations, and factuality limits

9 min · Metrics, retrieval ablations, and factuality limits

9 min

What RAG Changes—and Cannot Guarantee

Access, updateability, and the evidence boundary

10 min · Access, updateability, and the evidence boundary

10 min

Preview every chapter · Unlock full text, audio, and video from chapter 4

Plans

Ch.1-3 FreeCh.4-8 Premium

FAQ

What does RAG add to a language model?

RAG adds a query-dependent, inspectable text index alongside the generator’s parametric memory. The index can be replaced without retraining BART, but retrieved text is context, not verified truth.

How does RAG use retrieved passages to generate text?

RAG encodes the input, retrieves top-K passages, and conditions BART on each candidate with that input. It combines document-conditioned probabilities to generate tokens rather than copying the top passage.

What is the difference between RAG-Sequence and RAG-Token?

RAG-Sequence uses one latent document for a whole answer, whereas RAG-Token marginalizes documents separately for each token. Sequence-level conditioning favors coherence; token-level conditioning permits more multi-passage composition but diffuses attribution.

Does RAG guarantee factual or grounded answers?

No. RAG does not guarantee factuality, sentence-level attribution, or source authority. A corpus can be incomplete, retrieval can miss or mis-rank evidence, and the generator can misread or extend a passage.