Overview

This chapter explains how RAG learns to use retrieved passages without a labeled supporting document for every training example. Its answer-level loss marginalizes document-conditioned probabilities, rewarding candidate mixtures that make the target likely rather than selecting one human-designated source. During reported fine-tuning, the query encoder and BART generator change while the document encoder and index stay fixed. At inference, RAG-Token uses a next-token mixture, whereas RAG-Sequence needs document-specific beam search and Thorough or Fast Decoding. Retrieved-document count K changes recall, computation, and metric behavior rather than guaranteeing improvement.

Learning outcomes

RAG’s answer loss provides weak supervision for evidence assignment: it rewards target likelihood after retrieval and generation interact, not a single labeled source.
Fine-tuning updates the query encoder and BART generator in the reported setup; the document encoder and index remain fixed.
Increasing K can expose useful passages but also adds computation or distraction, and its measured effects vary by formulation, task, and metric.

Keywords

RAGSupervisionMarginalizationDecodingTrainingInference

Learning to Use Evidence: Training, Marginalization, and Decoding

This chapter explains how the original RAG system learns to use retrieved passages when training examples usually provide an input and an answer, but not a labeled supporting document. The focus is the connection between the latent-document objective and the choices made at inference time: which components change, which stay fixed, and how the number of retrieved passages affects cost and behavior. Readers will be able to interpret “weakly supervised retrieval” precisely rather than treating it as either fully supervised search or unexplained magic.

The two RAG formulations differ in how they assign documents to an answer, but both need a way to learn from that assignment. If a training example contains a question and a target answer, the system can be rewarded for predicting the answer. It may not be told which passage should receive credit. RAG uses the answer probability after combining candidate passages as the training signal. The document remains latent: it participates in the calculation without becoming an explicit label in the training record.

The answer loss teaches a hidden retrieval decision

Suppose a training pair contains an input x and a target sequence y. The retriever produces a distribution over the top-K candidate passages. For each candidate, the generator estimates how likely the target is when that passage is supplied with the input. RAG combines those document-conditioned probabilities, weighted by the retriever’s preferences, and minimizes the negative log of the resulting marginal probability. In simpler terms, the system is rewarded when the answer becomes likely through one or more of the passages it retrieved.

The calculation has two levels. Within one candidate passage, the generator’s sequence probability is built from the probabilities of its tokens in order: the first token, then the next token given the first, and so on. Across candidates, RAG combines those whole-sequence probabilities using the retriever’s document weights. A passage can therefore help by making an entire target sequence more likely, while another passage can also contribute if it makes the same target likely for a different reason. The training signal is not choosing between “retrieval” and “generation”; it is measuring how their joint path explains the target.

This objective teaches usefulness rather than a single prescribed explanation. If two passages can both help predict the target, the marginal calculation can preserve both contributions. If a passage is highly ranked but does not help the generator predict the answer, its weight need not make it decisive. The training signal can therefore adjust the query encoder and the generator together toward combinations that work for the downstream task. It does not require an annotator to mark one passage as the only correct source for every example.

That is the practical meaning of a latent retrieval choice. The system learns from the consequence of retrieval—how candidate passages affect the target probability—rather than from a complete, directly labeled retrieval trace. This is especially useful when the desired output is not a verbatim span. A passage can contain a clue, a definition, or a related expression that helps the generator produce the target even if no annotation identifies that passage as the official explanation.

The arrangement should not be described as “no supervision.” The retriever used to initialize RAG comes from DPR models trained with retrieval supervision for TriviaQA and Natural Questions.

Published by DeepMox

FAQ

Is RAG retrieval fully supervised?

No. During RAG fine-tuning, evidence assignment is weakly supervised by answer likelihood, although the initial DPR retriever was trained with retrieval supervision on TriviaQA and Natural Questions.

Which parts of RAG are updated during fine-tuning?

In the paper’s reported fine-tuning, the query encoder and BART generator are updated, while the document encoder and index remain fixed. Updating the document side would require recomputing representations and rebuilding the index.

What happens when RAG retrieves more documents at test time?

Retrieving more documents changes the candidate evidence and increases document-conditioned work, but it can also add useful, irrelevant, or conflicting material. The paper’s effects vary by task, formulation, and metric.

Previous10 minRAG-Sequence vs RAG-Token

Next9 minRAG Results: What the Experiments Show

How RAG Learns Evidence

Learning to Use Evidence: Training, Marginalization, and Decoding

The answer loss teaches a hidden retrieval decision

FAQ