Inside the Pipeline: Retrieval, Tools, and Memory

This chapter locates commercial influence in the system stages that precede and follow an AI assistant’s visible answer. The focus is not source code or a vendor inventory; it is the path by which a commercial signal can change the context, the available options, the next action, or the memory carried into a future conversation. Tracing that path helps explain why an upstream decision can matter more than the final sentence and why an audit that sees only the answer may miss the decisive intervention.

The distinction from the shopping experiment is important. There, researchers could compare recommendation behavior and inspect the language shown to participants. In a production architecture, the user may see neither the retrieval ranking that supplied the evidence nor the tool choice that constrained the options. The answer can look organic because the commercial influence entered earlier, at an interface the user never encountered.

Retrieval decides what the model gets to see

A retrieval-augmented generation system, often shortened to RAG, separates an answer into two broad operations. First, it finds material relevant to a query: the system represents the query, retrieves documents, may rerank them, and assembles a context. Second, the language model generates a response from that context. The division is useful because it shows that commercial influence does not have to be inserted into the generated prose. It can change which evidence is available before generation begins.

Suppose documents in the retrieval index are ranked not only by relevance but also by an advertiser-provided signal. The system will be more likely to place some documents in the context window. The model may then produce a response that is factually faithful to what it was given while emphasizing the category or narrative represented by those documents. A user asking for help choosing a product might receive an answer that makes one class of products seem like the natural starting point, not because the model explicitly announced a paid preference, but because the upstream context made that class more salient.

This is a framing intervention. It can alter which alternatives are discussed, which evidence is treated as central, and what the user understands the problem to be. It may also affect a later action: if the retrieved material points toward one service, source, or follow-up query, the user’s effective decision context has changed. The source framework therefore treats retrieval bias as primarily a content-framing problem with the potential to induce behavioral redirection as well.

The audit challenge follows directly from the architecture. A conventional sponsored result can be inspected as a result, its position compared with an organic result, and its label evaluated. A RAG intervention may leave the user with only the final answer. To determine whether the context was commercially biased, an auditor may need access to the retrieval index, ranking scores, advertiser signals, and the no-intervention retrieval distribution. Those are often controlled by the system operator rather than exposed to an ordinary user or an external reviewer.