RAG Is a Data-Preparation and Retrieval System
Dify's retrieval-augmented generation path is a data-preparation and search system, not a checkbox that makes a model knowledgeable. The specific question is how source material becomes retrievable units, how a query selects and ranks those units, and how Dify reconstructs context and source metadata before a workflow or Agent sees it. That view gives the reader a way to reason about RAG quality as a chain of choices rather than as a property of generation alone.
Indexing decides what can later be found
Return to the support assistant. If an employee asks about an expense policy, the answer depends first on whether the relevant policy was transformed into an addressable unit. Dify's indexing path extracts document content, cleans it according to processing rules, splits it into units, associates files, counts tokens, and stores the result in one or more searchable forms. Every later retrieval decision is constrained by what this preparation step preserved or discarded.
The ordinary paragraph path begins with extracted text and a segmentation rule. The rule can be automatic or explicitly configured, and it controls the maximum unit size, overlap, and separator behavior. Dify cleans the document before splitting it, drops empty results, assigns a node identifier and content hash, and retains document and dataset identity in metadata. The identifier lets the platform address a unit later; the hash helps relate an indexed version to its content. These are operational details with a semantic consequence: the system is not searching an undifferentiated document, but a collection of named pieces whose boundaries shape meaning.
Chunk size is therefore a decision about context, not just storage. A unit that is too broad may contain several policies and overwhelm a model with competing conditions. A unit that is too narrow may preserve the sentence “receipts are required” while losing the exception that follows it. Overlap can keep nearby language available across boundaries, but it also creates duplicate candidates and more material to rank. Dify provides the machinery for these choices; the repository does not establish one setting as best for every language, document type, or question.
Dify supports a parent-child structure for cases where the unit that should be searched is not the unit that should be shown as context. A parent can represent a larger paragraph or a full document, while child chunks are split more finely and used for high-quality vector retrieval. At query time, a child hit can be mapped back to its parent segment, with child content and scores retained as supporting detail. This can improve the chance of locating a precise phrase without forcing the final model to reason from a fragment that has lost its surrounding conditions.
The trade-off is not free. More granular children create more index objects and another reconstruction step. A parent may restore useful context but also reintroduce irrelevant material. A score attached to a child is not automatically a score for every sentence in the parent. Dify's retrieval service explicitly rebuilds the relationship between child chunks and parent segments, which makes that compromise visible instead of pretending that a single vector result represents the entire information structure.
A QA structure takes a different approach. The source is transformed into question-and-answer pairs, either from a supplied format such as a CSV or through an LLM-based formatting path. The question becomes the searchable content while the answer is retained as metadata associated with the indexed record. This can work well when users ask questions that resemble the intended question forms, but it adds dependence on the transformation model and on the quality of the generated pairs. A QA index is not a neutral copy of the source; it is a new representation with its own coverage and failure modes.