Why Retrieval Is a Composite System
RAGFlow’s retrieval layer decides which pieces of a prepared knowledge base become the context for a particular question. This chapter focuses on the signals and gates behind that decision—lexical matching, vector similarity, filtering, fusion, reranking, and structural reconstruction—so you can tune a search without treating one similarity score as a verdict. The key judgment is that retrieval is a policy for selecting evidence under constraints, not a lookup of the nearest vector.
Why one score cannot serve every query
A vector represents a piece of text as a position in an embedding space. A query and a chunk that are close in that space may discuss the same idea even when they use different words. That is useful for a question such as “How do I lower the device’s operating limit?” when the manual uses a more technical phrase. But semantic closeness is not the same as exact identity. A product code, regulation number, error code, or model designation may need to match literally. Two documents can be about the same subject while naming different models, dates, or legal provisions.
RAGFlow addresses this difference by combining two broad signals. Full-text retrieval looks for terms and their analyzed forms. Vector retrieval looks for semantic proximity. Hybrid search fuses the results with configurable weights. Increasing the vector share favors paraphrases and conceptual similarity. Increasing the full-text share favors exact terms and fixed identifiers. Neither setting is globally correct because the meaning of “relevant” changes with the question.
Imagine a knowledge base containing service manuals for several similar products. A user asks, “What is the inspection interval for model AX-410 under high-temperature operation?” The model code is a discriminating identifier; “inspection interval” and “high-temperature operation” describe the requested relation and condition. A vector-only search might retrieve a passage about a neighboring model because the surrounding language is highly similar. A lexical signal can keep the exact identifier influential. Conversely, if the user asks, “When does the device need more frequent checks?” a lexical match may miss a passage that uses “shortened maintenance cycle,” while vector similarity can bridge the wording difference.
The fusion stage should therefore be understood as a negotiated score, not an objective truth. RAGFlow’s search settings expose a vector-similarity weight, with the remainder assigned to full-text similarity. The retrieval implementation carries both scores through the result, and later ranking can use the configured balance. The score is useful for ordering candidates under a stated policy; it does not tell an operator that a chunk entails the answer or that every relevant chunk was found.
The diagram separates two ideas that are often collapsed. EligibilityFilter represents conditions such as dataset or document scope, metadata, and whether a chunk is available for retrieval. FusionWeights represents how eligible candidates are scored. RankFeatures and Reranker represent later ordering decisions. A chunk can be highly similar and still be absent because a metadata condition excluded it. A chunk can pass the filter and enter the candidate pool but rank below another chunk. Debugging becomes easier when those stages are not described as one mysterious “relevance” operation.
Reranking adds another distinction. Initial retrieval is optimized to assemble a candidate pool broad enough to contain useful passages. A rerank model then examines the query and candidate content more directly and reorders that pool. This can improve final ordering when lexical and vector signals disagree, but it adds a model call and latency. It also cannot rescue a relevant passage that initial retrieval never admitted. Raising the candidate count may reduce that risk while increasing processing work; lowering it may make search faster while shrinking the opportunity for the reranker to find the best passage.
Similarity thresholds create a similar trade-off at a different point. A higher threshold filters more aggressively, which can reduce irrelevant context but omit useful material expressed in unfamiliar language. A lower threshold broadens recall but may send noise to the answer model. The right threshold depends on whether the application prefers a narrow answer from strong evidence or a broader context that a later stage can interpret. It should be established with representative questions rather than copied from a generic example.
The number of chunks returned also changes the answer problem. Too few results can omit a qualification, a second comparison item, or the other half of a multi-step question. Too many results can fill the model’s context with near-duplicates and make the answer less focused. RAGFlow keeps candidate and returned-result settings distinct because the system may need a broad pool for reranking while sending only a smaller, ordered set to the next component. The operator is deciding not only which chunks rank highly, but how much evidence the model can reasonably inspect.
Eligibility, ranking, and reconstruction
Before comparing scores, RAGFlow can narrow the universe of possible evidence. A retrieval request can name one or more datasets and can be restricted to particular documents. Metadata conditions can limit results by attributes such as a date, department, category, or other generated field. Availability state prevents disabled or intermediate material from participating in ordinary retrieval. These filters are not minor conveniences. They encode authority and scope. If a user is asking about a current policy, a stale or disabled fragment should not become eligible merely because its words resemble the query.
The practical consequence is that a relevance failure may actually be a scope failure. Suppose the correct service table exists but its document was not selected, its metadata filter used the wrong value, or its chunk was disabled after a review. Changing vector weight will not help. The candidate never reached the scoring stage. RAGFlow’s retrieval-testing guidance therefore asks operators to check whether the target chunk exists, whether it belongs to the expected document, whether a metadata condition excludes it, and whether the result set contains unrelated material. The order matters because it distinguishes “not searched” from “searched but ranked poorly.”
Tags and dataset priority add domain-specific signals. RAGFlow can associate tag sets with chunks and queries, and it can assign a PageRank-like priority to a dataset when multiple knowledge bases are searched together. These features let an organization express information that pure text similarity cannot: a chunk may be especially useful for a business domain, or one maintained knowledge base may deserve greater weight than another. Such signals are still policy inputs. They should be explainable to the people who maintain the knowledge base, because a hidden priority can make search results look arbitrary.
The same principle applies to cross-language retrieval. Translating or expanding a query can make a question match content written in another language, but the system still needs a compatible representation and a meaningful scope. A cross-language setting can expand recall; it does not prove that a translated phrase preserves the legal, technical, or operational distinction in the source. Test queries should include the languages and terminology the application actually receives.
RAGFlow also separates the unit used for recall from the unit used for explanation. Child chunks can provide focused search targets inside a longer parent section. After child hits are found, the retrieval path can reconstruct or promote a parent-level result so the answering model receives more of the surrounding context. This helps with a question whose decisive condition appears near, but not inside, the most similar sentence. It introduces a dependency, however: parent and child identifiers, enablement state, source positions, and ordering must remain aligned after edits or reparsing. A broken relationship can produce either an overly narrow answer or a context window padded with the wrong material.
A table makes the issue especially clear. A row-level fragment may be the right unit for locating a model and interval, while the table or its surrounding heading provides the condition under which the interval applies. If retrieval returns only a cell-like passage, the answer model may attach the value to the wrong column. If it returns the entire table without a focused ranking signal, the relevant row may be buried. RAGFlow’s combination of structured parsing, field-level retrieval, metadata, and parent/child behavior is designed to give the system more than one way to preserve that relationship. It remains the operator’s responsibility to verify the result with source previews and representative questions.
This is also why returned evidence should be read as a set, not as a single winner. A multi-part question may require two chunks that are individually less similar than a third, generic passage. A comparison needs both alternatives. A procedure needs the prerequisite, the action, and the exception. A definition may need the heading that establishes scope. RAGFlow’s candidate-pool and top-result controls let an operator trade context pressure against coverage, but the application’s answer requirement must decide which side of that trade matters. If the question asks for a comparison and the result contains only one alternative, a high score is evidence of a retrieval gap, not a reason to invent the missing side.
The result fields can help expose that gap. A high vector score with weak term similarity suggests that the query and passage are conceptually close but may use different language. A strong term match with low vector similarity may indicate an identifier or phrase match that still needs semantic checking. A strong result that disappears after a metadata condition is applied points to scope rather than ranking. A result whose highlight points to a common phrase but whose source document is the wrong product is a precision failure. These readings are diagnostic hypotheses, not formal confidence values, but they direct the next experiment more reliably than changing several settings at once.
RAGFlow’s different document engines implement these ideas through different storage primitives. The Elasticsearch path builds text and KNN queries and can recover vector scores in a second pass while keeping chunk vectors in the index. The Infinity path maintains per-dataset tables and supports vector indexes, full-text indexes, filtering, and fusion. These are implementation differences beneath a common retrieval contract. The reader does not need to memorize engine-specific calls, but a deployment owner does need to know which engine is active when investigating a score, a missing field, a filter behavior, or a performance problem.
A healthy tuning loop starts with the question, not with the slider. First decide what a correct result must contain: an exact identifier, a definition, a comparison set, a time-bounded record, a table row, or a section with its qualifying conditions. Then inspect whether the prepared chunks contain that material. If they do not, return to parsing or chunking. If they do, check dataset/document scope and metadata. Next adjust the lexical/vector balance and threshold to address the observed error. Only after the candidate pool contains the needed material should reranking be used to improve ordering. This sequence avoids asking a later stage to compensate for an earlier omission.
The same tests should include negative cases. A query for one model should not routinely return a near-identical model. A date filter should exclude records outside its intended range. An identifier with punctuation should remain findable through lexical matching. A paraphrased question should still reach a passage that uses different terminology. A disabled or stale chunk should not reappear because its embedding remains in the document engine. These are not abstract metrics; they are checks that the selected evidence has the right scope and relationship to the question.
Search quality finally has to be judged at the handoff to the answer model. RAGFlow’s retrieval results expose source documents, relevance information, and, when configured, metadata and highlights. That makes it possible to ask whether the top result is merely similar or actually useful. It also makes clear why a score should not be presented as confidence. The model receives a ranked context, not a certified conclusion. The next step is to follow that ranked context through its lifecycle: inspect the source, test the retrieval, revise the knowledge fragments, and determine whether a bad answer begins before or after retrieval.