The Flywheel That Can Learn the Wrong Lesson
This series examines the Adaptive Data Flywheel built around NVInfo AI, NVIDIA's internal enterprise knowledge assistant, and asks a more demanding question than whether a deployed model can be updated quickly: can it improve while remaining answerable to evidence outside its own outputs? That question supports a practical judgment. Readers should come away able to distinguish a production signal from ground truth, understand why the distinction matters for an AI agent that learns over time, and recognize the conditions under which a data flywheel improves knowledge rather than merely improving its confidence in itself.
The phrase “data flywheel” sounds almost self-explanatory. A system produces interactions; those interactions become data; the data improve the system; the improved system produces better interactions; and the cycle accelerates. In a narrow operational sense, that is exactly the opportunity. But a flywheel has no built-in sense of truth. It only turns whatever evidence enters the loop into more influence over the next turn. If the evidence is incomplete, misinterpreted, or generated by the model itself, the loop can become efficient at repeating a mistake.
Why an enterprise agent needs to keep learning
An enterprise knowledge assistant is not answering a fixed examination. Employees ask about policies, benefits, finance, information technology, holidays, and internal documents. Their wording changes. Acronyms acquire local meanings. New documents appear, old rules change, and the distribution of questions encountered in production is different from the distribution imagined during development. A system that performs well at launch can therefore become less useful without any dramatic failure in its underlying language model.
NVInfo AI was designed as a retrieval-augmented generation system with a mixture-of-experts structure. In broad terms, a router directs a question toward a domain specialist; the system then reformulates the question when needed, retrieves and reranks relevant documents, generates an answer, and supplies citations. That structure creates speed and specialization, but it also creates several places where an answer can go wrong before a language model writes the final paragraph. A question can reach the wrong specialist, lose its meaning during rephrasing, miss a relevant document, or retrieve the right material and still present it poorly.
The reported baseline makes the operational pressure visible without making it the whole story. Before the flywheel work, NVInfo AI averaged about 12 seconds per query and received a correctness score of 4.2 out of 5 on a regression dataset. The system served more than 30,000 employees; weekly usage was described as about 2,000 unique queries across 800 users. At that scale, waiting for a complete retraining program or a wholesale infrastructure redesign is not the only possible response to every recurring failure. The practical attraction of a flywheel is that it can target the component responsible for a problem.
The Adaptive Data Flywheel frames that work as a MAPE-K loop: Monitor, Analyze, Plan, and Execute, connected by a shared knowledge base. Monitoring gathers what happened. Analysis asks which part of the pipeline most plausibly caused it. Planning selects data and a focused intervention. Execution evaluates the change and introduces it progressively rather than treating deployment as the end of the process. The idea is not to make the entire agent continuously rewrite itself. It is to make improvement a controlled path from an observed problem to a tested change.
That distinction matters. A flywheel is valuable because it makes learning more frequent and more local. It is trustworthy only if the evidence used at each turn remains capable of disagreeing with the model.
A useful signal is not yet a fact
The most accessible feedback mechanism in NVInfo AI is also one of the least complete: a thumbs-up or thumbs-down. The system also looks at indirect signs such as a user repeating a query or abandoning a session. Together with traces of the route selected, the rephrased query, retrieval results, latency, prompts, citations, and guardrail checks, these signals can reveal where to investigate.
They cannot, by themselves, answer every question that the investigation needs to ask. A user who marks an answer down may know that it was unhelpful without knowing whether the router, retriever, rephraser, or answer generator caused the problem. Users may give little explanation, report dissatisfaction more often than satisfaction, or misunderstand the domain issue. A repeated query may mean the first answer was wrong, but it may also mean the user is exploring a difficult subject. A binary rating is therefore an observation about an interaction, not a complete label for the world.
The case's own data show why the distinction is important. During three months, the team analyzed 495 negative feedback samples. It identified 26 routing errors, reported as 5.25 percent of that sample, and estimated query-rephrasal errors at approximately 3.2 percent from examining 250 cases. Those numbers helped prioritize work, but they do not establish the prevalence of the same failures across every employee query. The denominator is a selected feedback sample, and the rephrasal figure is explicitly extrapolated from a reviewed subset.
Consider the kinds of mistakes the team found. “How many vacation days does NVIDIA Canada have?” was sent to a Holidays Expert rather than a Policies Expert, which led to an inability-to-answer response. “RESS planning team” was expanded as “NVIDIA Resource Planning team” rather than “Real Estate & Site Services,” so the system failed to retrieve the intended department information. In both cases, the visible symptom appears at the end of the interaction, but the decisive error occurs earlier. A negative rating tells the team that something deserves attention. The trace and domain knowledge are needed to say what.
This is the first meaning of independent ground truth in the series: an authority that does not simply inherit the model's interpretation of the event. It might be a source document, a policy with a known answer, a citation that a reviewer can inspect, or a subject-matter expert who can correct an interpretation. The authority need not be perfect or detached from all human judgment. It must, however, be external to the unverified output that is being improved.
The second loop hiding inside the first
The NVInfo case uses models to help analyze and expand evidence. An LLM-as-a-Judge identified candidate routing errors, and a small set of reviewed examples supported the generation of approximately 5,000 synthetic query-rephrasing examples. That is not automatically a flaw. Synthetic data can extend a carefully understood pattern, especially when real labels are scarce and sensitive enterprise interactions cannot be retained in full. But the role of those examples matters. They are useful candidates and augmentation, not independent proof that the model's interpretation is correct.
A different research paper gives this warning a name: model collapse. It studies what happens when generated data pollute the training set of the next model generation. Early in the process, low-probability events—the tails of the original distribution—begin to disappear. Later, distinct modes can become entangled and the learned distribution can converge toward a low-variance approximation with little resemblance to the original. This is not simply a model forgetting a stored example. Successive models are reinforcing a distorted account of what counts as likely or real.
The mechanism is easy to miss because every individual step can look reasonable. A model samples from the distribution it has learned. The next model fits the available samples. Finite sampling fails to represent some rare events, while the model's function may introduce additional approximation errors. The next generation then treats that altered sample as evidence. Repeating the procedure compounds small deviations. A larger generated dataset may contain more tokens or examples without containing more independent information about the world.
That finding does not mean that every synthetic example is poisonous or that every feedback loop will collapse on the same schedule. The model-collapse results concern recursive generational training under stated theoretical and experimental conditions. Their relevance here is more basic: if an improvement system lets model-generated material replace the external distribution that defines correctness, it can lose the ability to notice what has disappeared.
The diagram captures the central tension. The operational loop can be productive: production behavior leads to a diagnosis, a targeted change, and a staged update. Generated output can participate in that loop as an aid. The danger begins when the generated branch becomes the only route back to the next target. At that point, the system is no longer merely learning from users and documents; it is learning from its own prior approximation of users and documents.
Better metrics are not the same as durable truth
The NVInfo results show why the flywheel is attractive. For routing, a fine-tuned Llama 3.1 8B model reportedly matched the 70B baseline's 96 percent accuracy while reducing measured latency from 0.26 seconds to 0.08 seconds. For query rephrasal, the fine-tuned 8B model reportedly improved accuracy from 73.8 percent to 77.5 percent and reduced latency from 1.9 seconds to 1.1 seconds. The smaller model was not successful merely because it was smaller: the untuned 8B router scored 14 percent, while prompt tuning reached 86 percent. The comparison is consistent with targeted data and specialization driving the improvement.
Those are meaningful operational gains. They are also bounded gains. The measurements are component-level results from the case's evaluation setup. They do not, by themselves, establish end-to-end answer correctness, representativeness across all users and domains, or stability over a long sequence of future updates. A local metric can improve because the intervention solves the sampled problem. It can also improve because the evaluation set has stopped seeing the cases that matter.
This is why an external reference must survive the cycle rather than appear once during initial development. In the NVInfo workflow, source documents, SME-corrected completions, and a regression dataset with expected answers and citations provide forms of continuity. Human review remains costly, feedback participation is limited, privacy rules restrict what can be stored, and synthetic examples require validation. Those constraints make independent evidence harder to collect, not less necessary.
The model-collapse research reaches a related conclusion from another direction. In its language-model experiment, repeatedly fine-tuned models degraded on the original Wikitext2 distribution when no original data were preserved. Retaining 10 percent of the original training data at each generation produced only minor degradation in the reported setting. That condition is not a universal quota, but it illustrates the value of keeping contact with an external distribution. The question is not simply whether the next model is better than the previous one. It is whether both can still be judged against something the previous model did not define.
The question that follows
The title's warning is therefore not a case against adaptive systems. NVInfo demonstrates that a structured loop can turn sparse, messy production evidence into focused improvements in speed and measured performance. The warning is about authority. Models can help surface failures, propose labels, generate variations, and stress-test a change. They should not quietly become the sole judge of whether their own outputs are correct, representative, or safe to promote.
The rest of this series follows that boundary through the system itself. It begins with where routing, retrieval, generation, telemetry, and feedback meet, because an abstract promise of learning is not useful until the interfaces are visible. From there, the central test remains consistent: what does each artifact show, what does it merely suggest, and what independent evidence could prove the model wrong? A durable flywheel is not the one that turns fastest. It is the one that can keep turning without losing the reference that tells it where it is going.