The Four Reliability Questions the Proponent Must Answer
Once an AI system has produced an expert-like inference, the proposed Rule 707 turns the general word “reliable” into four separate questions. This chapter explains what the proponent would need to establish: that the output will help the factfinder, rests on sufficient facts or data, comes from reliable principles and methods, and reflects a reliable application of those principles to the case. The value of the framework is diagnostic. It lets a court identify where a conclusion may fail instead of treating a fluent or technically impressive answer as one undivided object.
The four requirements are drawn from Rule 702, the Federal Rule governing expert testimony. Under the draft, they apply when the AI evidence is offered without an expert but would be treated as expert testimony if a human witness supplied the same conclusion. The draft is not a four-part accuracy guarantee, and it is not an enacted rule. It is a proposed gatekeeping structure for preventing the presentation format from deciding the reliability question.
A Gatekeeping Chain Starts With Fit
The first question is whether the AI evidence will help the trier of fact. In a trial, the factfinder may be a jury or a judge. “Help” does not mean that the output is novel, fast, persuasive, or generated by a sophisticated system. It means that the evidence assists the factfinder with a relevant issue in the case. An AI result can be technically accurate and still fail this test if it answers a question no one must decide, supplies a conclusion that does not make a disputed fact more understandable, or adds confusion without a dependable connection to the issue.
Consider a hypothetical facial-recognition output offered to show that a person appears in a video. The first gate is not yet whether the match is correct. It asks what proposition the output is supposed to help establish. Is the identity disputed? Does the system’s conclusion address that dispute, or is the image already identified through uncontested evidence? Is the output being offered as a lead, as a description of similarity, or as a conclusion that the person is the individual shown? The same technical result may have different value depending on the proposition for which it is offered.
This gate protects against a common shortcut: equating automated production with relevance or usefulness. A machine can generate a large amount of information without narrowing the issue the court must resolve. It can also turn a tentative pattern into a confident-sounding answer that distracts from stronger evidence. The proposed framework therefore begins with the legal and factual use of the output, not with the system’s technical prestige.
The second question is whether the output rests on sufficient facts or data. This is a grounding inquiry. The court may need to ask whether the input material is complete enough for the claimed purpose, whether it represents the population or conditions involved, and whether the system received the facts it needed rather than a convenient but distorted subset. A method that works on one population may not support the same conclusion about another. A model tested on clear, well-labeled examples may behave differently when the case involves missing, noisy, or unusual inputs.
The proposed committee note makes training data part of this inquiry. For an AI process used to produce an output about a population, the court should consider whether the training data are sufficiently representative of the population involved in the case. That is not a demand that every dataset perfectly mirror every case. It is a demand for a reasoned connection between the data on which the system learned or was validated and the people, objects, or conditions about which it is now speaking.
Sufficiency is therefore purpose-dependent. A small collection of records might be enough to answer a narrow descriptive question but inadequate to support a broad prediction. A clean recording might support an extraction but not a confident reconstruction of what was missing. A system could receive the correct file but still lack the contextual information needed to interpret it. The court is not simply counting inputs. It is asking whether the inputs can support the output that the proponent wants the factfinder to accept.
The first two gates work together. Helpfulness asks whether the output bears on a question the factfinder must answer. Sufficient facts or data ask whether the output has a sound factual base for that use. A result can be relevant but under-grounded, or well-grounded but irrelevant to the disputed issue. Keeping the questions separate prevents a proponent from using relevance as a substitute for data quality or data volume as a substitute for legal usefulness.
A Reliable General Method Still Needs Reliable Application
The third question concerns the principles and methods that generated the output. Here the court moves from the case’s purpose and inputs to the process itself. What kind of method made the prediction, classification, reconstruction, or recommendation? Has it been validated? Are its assumptions appropriate to the type of conclusion being offered? Are its limitations and error patterns understood well enough to support the claim? The proposed note identifies analytical error or incompleteness, inaccurate or biased data or formulas, and lack of interpretability as concerns that can undermine reliability.
This is where the distinction between a method and a result becomes essential. A particular output may look plausible. That does not show that the process that produced it is dependable. Conversely, a method may have a strong general validation record while a particular output is wrong because the wrong version was used, the input was incomplete, or the system was applied outside its intended purpose. The third gate asks whether the general process deserves confidence; it does not decide whether this one output was produced correctly.
Validation is relevant because it supplies evidence about how a method behaves beyond the single result offered in court. The committee note anticipates examining whether the process was validated in circumstances sufficiently similar to the case. Similarity can concern the population, the type of input, the task, or the conditions under which the system operates. The sources do not establish a universal acceptable error rate or a fixed validation design. They support a more limited point: a generic claim that a system was tested may be less informative than evidence showing that it was tested for a use sufficiently like the one before the court.
The fourth question concerns application. Even a sound method must be applied reliably to the facts of the case. That inquiry may include how the input was prepared, which records were included or excluded, what settings or assumptions were used, whether the system’s output was reproduced, and how the final conclusion was connected to the case-specific material. The method may be reliable in general but misapplied in this instance. A tool designed for one kind of comparison may be used for another. A conclusion may be extended beyond what the system actually evaluated.
The difference can be expressed through a simple contrast. “This method has been validated for comparing clear samples under defined conditions” is a claim about principles and methods. “This case’s sample was processed under those conditions, with the relevant information preserved, and the result supports this conclusion” is a claim about application. The second does not follow automatically from the first. It needs its own explanation.
The gates also explain why a single favorable fact cannot carry the entire burden. Suppose a party offers an AI-generated risk assessment in a case where the risk itself is a disputed issue. The output may be helpful in the narrow sense that it addresses the issue. That says little about whether the records supplied to the system represent the person or population at issue. If the input is adequate, the court must still ask whether the method was validated for that kind of assessment. If the method was validated, the court must still ask whether this record was processed in the way the validation assumed. Each answer narrows a different uncertainty.
The same logic applies to a recognition or reconstruction output. A high score may show that the system found a strong pattern under its own comparison procedure. It does not by itself establish that the comparison population was appropriate, that the source material was complete, or that the score supports the precise proposition for which the party offers it. A validation study may show that a method performs well on similar material, yet the particular item may have been altered, degraded, or handled in a way that changes the result. General reliability and case-specific reliability are related but not interchangeable.
Nor does the framework require the judge to personally reproduce every internal computation. The proposal instead asks whether the proponent can establish a reliable path from purpose to inputs, from inputs to method, and from method to conclusion. That path may be described by an expert, supported by objective validation, or challenged through records and competing evidence. But the path must exist. A conclusion that cannot be connected to its source material, its method, or its application leaves the court unable to tell whether the apparent answer is evidence or merely an output in search of a reason.
This chain also gives the opponent a more precise challenge than saying that AI is mysterious. The opponent can ask whether the output addresses the disputed issue, whether relevant data were omitted, whether the validation conditions match the case, whether the system preserves enough information to audit the result, or whether the proponent has changed the conclusion while moving from a score to a legal proposition. A focused challenge may reveal a gap even when no one can prove that the system is always inaccurate. The proposed framework is about the sufficiency of this showing, not a universal verdict on the technology.
The four gates therefore form a chain with distinct failure paths:
The sequence is conceptual, not a claim that every hearing will follow these exact words or that the draft supplies a fixed checklist. Its purpose is to show why a failure at one gate cannot always be repaired by evidence at another. More data do not necessarily make an irrelevant output helpful. A validated method does not necessarily cure an unrepresentative case input. A strong general result does not automatically establish reliable application. The proponent’s explanation must connect the parts.
Why Presentation Form Cannot Evade the Questions
The proposed rule is especially important when the AI output arrives without a human expert. If a human witness offered an opinion based on a model, Rule 702 would provide a recognized route for examining the witness’s qualifications, method, data, and application. When the output is entered directly or presented by a lay operator, the same reliability issues do not disappear. The draft’s anti-evasion logic is that the court should look at what the system contributed, not merely at who clicked the button.
This does not mean that the technician is irrelevant. They may explain when the program was used, what information was entered, how the output was retrieved, or how the record was preserved. Those facts may matter. But operating knowledge and methodological knowledge answer different questions. A person can use a system correctly while lacking the expertise to say whether the system’s inference is valid. The proposed Rule 707 is aimed at the gap between those forms of knowledge.
The same reasoning limits attempts to rely on labels. Calling an analytical result a business record or a “mere computer output” may describe how the item was stored or offered, but it does not establish that the method generated a reliable inference. The official report says the proposed rule is meant to prevent a party from evading Rule 702’s reliability requirements by offering the AI output directly or through a lay witness. The concern is not that labels are always misleading. It is that a label should not decide a question that turns on the substance of the reasoning.
The report also links the reliability analysis to a preponderance standard in discussing Rule 702 and Rule 707, while acknowledging that the draft Rule 707 text does not separately spell out that burden. The Committee was considering a related amendment to Rule 104(a) that would expressly state that a proponent must establish a preliminary fact by a preponderance of the evidence, but that amendment too remained a proposal. This is a useful example of why the wording must be read with procedural status in mind: the architecture is clear enough to explain, while the final text and precise operation remain unsettled.
What is clear within the proposed architecture is the direction of responsibility. Once the output is treated as an expert-like inference, the offering party must be able to connect usefulness, factual grounding, method reliability, and case-specific application. The next question is not whether those gates exist, but who can credibly carry the explanation. The committee’s preferred draft ordinarily looks to an expert, while leaving a narrow opening for other proof when validation and objective evidence can do the necessary work.