What a Court Would Need to See Before Trial
The proposed Rule 707 makes reliability a proof obligation, but a proof obligation is meaningful only if the court and the opposing party can inspect the material behind the AI conclusion. This chapter turns the committee’s guidance into a practical evidence package: data that fit the case, validation that resembles the case, records that permit reconstruction, independent scrutiny, and an explanation of how the system works. It also examines why the draft’s notice requirement and the City Bar’s proposed discovery changes are part of the reliability problem rather than administrative details.
The question is not whether a proponent must reveal every line of code or every internal parameter. The sources do not establish such a universal disclosure rule. The question is whether the proponent can expose enough of the path from input to output for a judge to evaluate the proposed reliability gates and for an opponent to challenge them before the factfinder hears a persuasive expert-like conclusion.
Reliability Is a Chain of Evidence, Not a Vendor Assurance
The first link is the relationship between the system’s inputs and the population or purpose involved in the case. The committee note says a court should consider whether inputs are sufficient for ensuring the validity of the resulting output, including whether training data are sufficiently representative of the population involved. This is an applicability question. A system may perform well on the data used to develop or test it while producing a less dependable conclusion for a different population, a different kind of source material, or a different task.
Representation does not mean that every case must have an identical training example. It means that the proponent must be able to explain why the data support the use being made of the system. If the output concerns identity, the relevant comparison material and the conditions of the source may matter. If the output concerns a prediction, the population and time period may matter. If the output reconstructs an image or audio passage, the completeness and quality of the original may matter. The precise question changes with the inference, but the burden is the same: connect the input to the conclusion rather than treat data volume as a substitute for fit.
The second link is validation under conditions sufficiently similar to the case. “The system was tested” is not a complete reliability argument. Tested for what purpose, on what population, with what kind of input, and under what operating conditions? A process may have a low error rate in one setting and an unknown error rate in another. A model may be validated for one kind of classification but used to make a broader recommendation. A reconstruction method may perform acceptably on clean sources but behave differently when the source has gaps or distortion.
The official committee note does not prescribe a universal error threshold or a single validation design. That absence is important. It means the source supports asking whether validation is comparable, not announcing that a particular error rate automatically passes or fails.