When a Good Submission Stops Being Good Evidence
This series examines student assessment in learning environments where generative AI can draft an essay, solve a problem, or produce working code before an instructor ever sees the student think. The opening question is narrow but consequential: when the visible submission may have been produced with substantial machine assistance, what does the assessment actually tell us about the student? The answer determines whether a grade certifies knowledge and judgment or merely records that a person was able to obtain a convincing output.
The aim is not to decide whether AI is good or bad for education. It is to understand how assessment can remain credible when AI is available, and to give educators and institutions a way to judge the alternatives. That requires a distinction often blurred in public debate: performing a task with a powerful tool is not the same as learning the knowledge and skills that make the performance transferable. A useful redesign must preserve both realities. Students may need to work with AI, just as professionals increasingly do, while an assessment may still need to establish what the student can understand, explain, question, and do.
The product is no longer the whole observation
For much of modern education, the submitted product has served as a convenient proxy for the learner. A paper was treated as evidence that the student could research and argue. A program was treated as evidence that the student understood the underlying logic. A completed set of exercises suggested that the student had acquired the relevant procedures. Those inferences were never perfect, but the distance between making the work and submitting it was usually limited enough to be manageable.
Generative AI widens that distance. A student can now ask for a structure, a paragraph, a proof, a calculation, or a function and receive something that looks finished. The output may be correct, fluent, and better organised than the student’s unaided attempt. Yet the assessment may be trying to certify a different thing: whether the student can construct an argument, select an appropriate method, notice an error, explain a result, or transfer the idea to a new situation. The visible product and the intended human capability are no longer guaranteed to travel together.
The OECD Digital Education Outlook 2026 names the resulting danger a “mirage of false mastery”: AI-enabled performance can rise while human skill development lags behind. This is not a claim that every assisted submission is invalid. It is a warning about inference. If the task allows a system to perform the very reasoning the assessment is meant to observe, a high score may be evidence of successful task completion without being strong evidence of durable learning.
That distinction also changes what “academic integrity” means in practice. Integrity is not only a question of whether a student broke a rule. It is a question of whether the assessment remains a trustworthy measurement of its stated construct. A student who uses a calculator in a context where calculation is not the target may still demonstrate the intended mathematics. A student who submits AI-generated code in a programming-fundamentals assessment may produce a working program while bypassing the target skill. The same tool can therefore be legitimate in one assessment and disqualifying in another, because the claim being made about the student is different.
This is why the central problem is validity. An assessment is valid when its evidence supports the conclusion we want to draw from it. In an AI-rich environment, the conclusion must be made explicit before the format is chosen. Are we certifying the ability to use available tools responsibly? The ability to reason without them? The ability to evaluate and improve their suggestions? These are related capabilities, but they are not interchangeable. A credible assessment cannot leave that choice implicit and then treat the final output as if it answered all three questions.
The diagram captures the basic fork. An output-only design can still produce a grade, but the inference may quietly include technological assistance that the assessor did not intend to measure. Adding process, explanation, and a later check does not make the evidence infallible. It does make the reasoning behind the grade more visible: the assessor can ask whether the student’s choices, understanding, and independent capability support the claim.
From catching AI to seeing the learner
The most tempting response is to focus on detection. If the problem is machine-generated work, perhaps a detector can identify it; if the problem is unauthorised assistance, perhaps stricter browser controls or a more detailed policy can prevent it. These measures may have a place in a wider governance system, but they are fragile as the main foundation of assessment. The authors of the conversational-exam case describe this as an “enforcement illusion”: rules cannot reliably control what students do in unsupervised settings, and detection cannot reliably reconstruct authorship from a finished text or program.
The deeper weakness is that policing asks a backward-looking question—who or what produced this object?—when the educational question is forward-looking: what can this student now understand and do? Even a perfect detector would not solve the validity problem by itself. It could identify a suspicious submission without showing whether the student has learned. Conversely, a student might use AI in a permitted way and still fail to acquire the target capability. Authorship evidence and learning evidence are related, but they are not the same evidence.
A stronger response begins with authentic practice. The systematic review of assessment in AI-infused higher education recommends tasks involving real-world application, problem solving, reflection, oral defence, collaboration, and multiple stages of work. These designs make it harder to substitute a generic response for the whole performance, but their value is not merely defensive. They ask students to exercise judgment in situations where choosing a method, interpreting a result, responding to another person, or revising a decision is part of the competence itself.
Authenticity must nevertheless be handled carefully. Professional work often includes documentation, online resources, collaboration, and AI. Recreating an artificially tool-free workplace may measure obedience to an exam constraint rather than readiness for the work the course describes. But a realistic task completed entirely by an AI system is not automatically evidence of the student’s professional competence either. Authenticity has two requirements: the task should resemble a meaningful context, and the assessment should make the student’s contribution visible within that context.
That is where human judgment and process evidence enter. A student can be asked to explain why they selected an approach, identify a weakness in an AI suggestion, show how they verified a claim, revise a result after feedback, or apply the underlying idea to a new case. A staged assignment can preserve drafts, decisions, and reflections. A live performance can reveal how the student responds when the first attempt fails. These observations do not need to document every keystroke or every thought. They need to be chosen because they bear on the capability the assessment intends to certify.
The shift is therefore from “How do we prove that the student did not use AI?” to “What evidence would convince us that the student understands and can use this knowledge?” Sometimes the answer will include an unaided component. Sometimes it will include a tool-mediated performance followed by explanation and independent transfer. Sometimes the most appropriate evidence will be a sequence of drafts and decisions rather than a single examination. The format follows the construct, not the other way around.
An assessment that asks the student to show the work
The OECD report offers a useful conceptual route through this choice: Evidence-Centred Design. Its basic discipline is to connect a claim about a learner, a task designed to elicit that capability, the observations collected during the task, and the inference drawn from those observations. This prevents a common shortcut in assessment design—starting with whatever a platform can record and then treating the recorded activity as meaningful simply because it is plentiful.
For example, a final diagnosis in a clinical exercise is product evidence. The questions a student asks a virtual patient, the sequence of history taking, and the way they respond to information are process evidence. Interaction traces or reflections about monitoring one’s approach may provide metacognitive evidence. Together, these streams can support a richer inference than the diagnosis alone. The important point is not that more data is always better. It is that each observation has to be connected to a claim, interpreted by someone accountable for the judgment, and bounded by what the evidence can actually show.
This perspective makes the conversational exam a particularly revealing case. In the engineering-computations course where it was developed, students code live in a familiar notebook environment while explaining their reasoning. Questions are tied to learning objectives, and follow-up prompts can ask for a conceptual variation, offer a hint, or simplify the task when a student is stuck. The format does not attempt to make AI disappear. It changes the evidence: the student must perform, articulate, respond, and adapt in the presence of an examiner.
The case is valuable because it turns a broad principle into an observable design. It is not proof that oral assessment is always superior, or that real-time performance cannot be faked under any circumstances. It is a proof of concept showing how authenticity, performance-based validity, structured observation, and logistical preparation can work together. The reported experience involved 58 students in small groups across ten half-hour sessions, but the authors also describe unfamiliarity, perceived unfairness, and difficulty handling extra-time accommodations. Feasibility is therefore part of the evidence, not the whole conclusion.
The question that follows is more demanding than “Can an AI write the assignment?” It is: what happens to a learner’s thinking when the answer arrives before the learner has had to form a question, test an idea, or recover from an error? Until that mechanism is understood, a better-looking submission can be mistaken for better learning. The next step is to examine how direct-answer assistance creates that false confidence—and how an assessment can reveal the difference after the tool is removed.